Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

20/11/2026

ESPN's Jimmy Pitaro on Super Bowl LXI: A Yearlong Story That Features a Game'

The sports broadcaster's chairman sees the upcoming game as an opportunity t...

20/11/2026

SVG GameDay, Ep. 40: New York Mets Brendan McKeon - Firing Up Fans in Flushing

In-venue and creative video staffers at the professional and collegiate level have one major thing in common: the intensity and attention to detail ramps up dur...

20/11/2026

Back at Belmont Park, Part 1: Historic Venue's Renovation Puts a Premium on the Fan Experience

A variety of LED displays, integrated broadcast infrastructure covers every angl...

20/11/2026

RAF Partners with Migu Video to Bring RAF15 Main Event to China

Real American Freestyle (RAF) has announced a distribution partnership with Migu Video, a Chinese digital sports platform, to carry the RAF15 main event between...

20/11/2026

Apple TV Becomes U.S. Home of F1 Academy Beginning in 2027

Apple and F1 Academy have announced that the all-female racing series will stream on Apple TV in the U.S. beginning in 2027. Fans will be able to watch every qu...

20/11/2026

mnchen.tv Relies on Riedel Technology for New Production Center

Regional broadcaster m nchen.tv has modernized its production infrastructure at a new location in northern Munich. Riedel MediorNet provides decentralized signa...

20/11/2026

Eutelsat Announces Three Executive Committee Appointments

Eutelsat has announced three appointments to its Executive Committee, effective November 2026, under CEO Jean-Fran ois Fallacher. Jean-Hubert Lenotte is appoin...

20/11/2026

Seven Melbourne Selects RTS and Magna Systems for Comms Upgrade at New Facility

After more than two decades at Harbour Esplanade in Docklands, the Seven Network has relocated its Melbourne headquarters to a new, custom-built ST 2110 facilit...

20/11/2026

Brazil's TV 3.0 Rollout Shows VVC Moving from Standards Work to Commercial Deployment

Brazil is among the first markets deploying Versatile Video Coding (VVC) as part...

20/11/2026

NBC Sports and The Soccer Tournament Agree to Two-Year Extension for TST 2027 and 2028

The Soccer Tournament (TST), the $1 million winner-take-all 7v7 world championsh...

20/11/2026

Back at Belmont Park, Part 2: NYRA Reaps Benefits of Reimagined Track With Expanded Broadcast Capabilities

Enhancements include a larger compound, an elevated camera count, and a whole lo...

20/11/2026

With Women's Sports Programming Skyrocketing, Content and Production Strategy Are Evolving

Leaders from ESPN, iHeartMedia, and Just Women's Sports explore creating new...

20/11/2026

Lemur Audio release Dorothe

New EQ plug-in aimed at mastering duties Lemur Audio's latest plug-in has just arrived, delivering a characterful EQ that's aimed primarily at maste...

20/11/2026

Soundtoys Black Friday Sale

Discounts of up to 75% now available Soundtoys' annual Black Friday sale is now live, with discounts of up to 75% applied across the company's range...

20/11/2026

SoundGhost release Fragment

New creative looping tool revealed Described as a cycling audio fragmenter , the latest addition to the SoundGhost range draws its inspiration from tape ph...

20/11/2026

Heritage Audio's Herchild 670 goes virtual

Fairchild recreation gets plug-in counterpart Heritage Audio's Herchild 670 offers a meticulous hardware recreations of the much-loved Fairchild 670 and...

20/11/2026

BONDI: 7 Minutes That Changed Australia

BONDI: 7 Minutes That Changed Australia 16 November, 2026 Media releases A landmark documentary chronicling the worst terror attack in modern Australian hi...

20/11/2026

Statement regarding SBS non-Executive Director Aaron Fa'Aoso

Statement regarding SBS non-Executive Director Aaron Fa'Aoso 20 November, 2026 Media releases Aaron Fa'Aoso has advised that he is stepping down a...

20/11/2026

PUBLIC NOTICE FROM THE DEPARTMENT OF SPORT, ARTS AND CULTURE

INVITATION FOR PUBLIC COMMENTS AND OBJECTIONS ON NOMINEES FOR APPOINTMENT TO THE COUNCIL OF THE NATIONAL FILM AND VIDEO FOUNDATION (NFVF) The Department of Spor...

20/11/2026

TVP Claims Top Spot in Poland's Content Distributor Rankings

UEFA Nations League football and autumn premieres propel TVP to number one as YouTube grows its TV screen share Warsaw, Poland, November 19, 2026 - Nielsen, a...

20/11/2026

The Gauge: Poland | October 2026

October's autumn weather clearly encouraged Poles to spend more time in front of their screens, resulting in both longer viewing times and a higher overall ...

20/11/2026

FIFA Finds Few U.S. Networks Interested in 2030 World Cup Broadcast Rights

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

20/11/2026

DigitalGlue releases creative.space Pulse, a free Mac app that shows creatives what is slowing them down

DigitalGlue releases creative.space Pulse, a free Mac app that shows creatives w...

20/11/2026

How NBCUniversal Telemundo Used Virtual Production for its FIFA World Cup 2026 Coverage

How NBCUniversal Telemundo Used Virtual Production for its FIFA World Cup 2026 C...

20/11/2026

What Film School Pays and What it Costs: The Federal Numbers

What Film School Pays and What it Costs: The Federal Numbers Mark Santos November 20, 2026 0 Comments Every editor has had the argument about film sch...

20/11/2026

3D Sound Specialist Turns to Nugen Halo Upmix

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

20/11/2026

World's First Hybrid Film Production Platform Debuts

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

20/11/2026

NBC Adds New Stats Feature for Premier League

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

20/11/2026

Hearst TV Names Shawnda Adams to Lead KOCO-TV Oklahoma City

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

20/11/2026

FCC Modernizes Drone Communications, Detection Rules

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

20/11/2026

OBSBOT Unveils 2 New PTZR Cameras

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

20/11/2026

New DigitalGlue Mac App Identifies Media Production Hurdles

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

20/11/2026

Beam Dynamics Introduces MCP for Faster, Smarter Technolo...

Beam Dynamics has introduced new Model Context Protocol (MCP) support, expanding how customers can work with the live operational data held in Beam. The new cap...

20/11/2026

Re:Vision Effects Cyber Week Sale Nov 24 to Nov 30

Re:Vision Effects Cyber Week Sale Nov 24 to Nov 30 Pierre Jasmin November 19, 2026 0 Comments Long time friends of the COW are up with a big sale! Us...

20/11/2026

Avid is offering up to 33% off annual subscriptions!

Avid is offering up to 33% off annual subscriptions! Brie Clayton November 19, 2026 0 Comments Hero image courtesy Deposit Photos Pro Tools Pro To...

19/11/2026

Live From the UFL Draft: League's First Live Draft Broadcast Is Set To Innovate, Bring Viewers Close to the Action

With a sports-bar vibe and all the teams onsite in a huge atrium in Ballpark Vil...

19/11/2026

SVG Students To Watch: Niamh Campbell, Seton Hall University

The senior from Phoenix and England is balancing life as a Division I tennis player with a growing career behind the microphone The future of live sports produ...

19/11/2026

RED Connect Adds Native Colorfront Color Management for Live RED Camera Feeds

RED Digital Cinema and Colorfront have partnered to integrate native Colorfront color management into RED Connect, letting productions apply Colorfront's co...

19/11/2026

SMPTE Publishes Engineering Report on Content Provenance and Authenticity

SMPTE has published an engineering report summarizing an industry survey by its Study Group on Content Provenance and Authenticity in Media. The findings point ...

19/11/2026

CTech and Skyline Communications Partner on Integrated SATCOM Operations for Defense

CTech and Skyline Communications have announced a partnership to deliver integra...

19/11/2026

EITB Deploys SMPTE ST 2110 Infrastructure with Lawo HOME mc DSP Processing

Euskal Irrati Telebista (EITB), the Basque Country's public media group, has completed Phase 1 of a five-studio renewal with the commissioning of a Lawo IP ...

19/11/2026

Esports Foundation Expands Road to Esports Nations Cup 2027 with New Qualifiers, Larger Rosters and National Rankings

The Esports Foundation (EF) has announced an expanded Road to ENC 2027 for the E...

19/11/2026

AWSN Holds First Women's Sports Awards Show, VEON: Women Who Win, in New York City

All Women's Sports Network (AWSN) held its first awards show, VEON: Women W...

19/11/2026

Gracenote Extends DIRECTV Partnership for Sports Hubs

Gracenote, the content intelligence business unit of Nielsen, has extended its relationship with DIRECTV. Gracenote will continue to supply visual imagery and p...

19/11/2026

AES Technical Committee Expands Loudness Website with Dialogue Intelligibility Content

The Audio Engineering Society (AES) Technical Committee on Broadcast and Online ...

19/11/2026

NDI 6.4 Adds Recording and Playback APIs, HEVC 4:4:4 and Wi-Fi Receiver Buffer

NDI has released NDI 6.4, which adds Recording and Playback APIs, support for HEVC 4:4:4 and a configurable receiver buffer for Wi-Fi. The update also includes ...

19/11/2026

Lemonlight Announces Hero, an AI Production Operating System for Video

Lemonlight, the video production company behind more than 50,000 brand videos, has announced Hero, an AI production operating system, along with a new AI agent ...

19/11/2026

FanConnect Brings JPEG XS Beyond the Control Room, Cuts In-Venue IPTV Latency 89% for University of Maryland Athletics

In a first-of-its-kind deployment at XFINITY Center, the digital signage/display...

19/11/2026

NBCU Intros Peacock Performance View for Premier League Coverage

For the first time on a US streaming service, real-time player identification, contextual stats, and visual insights appear right in the live stream, available ...