Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

07/04/2026

Avid to Debut Avid Content Core on AWS at 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

07/04/2026

Frequency Launches Smarter Ways to Operate Streaming Chan...

Frequency, the engine behind many of the world's leading streaming television channels, at NAB 2026 will be launching new Studio services to help content ow...

07/04/2026

Ikegami to Introduce Expanded Range of Broadcast Producti...

Ikegami USA has chosen NAB 2026 in Las Vegas as the launch platform for new additions to its range of broadcast-quality television production equipment. These w...

07/04/2026

Kiloview Advancing Broadcast IP Workflows with a Smarter...

April 19, 2026, Las Vegas Kiloview, an innovative provider of AV-over-IP technologies, will showcase its latest broadcast IP solutions at NAB 2026, presenting...

07/04/2026

Bitmovin Adds Support for SGAI in its Playback Products t...

Bitmovin has announced support for Server-Guided Ad Insertion (SGAI) across its playback products using HLS interstitials, enabling more advanced ad-supported s...

07/04/2026

Synamedia unveils AI by Quortex - a just-in-time AI-plugi...

Synamedia is unveiling AI by Quortex at The NAB show, a just-in-time AI plugin framework that applies intelligence only when needed across video processing, dis...

07/04/2026

Cuez Brings Four New Innovations to NAB 2026 From Story-C...

Cuez will showcase four additions to its cloud-based newsroom, rundown and automation platform at NAB Show 2026 (April 18 22, Las Vegas, Booth N1867): Cuez ...

07/04/2026

Barix Extends Transport Options for Multi-Engine IP Encoder

Barix Extends Transport Options for Multi-Engine IP Encoder Brie Clayton April 7, 2026 0 Comments New for NAB, Barix adds SRT and RIST support to Mult...

07/04/2026

Elite Media Technologies Selects Interra Systems' BATON File-Based QC Solution

Elite Media Technologies Selects Interra Systems' BATON File-Based QC Soluti...

07/04/2026

Tightrope Media Systems to Debut Cablecast LiveBridge for Simultaneous Streaming at NAB 2026

Tightrope Media Systems to Debut Cablecast LiveBridge for Simultaneous Streaming...

07/04/2026

Cuez Brings Four New Innovations to NAB 2026: From Story-Centric Newsroom to Open AI Agent Framework

Cuez Brings Four New Innovations to NAB 2026: From Story-Centric Newsroom to Ope...

07/04/2026

ASG Names Andrea Cummis VP of Systems Engineering

Share Copy link Facebook X Linkedin Bluesky Email...

07/04/2026

KTVJ Completes Major Signal Upgrade

Share Copy link Facebook X Linkedin Bluesky Email...

07/04/2026

Hearst's WDSU to Air Million Dollar Rodeo Competition

Share Copy link Facebook X Linkedin Bluesky Email...

07/04/2026

Grass Valley Launches Future Playmakers Program

Share Copy link Facebook X Linkedin Bluesky Email...

07/04/2026

Saranyu Technologies Launches MATCH - Multi-View Sports S...

Designed for synchronized multi-stream playback, low-latency delivery, and real-time analytics, MATCH introduces a unified viewing experience for sports broadca...

07/04/2026

Berklee Students to Honor George Martin with Performance of Original Scores

Berklee Students to Honor George Martin with Performance of Original Scores The orchestra, led by associate professor Xander Rovang, will perform several work...

07/04/2026

The GAA Championships on RT

THE GAA CHAMPIONSHIPS ARE BACK WITH A BANG LIVE AND FREE TO AIR ACROSS ALL RT PLATFORMS The Sunday Game and The Sunday Game Live return on Sunday 12 April L...

06/04/2026

Fab Five' Reunion Drives TNT and CBS's Experimental Final Four Altcast Built on REMI Workflow

Michigan legends bring a new voice to the broadcast as TNT Sports and CBS Sports...

06/04/2026

SVG New Sponsor Spotlight: Optikka CEO Daniel Evans on Scaling Sports Content with Programmatic Graphics

From high school sports all the way up to the major leagues, building high-quali...

06/04/2026

Quickplay and TwelveLabs Join AWS Business Outcomes Xcelerator Program

Quickplay, an AI company for the media and entertainment industry, has been accepted into the Advanced tier of the TwelveLabs Ecosystem Partner Program. Quickpl...

06/04/2026

Grass Valley Launches Future Playmakers Program for Students in Sports Production and Media Technology

Grass Valley has announced the Future Playmakers Program, a global initiative to...

06/04/2026

SVG All-Stars: Raasean Robinson, Gerente de Posproduccin y Operaciones de Estudio, FOX Deportes

El l der de operaciones impulsa la producci n en estudio mientras encuentra insp...

06/04/2026

SVG All-Stars: Raasean Robinson, Manager, Post Production and Studio Operations, FOX Deportes

The ops leader helps lead the charge in studio for the Spanish-language broadcas...

06/04/2026

Behind The Mic: SiriusXM Shares 2026 Masters Broadcast Team; ESPN to Produce Over 140+ Hours of Masters Live Coverage

Behind The Mic provides a roundup of recent news regarding on-air talent, includ...

06/04/2026

NHL Opens Innovation Lab in Partnership with Verizon, New Jersey Devils

The National Hockey League (NHL), in partnership with Verizon and the New Jersey Devils, today announced the opening of the NHL Innovation Lab powered by Verizo...

06/04/2026

ESPN+ To Stream Inaugural Rock League Curling Season

Rock League, a new professional curling league, has announced that ESPN+ will stream its inaugural 2026 season for fans in the United States. The first Rock Lea...

06/04/2026

ASG Appoints Andrea Cummis as VP of Systems Design and Engineering

Advanced Systems Group has announced the appointment of Andrea (Andy) Cummis as Vice President of Systems Design and Engineering. In this role, she will lead de...

06/04/2026

Source Media Group Launches Source Golf, a Creator-Driven YouTube Network Targeting Next-Gen Fans

Backed by Bolt Ventures, the venture brings Bryson DeChambeau, Grant Horvat, and...

06/04/2026

How the NHL's Innovation Lab Will Take Broadcast, Fan, and Team Tech to New Heights

With this environment we can start that collaboration even earlier because we ca...

06/04/2026

K-Pop Artist ENHYPEN Host The Blood Diary,' a New Video Podcast Series From HYBE

Like the immortal lives of vampires, some stories never really end. That's t...

06/04/2026

From Audio to IRL: How Let's Get Haunted' Is Building Community With Spotify RADAR

As podcasting continues to evolve, growth increasingly means building beyond aud...

06/04/2026

FSK Audio update Bark24 Dyn

Multiband dynamics plug-in enhanced California-based developer FSK Audio have released a significant update for their innovative multiband dynamics processo...

06/04/2026

IK Multimedia introduce ToneNET Preset Sharing

Share official & user-created full-rig presets IK Multimedia's latest TONEX update makes it possible for users of the popular amp and effects modelling ...

06/04/2026

Baseball 2026: More AI, Better Viewing Choices

Share Copy link Facebook X Linkedin Bluesky Email...

06/04/2026

JB&A Announces Details for its Pre-NAB 2026 Event

Share Copy link Facebook X Linkedin Bluesky Email...

06/04/2026

Dalet Showcases Dalia Agentic AI and End-to-End Media Workflows at NAB Show 2026

Dalet Showcases Dalia Agentic AI and End-to-End Media Workflows at NAB Show 2026 Brie Clayton April 6, 2026 0 Comments Dalet, a leading technology and...

06/04/2026

OpenDrives Shows Off Sports Expertise in Sports Business Hub located in NAB Show's West Hall

OpenDrives Shows Off Sports Expertise in Sports Business Hub located in NAB Show...

06/04/2026

Proton to Demonstrate 3D Application at NAB 2026

Proton to Demonstrate 3D Application at NAB 2026 Brie Clayton April 6, 2026 0 Comments Yet further creative potential unleashed through innovation in ...

06/04/2026

Autoscript Highlights Voice-Driven Prompting and PTZ Solutions at NAB 2026

Autoscript Highlights Voice-Driven Prompting and PTZ Solutions at NAB 2026 Brie Clayton April 6, 2026 0 Comments Experience Autoscript Voice, PTZ prom...

06/04/2026

Mediaproxy Highlights Significant Enhancements to its LogServer suite at NAB Show 2026

Mediaproxy Highlights Significant Enhancements to its LogServer suite at NAB Sho...

06/04/2026

Re-Architectured PCC Software Streamlines and Enhances the Full High-Speed Imaging Workflow

Wayne, N.J., April 6th, 2026 Phantom High-Speed announces the release of PCC 4...

06/04/2026

Tribeca Studios And Lilly Announce Winners Of Inaugural Vital Stories Filmmaker Program

April 6th, 2026 TRIBECA STUDIOS AND LILLY ANNOUNCE WINNERS OF INAUGURAL VITAL...

06/04/2026

Netflix Expands Kids Entertainment Lineup With Playground App for Games, New Shows & Returning Favorites

Back to All News Netflix Expands Kids Entertainment Lineup With Playground App ...