Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

24/07/2026

Mediagenix To Feature 'Real-Time Media Enterprise Platform At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

24/07/2026

Peacock Posts First Profits, Adds 2 Million Subs in Q2 2026

Share Copy link Facebook X Linkedin Bluesky Email...

24/07/2026

Edward F. OKeefe Named President and CEO of The WNET Group

Share Copy link Facebook X Linkedin Bluesky Email...

24/07/2026

TiVo Ads, OpenGlass Launch Programmatic Access to the TV Homescreen

Share Copy link Facebook X Linkedin Bluesky Email...

24/07/2026

Digital Alert Systems DASDEC Platform Supports FCCs New P...

Current DASDEC software enables customers to comply with the FCC's new 15-character password mandate without requiring updates LYNDONVILLE, N.Y. July 23,...

24/07/2026

SDVI to Showcase Next-Generation Rally Platform and New T...

At the IBC2026 Show, SDVI will give European media organizations their first hands-on look at the next-generation Rally media supply chain management platform f...

24/07/2026

Sonnet Technologies at IBC2026

Sonnet Technologies at IBC2026 Brie Clayton July 23, 2026 0 Comments Hero image: Sonnet Twin25G T5 Thunderbolt 5 Adapter At this year's IBC Sho...

24/07/2026

Musora Enhances Online Music Lessons with Blackmagic Design

Musora Enhances Online Music Lessons with Blackmagic Design Brie Clayton July 23, 2026 0 Comments Director of Production Josh Holliday details workflo...

23/07/2026

Record-breaking World Cup has advertisers cheering

Staines-upon-Thames, UK, 23rd, JULY, 2026 - Yospace, the trusted leader in Dynamic Ad Insertion (DAI), stitched 11.57 billion one-to-one addressable advertisem...

23/07/2026

SVG Students To Watch: Shansley Glenn, University of Oklahoma

This Spring 2026 graduate from Honolulu found a home at SoonerVision as a live-production star and is ready to enter the industry as a replay op on Prime Video&...

23/07/2026

Miked Up Is an Everyday Deal for the Athletes Unlimited Softball League

As the AUSL postseason begins tonight, listening in to players during games enhances fan engagement for the young organization Ballplayers chatting with announ...

23/07/2026

Snap Announces AR and Creator Activations at Esports World Cup 2026 in Paris

Snap Inc. and the Esports Foundation have announced Snap's activations at the Esports World Cup 2026 in Paris, running July 6 through August 23. For the fir...

23/07/2026

Sony Introduces FX5 Full-Frame Cinema Line Camera

Sony has announced the FX5, a full-frame Cinema Line camera built around a newly developed fully stacked Exmor RS CMOS image sensor and BIONZ XR2 image processi...

23/07/2026

TNDV Powers Pickleball Slam 4 Broadcast for Ohlmeyer Kusserow Group

TNDV (a division of Live Media Group) provided technical production support for the Pickleball Slam 4 live broadcast from Seminole Hard Rock Live in Hollywood, ...

23/07/2026

LTN Surpasses 3,000 Satellite-to-IP Migrations as FCC Approves Upper C-Band Auction Rules

LTN has announced it has completed satellite-to-IP migrations for more than 3,00...

23/07/2026

Daktronics Announces 2026 DakClassroom Student Showcase Winners

Daktronics has announced the winners of the 2026 DakClassroom Student Showcase, a national competition recognizing student work in media production, storytellin...

23/07/2026

beIN Sports Extends Exclusive Roland-Garros Rights Across 10 Asian Markets Through 2031

The French Tennis Federation (FFT) and beIN Media Group have announced a five-ye...

23/07/2026

Clear-Com Provides Intercom System for Fryderyk Awards Gala at Tauron Arena Krakw

Clear-Com provided the communications infrastructure for the Fryderyk Awards Gal...

23/07/2026

How the FIFA World Cup's Record Revenue Masks a Structural Market Contraction

The 2026 FIFA World Cup is, on paper, an unparalleled financial success. Total t...

23/07/2026

From Green Screen to LED: How BBC Sport's World Cup Studio Became the Blueprint for Match of the Day

Standing inside HQ3 at MediaCity Studios during the FIFA World Cup 2026, it quic...

23/07/2026

How the 2026 FIFA World Cup Tightened the Bond Between the FOX Sports Digital, Linear Teams

Having attracted upwards of 17 billion views throughout the tourney, the broadca...

23/07/2026

Low End Candy release Preverb

Quickly creates reverse reverb effects Low End Candy have recently released a new plug-in that offers a quick and simple way to create reverse reverb effect...

23/07/2026

Expressive E Noisy 2.6 arrives

New engines, resonators, effects & more Expressive E's versatile soft synth has just been treated to an update that greatly expands its sonic range. Noi...

23/07/2026

SGL Carbon expands photovoltaic capacity at its Ort im Innkreis site

SGL Carbon has further expanded its photovoltaic capacity at its Ort im Innkreis site in Austria. With the commissioning of new PV carports, the sites solar sys...

23/07/2026

NITV launches new By Mob For Mob series in partnership with NRMA Insurance

NITV launches new By Mob For Mob series in partnership with NRMA Insurance 23 July, 2026 Media releases National Indigenous Television (NITV) has launched ...

23/07/2026

Rohde & Schwarz presents new software for three-phase power analysis on MXO series oscilloscopes

Rohde & Schwarz presents new software for three-phase power analysis on MXO seri...

23/07/2026

Why The Next Big Opportunity in Sports Lies Outside the Stadium

In our latest blog, Laura Rognoni explores how the launch of Play Snooker reflects a wider shift in sports economics - from monetising peak broadcast audiences ...

23/07/2026

Scaling Smart with Vivid Broadcast

Download Now! To receive a short case study showcasing Vivid Broadcast's audio setup, please enter the following details: -- First Name* First Name ...

23/07/2026

Sony Unveils FX5 Full-Frame Camera

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

Sachtler To Show Akti8T Fluid Head, Flowtech75 Tripod At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

NextGen TV Launches in Cleveland

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

Yospace Stitched 11.57 Billion Ads into Live Streams During World Cup

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

At IBC2026 Witbe Will Showcase Tests That Build Themselve...

Witbe brings its AI-native testing and monitoring infrastructure to Amsterdam: demonstrations on real devices, a peer-reviewed technical paper in the conference...

23/07/2026

Quickplay Caretta Research Evergent and Yospace Announce...

Quickplay, Caretta Research, Evergent, and Yospace have announced that the renowned IBC Pride Happy Hour will once again bring the vibe and fun back to IBC on F...

23/07/2026

IBC2026 Interra Systems Brings Trusted Media Intelligence...

At IBC2026, Interra Systems will present this year's focus: operational AI at scale for QC, monitoring, and captioning workflows. The industry has moved pas...

23/07/2026

Illusionfx is finally coming!

Illusionfx is finally coming! Alex Perea July 23, 2026 0 Comments The new website is now live to mark the occasion. Since the last update, feedback h...

23/07/2026

Evergent showcases AI agents for streaming retention at I...

Evergent (5.A86) will showcase its Agentic Revenue Orchestration Platform for streaming services, sports leagues, pay-TV operators at IBC2026 (RAI Amsterdam, 11...

23/07/2026

Brian Jones Named NCTCs New CFO

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

Scripps Unites TV Operations Under Dean Littleton's Leadership

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

2D House is First North American Rental Facility to Offer...

David Dvir, 2D founder (left), Sundeep Reddy (center), and Jean-Marc Bouchut (right), both of Zeiss...

23/07/2026

Screen Australia announces new national documentary accelerator program

Screen Australia announces new national documentary accelerator program 23 July 2026 Screen Australia has today announced the Above the Line Documentary Accel...

23/07/2026

At AI Summit, South Korea Outlines Its AI Future With NVIDIA and Partners

At this week's AI Summit in San Francisco, South Korean President Jae Myung Lee and some of the country's top business leaders and researchers are meeti...

23/07/2026

X-Rite Pantone and Fashion Institute of Technology (FIT) Partner to Advance Digital Color Education

X-Rite Pantone and Fashion Institute of Technology (FIT) Partner to Advance Digi...

23/07/2026

New Samsung Galaxy Foldable range available now to pre-order on Sky Mobile

Thursday 23 July 2026 New Samsung Galaxy Foldable range available now to pre-order on Sky Mobile Sky Mobile, the award-winning network voted this year by Expe...

23/07/2026

Jason Statham leads Sky Original action thriller MUTINY in explosive new trailer

Thursday 23 July 2026 Jason Statham leads Sky Original action thriller MUTINY in explosive new trailerTurn on cookies to view this content. Go to Privacy optio...

23/07/2026

Riedel Implements RefCam and Easy5G for the First Time at an International Soccer Match

Wuppertal July 23, 2026 Riedel Implements RefCam and Easy5G for the First Time...