Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

North America Stories

14/08/2026

LiveU, Panasonic LUMIX Launch 'Live Anywhere' Production Workflow

Share Copy link Facebook X Linkedin Bluesky Email...

14/08/2026

PBS Station Takes Iron Mountain to Court Over Archives

Share Copy link Facebook X Linkedin Bluesky Email...

14/08/2026

TV Globo Chooses Triveni Digital to Support TV 3 0 Footba...

Leading Brazilian Broadcaster Will Use Triveni Digital's TV 3.0 Broadcast Chain and Service Quality Assurance Solutions to Support Next-Generation Televisio...

14/08/2026

Beam Dynamics Makes IBC Debut with Major Expansion into L...

Beam Dynamics will make its IBC debut this September, marking a major step in the company's growth. Beam helps broadcast, live production and professional A...

14/08/2026

Conservative Groups Urge FCC to Stay Out of Affiliate Deals

Share Copy link Facebook X Linkedin Bluesky Email...

14/08/2026

LiveU and Panasonic LUMIX Launch 'Live Anywhere' Production Workflow

Share Copy link Facebook X Linkedin Bluesky Email...

14/08/2026

Cineverse Launches New Tools to Repurpose Audio Ads for CTV

Share Copy link Facebook X Linkedin Bluesky Email...

14/08/2026

Hearst Television Ups Derek Schnell to VP, News

Share Copy link Facebook X Linkedin Bluesky Email...

14/08/2026

amaran Makes Its Global Debut in Audio With the Launch of...

amaran, pioneer of lighting tools for creators, today announced the amaran Mic, a 99 wireless desktop microphone that brings the brand into audio for creators ...

14/08/2026

Creative COW Roundtable: Introduction to the New After Effects Assistant - AEAI

Creative COW Roundtable: Introduction to the New After Effects Assistant - AEAI Brie Clayton August 13, 2026 0 Comments On August 23, 2026 at 9am PDT/...

14/08/2026

Life on Waves Posted with DaVinci Resolve Studio

Life on Waves Posted with DaVinci Resolve Studio Brie Clayton August 13, 2026 0 Comments Collaborative editing and color grading workflow supported by...

14/08/2026

Why Bulletproof Video Project Archiving Is Non-Negotiable

Why Bulletproof Video Project Archiving Is Non-Negotiable John Carboni August 13, 2026 0 Comments In video production, the work is far from over when ...

14/08/2026

Stacked with Boston Conservatory Talent, Shakespeare on the Common Celebrates Its 30th Anniversary

Stacked with Boston Conservatory Talent, Shakespeare on the Common Celebrates It...

13/08/2026

LaLiga Renews Broadcast Agreements with ESPN and DSports in South America Through 2031/32

LaLiga has renewed broadcast agreements with ESPN and DSports covering Spanish-l...

13/08/2026

Rockstar Energy Open Builds Third Portland Show Around AI-Powered Vertical Feed, Remote Operators

Three days of skateboarding competitions, live music, and other programming will...

13/08/2026

IBC 2026: Matrox Video to Showcase IP and Software-Defined Technologies

Matrox Video will showcase its portfolio of IP and software-defined technologies at IBC2026 (Booth 7.B15, RAI Amsterdam, September 11-14). The portfolio includ...

13/08/2026

IBC 2026: SMPTE to Present Sessions, Workshops, and Networking Events

SMPTE will present expert-led sessions, professional workshops, and networking events at IBC2026. The organization will be located in the IBC2026 Owner Pavilion...

13/08/2026

IBC 2026: Clear-Com to Showcase Communications Solutions

Clear-Com will exhibit at IBC2026 (Stand 10.D29, September 11-14, Amsterdam), showcasing FreeSpeak Cell, Avalon IP Station, updates to Arcadia Central Station, ...

13/08/2026

Advanced Systems Group Appoints Eran Binyamin Zeitoun as Senior Technical Program Manager

Advanced Systems Group (ASG) has appointed Eran Binyamin Zeitoun as Senior Techn...

13/08/2026

2026 SVG Venues & Teams Summit Heats Up Kaseya Center in Miami

Miami HEAT, Florida Panthers, Orlando Magic, University of Miami join leading vendors for a day of gameday-production talk in South Florida The 2026 SVG Venues...

13/08/2026

MLB at Field of Dreams Returns as Netflix Joins the Effort, Leans Into Cinematic Look at Newly Permanent Ballpark

NBC Sports is producing the Twins-Phillies game alongside the streamer, which is...

13/08/2026

IBC 2026: Oracle to Showcase Cloud Media Workflows

Oracle Cloud Infrastructure (OCI) will exhibit at IBC2026 (Hall 5, Booth 5.A21, September 11-14, Amsterdam), demonstrating cloud-based workflows across low-late...

13/08/2026

IBC 2026: Dalet to Participate Accelerator Programme with AI Live Sports Workflow Demo

Dalet will participate in the IBC2026 Accelerator Media Innovation Programme, co...

13/08/2026

Cox Media Group Announces Local Broadcast Deals with Orlando Magic and Charlotte Hornets

Cox Media Group (CMG) has announced multi-year local broadcast rights agreements...

13/08/2026

Jnger Audio to Demonstrate Cloud-Based DTV+ Workflow at SET Expo 2026

J nger Audio will join Fraunhofer IIS and industry partners to showcase a cloud-based, DTV+-compliant workflow at SET Expo 2026 in S o Paulo, August 17-20. DTV+...

13/08/2026

IBC 2026: Olympic Broadcasting Services to Receive International Honour for Excellence

IBC has announced that Olympic Broadcasting Services (OBS) will receive the Inte...

13/08/2026

NESN Expands Red Sox Broadcast Distribution for Jimmy Fund Telethon

NESN has partnered with WSBK-TV38 to broadcast Boston Red Sox games on August 17 and 18 over the air, extending the reach of the 24th annual WEEI/NESN Jimmy Fun...

13/08/2026

LiveU and Panasonic LUMIX Launch Live Anywhere Mobile Production Workflow

LiveU and Panasonic LUMIX have announced the Live Anywhere workflow, demonstrated at NAB 2026 and InfoComm 2026 in collaboration with Social180Group. The work...

13/08/2026

IBC 2026: Chyron to Showcase Sports Workflow Solutions

Chyron will demonstrate its sports analysis, graphics, remote production, virtual production, and venue control solutions at IBC2026 (Booth 7.A23). This year...

13/08/2026

Ratings Roundup: Pro Football Hall of Fame Game Draws 7 Million During Quiet August for Sports

Ratings Roundup is a rundown of recent ratings news and is derived from press re...

13/08/2026

ESPN/ABC, NBC/Peacock, Prime Video Set National Slates for Year 2 of NBA Rights Deal

Season tips off Oct. 20 and runs through April 11; Christmas Day, NBA Cup, All-S...

13/08/2026

Tubi Media Group Partners with Gracenote to Unlock Additional Value in Content Discovery and Programmatic CTV Advertising

New Agreement Aims to Leverage Gracenote Infrastructure to Create More Relevant ...

13/08/2026

CMG Expands Local Sports Offering With Two NBA Deals

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

Olympic Broadcasting Services to receive IBC2026 Internat...

IBC today announced that its most prestigious honour, the International Honour for Excellence (IHFE), will be presented to Olympic Broadcasting Services (OBS), ...

13/08/2026

Dalet Demonstrates the Future of AI-Orchestrated Live Spo...

Dalet, a leading technology and service provider for media-rich organizations, today announced its participation in the IBC2026 Accelerator Media Innovation Pro...

13/08/2026

dzjinius to showcase new dzjinius on-set application at I...

dzjinius, the all-in-one production management platform for the media and entertainment industry, will showcase its new application, dzjinius on-set , at IBC20...

13/08/2026

BCE Demonstrates Sovereign Cloud Media Workflows powered...

Broadcasting Center Europe (BCE), a leading European provider of media services, system integration and product development, will demonstrate an end-to-end sove...

13/08/2026

Clear-Com to Showcase Communications Solutions for Broadc...

Clear-Com is returning to IBC 2026, taking place September 11-14 in Amsterdam, Netherlands. At Stand 10.D29, Clear-Com will spotlight the award-winning FreeSpe...

13/08/2026

Leader to highlight advanced Test and Measurement solutio...

Test & measurement innovator, Leader Electronics of Europe, will showcase its comprehensive portfolio of Leader, PHABRIX and LeaderPhabrix test and measurement ...

13/08/2026

KRK Brings Advanced Wireless Control to Professional Monitoring With New V Series Five Studio MonitorsKRK Brings Advanced Wireless

KRK Brings Advanced Wireless Control to Professional Monitoring With New V Serie...

13/08/2026

Anthony Noack Captures Artist's Journey in Estonia with Blackmagic Design

Anthony Noack Captures Artist's Journey in Estonia with Blackmagic Design Brie Clayton August 12, 2026 0 Comments Solo filmmaker documents concept...

13/08/2026

ARRI Image Science makes its mobile debut in the HONOR Robot Phone

ARRI Image Science makes its mobile debut in the HONOR Robot Phone Brie Clayton August 12, 2026 0 Comments Hero image displays the HONOR Robot Phone, ...

13/08/2026

ASG Adds Eran Binyamin Zeitoun as Senior Technical Program Manager

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

Vinten To Feature Manual, Robotic Camera Support Offerings At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

SMPTE Unveils IBC 2026 Plans

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

IP Showcase on the Water Announces Schedule for IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

Spectrum, Optimum Ink New Deals for News Carriage and Ads

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

IAB Tech Lab Updates Privacy Standards

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

Suitest Highlights New AI-Powered Device Testing and Qual...

Suitest's Test Automation Platform Simplifies Video Quality Assurance, Monitoring, and Device Certification Across the OTT Ecosystem At IBC2026, Suitest, a...

13/08/2026

Cinegy showcases powerful, agile, cost-effective media so...

Cinegy GmbH, the premier provider of software-defined television technology, will have its key software solutions ready to demonstrate at SET Expo 2026 (18 20...