Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

23/07/2026

Why The Next Big Opportunity in Sports Lies Outside the Stadium

In our latest blog, Laura Rognoni explores how the launch of Play Snooker reflects a wider shift in sports economics - from monetising peak broadcast audiences ...

23/07/2026

Scaling Smart with Vivid Broadcast

Download Now! To receive a short case study showcasing Vivid Broadcast's audio setup, please enter the following details: -- First Name* First Name ...

23/07/2026

Sony Unveils FX5 Full-Frame Camera

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

Sachtler To Show Akti8T Fluid Head, Flowtech75 Tripod At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

NextGen TV Launches in Cleveland

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

Yospace Stitched 11.57 Billion Ads into Live Streams During World Cup

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

At IBC2026 Witbe Will Showcase Tests That Build Themselve...

Witbe brings its AI-native testing and monitoring infrastructure to Amsterdam: demonstrations on real devices, a peer-reviewed technical paper in the conference...

23/07/2026

Quickplay Caretta Research Evergent and Yospace Announce...

Quickplay, Caretta Research, Evergent, and Yospace have announced that the renowned IBC Pride Happy Hour will once again bring the vibe and fun back to IBC on F...

23/07/2026

IBC2026 Interra Systems Brings Trusted Media Intelligence...

At IBC2026, Interra Systems will present this year's focus: operational AI at scale for QC, monitoring, and captioning workflows. The industry has moved pas...

23/07/2026

Illusionfx is finally coming!

Illusionfx is finally coming! Alex Perea July 23, 2026 0 Comments The new website is now live to mark the occasion. Since the last update, feedback h...

23/07/2026

Evergent showcases AI agents for streaming retention at I...

Evergent (5.A86) will showcase its Agentic Revenue Orchestration Platform for streaming services, sports leagues, pay-TV operators at IBC2026 (RAI Amsterdam, 11...

23/07/2026

Brian Jones Named NCTCs New CFO

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

Scripps Unites TV Operations Under Dean Littleton's Leadership

Share Copy link Facebook X Linkedin Bluesky Email...

23/07/2026

2D House is First North American Rental Facility to Offer...

David Dvir, 2D founder (left), Sundeep Reddy (center), and Jean-Marc Bouchut (right), both of Zeiss...

23/07/2026

Screen Australia announces new national documentary accelerator program

Screen Australia announces new national documentary accelerator program 23 July 2026 Screen Australia has today announced the Above the Line Documentary Accel...

23/07/2026

Centra returning as the official food partner of The Traitors Ireland

RT Commercial has today announced Centra as the official food partner once again of The Traitors Ireland, coming to RT One and RT Player later this year. Ce...

23/07/2026

Honda sponsors Today with David McCullagh on RT Radio 1

RT Commercial today announced Honda as sponsor of Today with David McCullagh RT Radio 1, weekdays 9am 11am. Broadcasting directly after Morning Ireland in ...

23/07/2026

RT NEWS ANNOUNCES NEW JOURNALIST REPORTING FROM ACROSS CORK AND KERRY

RT News is pleased to announce Barry Gallagher has been assigned as the new journalist covering news from Cork, Kerry and the wider Southern region Working al...

23/07/2026

GeForce NOW Sets Sail With Path of Exile: Curse of the Allflame' Joining the Cloud

Lock in and load up the cloud. GFN Thursday brings fresh updates and new adventu...

22/07/2026

ESPN's AutoMic Technology Pays Off at Wimbledon; Baker Credits Team, AELTC for Successful Fortnight

Among other tweaks to coverage, the broadcaster added announce positions in the ...

22/07/2026

Australian Sports Technologies Network Reports Industry Growth of 16% in 2026

The Australian Sports Technologies Network (ASTN) has reported a sharp acceleration in Australia's sportstech sector, with revenue growth rising from 10% in...

22/07/2026

Production Wins, Man!': Men in Blazers' Founder Roger Bennett on Winning the FIFA World Cup as a Digital Media Brand

From a retrofitted broadcast bus to packed Match Day Live! events, Bennett detai...

22/07/2026

TMRW Sports Appoints Brendan Donohue as President, Flag Football

TMRW Sports has named Brendan Donohue as President, Flag Football, where he will lead the company's professional flag football business in partnership with ...

22/07/2026

Stephen Arnold Music Creates Audio for FIFA World Cup Projection Mapping Display at Rockefeller Center

Stephen Arnold Music (SAM) designed and produced soundtracks for nine projection...

22/07/2026

Harmonic Donates $25,000 to Venezuela Earthquake Relief Through Global Empowerment Mission

Harmonic has announced a $25,000 donation to Global Empowerment Mission (GEM) fo...

22/07/2026

Telemundo Acquires Exclusive U.S. Spanish-Language Rights to UEFA Mens Club Competitions Beginning 2027/28

Telemundo has announced it has acquired from UC3 exclusive U.S. Spanish-language...

22/07/2026

The Rest Is Football Returns to Netflix for 2026/27 Premier League Season Through 2028 European Championship

The Rest Is Football, hosted by Gary Lineker, Alan Shearer, and Micah Richards, ...

22/07/2026

Sports Studio and TheSoul Group Announce YouTube and Social Distribution Partnership

Sports Studio has announced a partnership with TheSoul Group to distribute Free ...

22/07/2026

Daktronics Concludes 2026 High School Video Summit

Daktronics has concluded its 2026 High School Video Summit, a two-day hybrid event combining virtual sessions with hands-on host school visits across the countr...

22/07/2026

FOR-A America to Exhibit at MMCA 2026 with First U.S. HVS-Q12 Sale Announced

FOR-A America will exhibit at the MMCA Show in Pensacola, Florida (July 21-24), marking the company's first appearance at the house of worship AV event. FOR...

22/07/2026

New York Yankees Select Lumen Technologies for Stadium and Data Center Connectivity

The New York Yankees have selected Lumen Technologies (NYSE: LUMN) to connect Ya...

22/07/2026

IBC 2026: Calrec To Exhibit IP Audio Ecosystem, Announce New Products

Calrec will exhibit at IBC 2026 (Stand 8.C47) with demonstrations of its IP-native audio ecosystem built around the Argo console range, and will announce two ne...

22/07/2026

Solid State Logic Appoints Algam EKO as Italian Distributor

Solid State Logic has announced the appointment of Algam EKO as its distributor for SSL large format consoles in Italy, covering the Live, Broadcast, and Studio...

22/07/2026

FloSports Launches Free Streaming Channel on Prime Video

FloSports has announced the launch of the FloSports Channel, a free ad-supported streaming channel premiering on Prime Video in the U.S. on July 21. The channel...

22/07/2026

Savannah Bananas Deploys Calrec Audio Infrastructure for Banana Ball TV REMI Broadcasts

Banana Ball TV (BTV), the broadcast operation of the Savannah Bananas, has insta...

22/07/2026

SiriusXM and Audacy Announce Agreement To Add 42 Local Stations to SiriusXM

SiriusXM and Audacy have announced an agreement bringing 42 Audacy sports, news, and talk stations from 29 U.S. markets to SiriusXM. Twenty-three Audacy sports ...

22/07/2026

Overtime, FOX One Partner on Social-First FIFA World Cup Series Race to Glory for the Creator Generation

The collaboration produced more than 500 short-form videos across four cities th...

22/07/2026

Green Bay Packers Escalate In-House Video Production With New Content Studio

At 2,600 sq. ft., the new home of digital content houses a main studio, podcast space, and an auxiliary control room NFL franchises are accelerating their prod...

22/07/2026

ERA II Vocal Codex from Eduardo Tarilonte

Library now available in Engine Player Another of Eduardo Tarilonte's popular sample libraries has just been moved over to Engine Audio's Engine Pla...

22/07/2026

Cloudvocals improved FlashTrack FT-15 DSP Microphone

All-in-one mic & DSP interface overhauled Cloudvocal have just refreshed the design of their all-in-one mic, audio interface and DSP processor, introducing ...

22/07/2026

Electro-Harmonix launch the Deluxe Big Muff Pi 2

The final boss of the lost dual-op-amp circuit Electro-Harmonix have just introduced the Deluxe Big Muff Pi 2, which they say offers their latest take on t...

22/07/2026

Clear-Com Delivers Integrated Communications for Fryderyk Awards Gala in Krakw

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({ content_source:......

22/07/2026

IAB Tech Lab Updates Podcast Technical Measurement Guidelines

Share Copy link Facebook X Linkedin Bluesky Email...

22/07/2026

FCC Adopts Plan for C-Band Auction in July 2027

Share Copy link Facebook X Linkedin Bluesky Email...

22/07/2026

LTN surpasses 3000 satellite-to-IP migrations as FCC vote...

Proven managed IP solution helps broadcasters confidently navigate the future of broadcast distribution LTN announces it has completed successful satellite-to-...