Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

North America Stories

22/07/2026

ESPN's AutoMic Technology Pays Off at Wimbledon; Baker Credits Team, AELTC for Successful Fortnight

Among other tweaks to coverage, the broadcaster added announce positions in the ...

22/07/2026

Australian Sports Technologies Network Reports Industry Growth of 16% in 2026

The Australian Sports Technologies Network (ASTN) has reported a sharp acceleration in Australia's sportstech sector, with revenue growth rising from 10% in...

22/07/2026

Production Wins, Man!': Men in Blazers' Founder Roger Bennett on Winning the FIFA World Cup as a Digital Media Brand

From a retrofitted broadcast bus to packed Match Day Live! events, Bennett detai...

22/07/2026

TMRW Sports Appoints Brendan Donohue as President, Flag Football

TMRW Sports has named Brendan Donohue as President, Flag Football, where he will lead the company's professional flag football business in partnership with ...

22/07/2026

Stephen Arnold Music Creates Audio for FIFA World Cup Projection Mapping Display at Rockefeller Center

Stephen Arnold Music (SAM) designed and produced soundtracks for nine projection...

22/07/2026

Harmonic Donates $25,000 to Venezuela Earthquake Relief Through Global Empowerment Mission

Harmonic has announced a $25,000 donation to Global Empowerment Mission (GEM) fo...

22/07/2026

Telemundo Acquires Exclusive U.S. Spanish-Language Rights to UEFA Mens Club Competitions Beginning 2027/28

Telemundo has announced it has acquired from UC3 exclusive U.S. Spanish-language...

22/07/2026

The Rest Is Football Returns to Netflix for 2026/27 Premier League Season Through 2028 European Championship

The Rest Is Football, hosted by Gary Lineker, Alan Shearer, and Micah Richards, ...

22/07/2026

Sports Studio and TheSoul Group Announce YouTube and Social Distribution Partnership

Sports Studio has announced a partnership with TheSoul Group to distribute Free ...

22/07/2026

Daktronics Concludes 2026 High School Video Summit

Daktronics has concluded its 2026 High School Video Summit, a two-day hybrid event combining virtual sessions with hands-on host school visits across the countr...

22/07/2026

FOR-A America to Exhibit at MMCA 2026 with First U.S. HVS-Q12 Sale Announced

FOR-A America will exhibit at the MMCA Show in Pensacola, Florida (July 21-24), marking the company's first appearance at the house of worship AV event. FOR...

22/07/2026

New York Yankees Select Lumen Technologies for Stadium and Data Center Connectivity

The New York Yankees have selected Lumen Technologies (NYSE: LUMN) to connect Ya...

22/07/2026

IBC 2026: Calrec To Exhibit IP Audio Ecosystem, Announce New Products

Calrec will exhibit at IBC 2026 (Stand 8.C47) with demonstrations of its IP-native audio ecosystem built around the Argo console range, and will announce two ne...

22/07/2026

Solid State Logic Appoints Algam EKO as Italian Distributor

Solid State Logic has announced the appointment of Algam EKO as its distributor for SSL large format consoles in Italy, covering the Live, Broadcast, and Studio...

22/07/2026

FloSports Launches Free Streaming Channel on Prime Video

FloSports has announced the launch of the FloSports Channel, a free ad-supported streaming channel premiering on Prime Video in the U.S. on July 21. The channel...

22/07/2026

Savannah Bananas Deploys Calrec Audio Infrastructure for Banana Ball TV REMI Broadcasts

Banana Ball TV (BTV), the broadcast operation of the Savannah Bananas, has insta...

22/07/2026

SiriusXM and Audacy Announce Agreement To Add 42 Local Stations to SiriusXM

SiriusXM and Audacy have announced an agreement bringing 42 Audacy sports, news, and talk stations from 29 U.S. markets to SiriusXM. Twenty-three Audacy sports ...

22/07/2026

Overtime, FOX One Partner on Social-First FIFA World Cup Series Race to Glory for the Creator Generation

The collaboration produced more than 500 short-form videos across four cities th...

22/07/2026

Green Bay Packers Escalate In-House Video Production With New Content Studio

At 2,600 sq. ft., the new home of digital content houses a main studio, podcast space, and an auxiliary control room NFL franchises are accelerating their prod...

22/07/2026

IAB Tech Lab Updates Podcast Technical Measurement Guidelines

Share Copy link Facebook X Linkedin Bluesky Email...

22/07/2026

FCC Adopts Plan for C-Band Auction in July 2027

Share Copy link Facebook X Linkedin Bluesky Email...

22/07/2026

LTN surpasses 3000 satellite-to-IP migrations as FCC vote...

Proven managed IP solution helps broadcasters confidently navigate the future of broadcast distribution LTN announces it has completed successful satellite-to-...

22/07/2026

Chaos Releases V-Ray 7 Update 4 Connecting Real-Time Work...

Today, Chaos announced V-Ray 7, update 4, eliminating the need for asset conversions and tool switching, which slow down media and entertainment pipelines. Arti...

22/07/2026

An AI playground for the Project Indigo camera app

An AI playground for the Project Indigo camera app Marc Levoy July 22, 2026 0 Comments At left is the user interface of our Playground, running on an ...

22/07/2026

RTS Chooses Custom Consoles Desks and MediaWalls for Stud...

Custom Consoles announces the completion of a large technical furniture project for Radio T l vision Suisse (RTS) at the networks recently inaugurated productio...

22/07/2026

Graham Media's WDIV Opens Satellite Newsroom/Coffee Shop

Share Copy link Facebook X Linkedin Bluesky Email...

22/07/2026

FCC Approves License Transfer of WTVQ-DT to Scripps

Share Copy link Facebook X Linkedin Bluesky Email...

22/07/2026

Spectrum Reach Expands Operations in N.Y.

Share Copy link Facebook X Linkedin Bluesky Email...

22/07/2026

Miris spatial streaming platform adds support for glTF in...

Culver City, CA, July 21, 2026 3D streaming infrastructure provider Miris today announced glTF intake, including the binary .glb form, to its spatial streamin...

22/07/2026

Documentary Lo Invisible Captured with Blackmagic PYXIS 6K

Documentary Lo Invisible Captured with Blackmagic PYXIS 6K Brie Clayton July 21, 2026 0 Comments Camera's versatile design allowed filmmakers to c...

22/07/2026

AAF Bridge converts AAF and XML Timelines to Ableton Live - and back

AAF Bridge converts AAF and XML Timelines to Ableton Live - and back Jerome Gardier July 21, 2026 0 Comments A two-way converter for macOS and Windows...

22/07/2026

From Class to Concert: Explore a Student-Led Production of David Bowies Work Through Photos

From Class to Concert: Explore a Student-Led Production of David Bowies Work Thr...

22/07/2026

NVIDIA Open Sources First GPU-Accelerated Medical Physics Simulation Framework

Before a healthcare robot can be useful in the real world, it has to learn how the physical world pushes back. Anatomy varies. Instruments bend, press, slip and...

21/07/2026

FIFA World Cup 2026: The Firsts, Breakthroughs, and Lasting Legacies

With winners Spain having already paraded the World Cup trophy through Madrid in front of millions of fans and the dust now settling on an epic tournament of th...

21/07/2026

NHL Launches Centralized Production Operation for Regional Broadcasts; Four Teams Aboard To Start

In a time of disarray in regional sports broadcasting, the league will provide c...

21/07/2026

With Boland Monitors, NEPs PGA TOUR Unit Ready for Future

When NEP designed one of its largest recurring mobile production unit fleets for PGA TOUR coverage, the leading media services provider faced a familiar challe...

21/07/2026

TVP Claims Top Spot as the World Cup and the Opole Music Festival Drive June Viewership in Poland

Warner Bros. Discovery's unbroken lead is interrupted as sports and live mus...

21/07/2026

Savannah Bananas Tap Calrec to Expand Banana Ball TV Coverage

Share Copy link Facebook X Linkedin Bluesky Email...

21/07/2026

Where Growth Takes Root Viaccess-Orca Showcases the Found...

Across the TV and streaming ecosystem, industry players face mounting pressure to do more with less. They must simplify operations, protect premium content and ...

21/07/2026

Tuxera breaks performance ceilings and brings SMB/NFS mul...

Tuxera, a leading prover of quality-assured file systems and networking technologies, is bringing its latest advances in connectivity performance to the media w...

21/07/2026

COW Jobs: Apple Motion Expert for Custom FCP Rigged Graphics Package

COW Jobs: Apple Motion Expert for Custom FCP Rigged Graphics Package Brie Clayton July 21, 2026 0 Comments HIRING: Apple Motion Expert for Custom FCP ...

21/07/2026

Cinegy directly addresses the real issues of software-def...

Cinegy GmbH, the premier provider of software-defined television technology, will use its presence at IBC2026 (stand 7.A01, Amsterdam RAI, 11 14 September) to...

21/07/2026

Calrec brings leading audio solutions and long term busin...

Calrec will be located in Hall 8, on Stand C47 Beyond bigger For years, broadcast facilities were built around one assumption: provision for the biggest pro...

21/07/2026

Pebble takes the next step towards the future of media de...

Pebble, the leading automation, content management and integrated channel specialist, will discuss its future-facing developments for the new generation of medi...

21/07/2026

nxtedition Brings Production, AI and Automation Together...

nxtedition returns to IBC2026 with its consolidated production platform, showing how scripting, editing, graphics, AI-assisted tools and automation can work wit...

21/07/2026

Big Blue Marble brings C2PA Content Credentials to Cloud...

C2PA content signing helps broadcasters, public institutions, and publishers give audiences a verifiable record of where their video content came from and how i...

21/07/2026

HDHomeRun Enables Operation During Internet Outages

Share Copy link Facebook X Linkedin Bluesky Email...

21/07/2026

Calif. Federal Judge Pauses Paramount-WBD Merger

Share Copy link Facebook X Linkedin Bluesky Email...

21/07/2026

AWARN Rebuts Weigel Claims of 3.0 EAS Problems

Share Copy link Facebook X Linkedin Bluesky Email...