Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

27/07/2026

AES Educational Foundation Announces 2026/2027 Scholarship and Grant Recipients

The Audio Engineering Society Educational Foundation has named its scholarship and grant recipients for the 2026/2027 academic year. The awards support students...

27/07/2026

ESPN and MEAC Reach Multi-Year Media Rights Agreement Through 2030-31

ESPN and the Mid-Eastern Athletic Conference (MEAC) have signed a multi-year media rights agreement covering basketball championships and select football and ba...

27/07/2026

Yahoo Sports and Cameo Integrate Celebrity Video Bookings into Yahoo Fantasy

Yahoo Sports and Cameo have partnered to allow fantasy football users to book personalized celebrity videos directly within the Yahoo Fantasy app and website. T...

27/07/2026

Eutelsat to Receive $504 Million in FCC Upper C-Band Transition Payments

Eutelsat has welcomed the FCC's order establishing the regulatory framework for the transition of Upper C-band spectrum in the United States. Under the orde...

27/07/2026

ESPN Acquires U.S. and International Rights to International Dance League

ESPN and the International Dance League (IDL) have signed a global rights agreement for the 2026 season. Events will air bi-weekly on ESPN2 at 7 p.m. ET beginni...

27/07/2026

NBCUniversal and YouTube Announce Multi-Year Global Partnership

NBCUniversal and YouTube have signed a multi-year global partnership that includes bundling Peacock with YouTube Premium, extending NBCUniversal's distribut...

27/07/2026

Two Weeks Away: 2026 SVG Venues & Teams Summit at Miamis Kaseya Center To Reunite In-Stadium Production Community

Agenda includes presentations from the Miami Heat, Inter Miami, Florida Panthers...

27/07/2026

Behind The Mic: CBS Sports Bolsters College Football Analyst Team

Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...

27/07/2026

PBR Goes Animated With BS: Bull Stories, Giving Its Bucking Bulls a Voice

Ten one-minute shorts will roll out across CBS, The CW, Paramount+ during the 2026 PBR Teams season PBR is officially moving into scripted animation. The Profe...

27/07/2026

Vochlea announce DubBox

New app turns vocal percussion into polished loops Vochlea, the company behind the voice-to-MIDI instrument Dubler 2, have just announced the launch of a ne...

27/07/2026

Sonora Cinematic launch Pure 12-String for Soundbox

Latest MPE-capable instrument completes Pure Trilogy line-up Sonora Cinematic have just released the third and final instalment in their series of acoustic ...

27/07/2026

fedDSP introduce Opto compressor plug-in

Emulates four varieties of optical compression The latest addition to the fedDSP line-up has just been announced, and packs four distinct optical compressor...

27/07/2026

Drawmer reveal the OC1 & FC1

Optical & FET compressors join 500-series line-up At our recent GearExpo UK show, we heard from Drawmer that they would soon be launching a pair of new 500-...

27/07/2026

T-Mobile Tops Ookla's Mobile Network Rankings

Share Copy link Facebook X Linkedin Bluesky Email...

27/07/2026

Pebble to Feature Virtualization, Cloud-Based Playout Tech at IBC

Share Copy link Facebook X Linkedin Bluesky Email...

27/07/2026

NBCU Telemundo Turns To Chyron PRIME 5 For World Cup Graphics

Share Copy link Facebook X Linkedin Bluesky Email...

27/07/2026

Lightware Adds SNMPv3 Monitoring to Taurus Devices

Lightware announces SNMPv3-based monitoring support for selected Taurus devices, available from firmware v1.20.0. The update enables IT administrators and AV in...

27/07/2026

Mediagenix Showcases the Real-Time Media Enterprise at IB...

Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, will showcase its vision for the Real-Time...

27/07/2026

Official trailer: Gugu Mbatha-Raw stars in Sky Original supernatural thriller Possession

All episodes available Thursday 6 August on Sky and streaming service NOWMonday ...

27/07/2026

Seven paradoxes shaping the next era of media production - Episode 4

Why Confidence MattersIn this series, we explore the technologies, architectures and operational realities shaping modern media operations. Along the way, we ex...

27/07/2026

Apple brings AppleCare One to the UK, streamlining coverage - UPDATE - Posted on 27 July 2026

Apple today announced AppleCare One - a new way for customers to protect multipl...

27/07/2026

944,000 VIEWERS WATCH RT'S COVERAGE OF MAYO'S HISTORIC WIN, AFTER A 75 YEAR WAIT TO LIFT THE SAM MAGUIRE

Yesterday witnessed truly memorable scenes in Croke Park, bringing the curtain d...

27/07/2026

SES Statement On FCC Upper C-band Auction Decision

Luxembourg, July 27, 2026 - SES, a leading space solutions company, issued the following statement following the U.S. Federal Communications Commission approval...

27/07/2026

Industry Leaders Unite in Open Secure AI Alliance for AI Safety and Security

Open source software is a critical pillar of the global economy. It underpins cloud computing, financial services, manufacturing, telecommunications, government...

26/07/2026

Audio Vivid: A New Destination for Immersive Audio

What is Audio Vivid? A New Destination for Immersive Audio Over the past several years, immersive audio has evolved from a niche production consideration into a...

26/07/2026

NVIDIA Harnesses Vera CPU to Speed Up Design of Next-Generation CPUs and GPUs

The complexity of modern chip design continues to grow as engineering teams work to develop increasingly sophisticated CPUs, GPUs and AI systems. To help meet t...

25/07/2026

Vochlea introduce DubBox

New app turns vocal percussion into polished loops Vochlea, the company behind the voice-to-MIDI instrument Dubler 2, have just announced the launch of a ne...

25/07/2026

Study: T-Mobile Remains Best Mobile and Fastest 5G Network

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Mark Bridges to Lead CableLabs

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Total YouTube Views for All FIFA World Cup Content Top 200 Billion

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Pebble To Address Future Of Media Operations At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Paramount Agrees to Pause Warner Bros. Discovery Deal

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Export FCP7 to XML Without Needing FCP7 Installed

Export FCP7 to XML Without Needing FCP7 Installed Dave Rice July 24, 2026 0 Comments I've created an open source tool that allows you to export FC...

25/07/2026

Documentary Sell Your House Edited with DaVinci Resolve Studio

Documentary Sell Your House Edited with DaVinci Resolve Studio Brie Clayton July 24, 2026 0 Comments Filmmakers post feature documentary from opposite...

25/07/2026

Time to Act: Bringing a New Opera to Life

Time to Act: Bringing a New Opera to Life Boston Conservatory at Berklee vocal arts students played a vital role in shaping the opera's creative developme...

25/07/2026

Beyond the Score

Beyond the Score For Dean of Music Kevin Haden, the boundless possibilities of classical music begin with process. July 24, 2026 By Kevin Haden Michelle ...

25/07/2026

Reading the Room

Reading the Room As music director of two major orchestras, conductor Jonathon Heyward aims to make the concert hall a space for everyone. July 24, 2026 By ...

25/07/2026

First look revealed for YAGA, a seductive mystery thriller coming to Sky and NOW this October

Saturday 25 July 2026 First look revealed for YAGA, a seductive mystery thrille...

24/07/2026

ESPN Adds New Production Features for NFL FLAG Championships

Referee hat cameras, player/coach mics, and virtual graphics will be deployed at the three-day event for the first time The NFL FLAG Championships return to ES...

24/07/2026

Qvest, Ateme and Scaleway Deliver Sovereign OTT Platforms for European Broadcasters

Qvest, Ateme, and Scaleway have partnered to offer broadcasters, media organizat...

24/07/2026

LTN Reaches 3,000 Satellite-to-IP Migrations as FCC Votes on C-Band Auction

LTN has completed satellite-to-IP migrations for more than 3,000 broadcaster, MVPD (Multi-Channel Video Programming Distributor), head-end, and content owner si...

24/07/2026

Chyron Releases Virtual Placement 8.1

Chyron has released Virtual Placement 8.1, an update to its live sports virtual graphics platform. The release adds workflow and graphics changes across chroma ...

24/07/2026

Riedel Deploys RefCam and Easy5G at International Soccer Match

Riedel's RefCam Live was used for the first time at an international soccer match during a friendly between the German men's national team and Ghana in ...

24/07/2026

USSI Global Offers Services for Next Phase of C-Band Spectrum Transition

USSI Global said it is prepared to help broadcasters, satellite network operators, and media organizations manage the next phase of C-band spectrum clearing, fo...

24/07/2026

Fubo Launches The Athletic Video Hub on CTV

FuboTV and The Athletic have launched The Athletic Video Hub on Fubo, bringing The Athletic's sports video content to connected TV (CTV) for the first time....