Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

07/05/2026

Sudan Digital Investigation Hub launches to support journalists navigating disinformation in conflict

Journalists reporting on Sudan are working in one of the most complex and fast-m...

07/05/2026

Victory+ CEO Neil Gruninger on the Impact of the Dallas Cowboys Deal, Creator-Led Content, and Free In-Market Access for Fans

Multi-year partnership positions Victory+ as a free home for Dallas Cowboys orig...

07/05/2026

SVG Students To Watch: Ella Peterson, University of Illinois

From sideline reporting to directing and producing, the talented sophomore is building a well-rounded foundation for a career in live sports production In the ...

07/05/2026

NAB 2026: Cobalt Digital Wins Two Future Best of Show Awards

Cobalt Digital has received two Future Best of Show awards at NAB Show 2026. The COBALT blueCORE platform was recognized by TV Tech, and the COBALT PACIFIC ULL-...

07/05/2026

Scripps National Spelling Bee Revamps Broadcast; Mina Kimes to Host, Embassy Rows Michael Davies to Produce

ESPN analyst Mina Kimes will host the televised semifinals and finals of the 202...

07/05/2026

MUBI Selects Bitmovin as Cloud VOD Encoding Partner

Bitmovin has announced that MUBI, the global film streaming platform, has selected Bitmovin as its cloud VOD encoding partner. Bitmovin's encoding infrastru...

07/05/2026

NBCUniversal Telemundo Enterprises and U.S. Soccer Extend Spanish-Language Media Rights Through 2030

NBCUniversal Telemundo Enterprises and the U.S. Soccer Federation have announced...

07/05/2026

Angel City Football Club and Victory+ Announce Regional Broadcast Partnership

Angel City Football Club (ACFC) and Victory have announced a regional broadcast partnership bringing live match coverage to fans across the greater Los Angeles...

07/05/2026

Spiideo Launches AI Highlights for Automated Highlight and Clip Creation

Spiideo has announced the launch of AI Highlights inside Spiideo Play, its automated sports production platform. AI Highlights combines video, event data, audio...

07/05/2026

Leostream Announces Unified HPC Remote Access Ecosystem for Post-Production and Enterprise Workflows

Leostream Corporation has announced a unified remote access ecosystem for high-p...

07/05/2026

Victory+ Adds Atlanta Dream Local Broadcasts to Free Streaming Platform

The Atlanta Dream has partnered with Victory to stream all locally broadcast Dream games for free, expanding the team's digital distribution strategy and g...

07/05/2026

Riedel Communications Appoints Marc Engroff as CFO; Frank Eischet Moves to Group COO Role

Riedel Communications has announced the appointment of Marc Engroff as Chief Fin...

07/05/2026

ARD Extends SES Satellite Distribution Agreement Through 2039

SES and ARD, Germany's largest public broadcasting network, have announced a long-term extension of their satellite distribution partnership through 2039. U...

07/05/2026

Case Study: How Pro Cheer League Built a Broadcast-First Arena Production

UpLight Technologies delivers a flexible video and lighting system for a new televised sportLaunching the Pro Cheer League required more than creating a compell...

07/05/2026

Full Day Productions and GSE Worldwide Launch Womens College Volleyball Showcase at AT&T Stadium

Full Day Productions and GSE Worldwide have announced Spikes Under the Lights, a...

07/05/2026

Deine persnliche Spotify DJ Experience ist jetzt in vier neuen Sprachen und weiteren Mrkten verfgbar

Seit dem Start im Jahr 2023 hat DJ (Beta) das personalisierte H rerlebnis von 94...

07/05/2026

Il DJ personalizzato di Spotify ora disponibile in 4 nuove lingue e in pi Paesi

Da quando stata presentata nel 2023, DJ (beta) ha aiutato a definire un'esperienza d'ascolto pi personalizzata per 94 milioni di utenti Spotify Premi...

07/05/2026

Your Personalized DJ Experience Is Expanding With 4 New Languages and More Markets

Since launching in 2023, DJ (beta) has helped shape a more personalized listenin...

07/05/2026

Seu DJ personalizado do Spotify t chegando em mais lugares - agora em 4 novos idiomas

Desde o lan amento em 2023, o DJ (beta) j ajudou a deixar a experi ncia de ouvi...

07/05/2026

Save Your Personal Podcast to Spotify and Listen Anywhere

From our earliest days, Spotify has been built on a simple principle: Great audio should be easy to reach. It's what's driven us to expand from music to...

07/05/2026

Spotify Celebrates API Heritage Month With New Music and Artist Spotlights

Asian and Pacific Islander artists continue to shape the global soundscape, pushing creative boundaries and connecting with fans worldwide. This Asian & Pacific...

07/05/2026

Symphonic Elements Chooir by UJAM

Company announce long-requested choir instrument In the latest expansion of their Symphonic Elements series, UJAM have introduced an all-new vocal instrumen...

07/05/2026

Buchla debut Ziggy synthesizer

Modular-inspired sound generation with digital control Buchla have introduced Ziggy, a new compact synthesizer built around the company's complex oscill...

07/05/2026

Polyend Drums hybrid drum machine

Combines analogue voices with digital synthesis and sequencing Polyend have announced Drums, a new hybrid analogue and digital drum machine that combines sy...

07/05/2026

Nektar Panorama CS12: Reaper & Fender Studio Pro support

Control surface now officially supports four DAWs When Nektar launched the Panorama CS12 control surface, it worked exclusively with Apple's Logic Pro, ...

07/05/2026

SGL Carbon: Expected lower sales, but EBITDA margin improved. Forecast confirmed

In the first quarter 2026, SGL Carbon generated consolidated sales of 184.5 million, which was 49.8 million, or 21.3%, lower than in the prior year (Q1 2025: ...

07/05/2026

Rohde & Schwarz and Greenerwave achieve precise and fast ESA antenna characterization using near-field technology

Rohde & Schwarz and Greenerwave achieve precise and fast ESA antenna characteriz...

07/05/2026

L3Harris Red Wolf + SKY RAIDER II INTERNATIONAL: Unmatched Flexibility for Kinetic Strike, EW, ISR and Support Missions

Two multi-role L3Harris products - the Red Wolf launched effects vehicle and SK...

07/05/2026

US Air Force Selects L3Harris to Develop Digital Infrastructure for its Advanced Battle Management System Network

L3Harris will be developing key features of a secure and resilient digital infra...

07/05/2026

New Nielsen Report Shows The Profound Impact of Asian Influence on Sports and Pop Culture

Alysa Liu and Shohei Ohtani Help Drive Viewership as 91 of the Top 100 Broadcast...

07/05/2026

IAB Tech Lab Introduces Standardized Signals For CTV Ad Portfolio

Share Copy link Facebook X Linkedin Bluesky Email...

07/05/2026

Cobalt Digital Wins Two Future Best of Show Awards at 202...

Cobalt Digital Wins Two Future Best of Show Awards at 2026 NAB Show Manufacturer Recognized by TV Tech and TVBEurope for Innovation in signal processing Cobal...

07/05/2026

FOR-A delivers creativity practicality and technical exce...

Software and hardware platforms, AI power and user-friendliness on show...

07/05/2026

Intinor brings advanced IP transport diagnostics and HDR...

Intinor will demonstrate its latest technical enhancements for the Direkt series at BroadcastAsia 2026. With a continued focus on reliable contribution and remo...

07/05/2026

Bitmovin Powers Global Film Streaming Platform MUBI

Bitmovin has announced that MUBI has chosen Bitmovin as its cloud VOD encoding partner, replacing MUBI's legacy on premises encoding setup to improve scalab...

07/05/2026

Meet Graduates from Berklees Class of 2026

Meet Graduates from Berklees Class of 2026 Members of this years graduating class reflect on their proudest moments at Berklee and look ahead to whats next. ...

07/05/2026

Convergence is Now Established Reality,' Says NAB 2026 Report

Share Copy link Facebook X Linkedin Bluesky Email...

07/05/2026

Interra Systems to Showcase Best in Class QC Monitoring a...

At BroadcastAsia 2026, Interra Systems will demonstrate its latest innovations in automated quality control (QC), real-time monitoring, and captioning. The comp...

07/05/2026

PlayBox Neo shines a spotlight on delivering Secure Scala...

At this year's Broadcast Asia, PlayBox Neo is set to unveil recent innovations across its PlayBox Neo Suite and integrated range of broadcast media solution...

07/05/2026

BFBS selects Synamedia to deliver New Digital Platform fo...

Strategic three-year partnership marks delivery phase of BFBS' next-generation media vision Media organisation and military charity BFBS (British Forces Br...

07/05/2026

Amagi Unlocks New Opportunities for CTV Performance with...

Amagi, the agentic industry cloud platform for unified broadcast, streaming, and monetization, today announced the availability of its In-Content Ads offering v...

07/05/2026

Lightware Introduces Rack Taurus For Clean Centralized AV...

Lightware Visual Engineering has developed Rack Taurus for rack-based environments such as education and corporate training spaces. It enables clean and simple ...

07/05/2026

Jigsaw24 Showcases End-to-End Media Workflows at MPTS 202...

Jigsaw24, the UK's leading media equipment supplier and systems integrator, will showcase its expanded end-to-end workflow capabilities at MPTS 2026, signal...

07/05/2026

Ease Live Granted US Patent for Technology Powering Perfe...

Ease Live, an Evertz company, today announced that it has been granted a US Patent for its proprietary graphical overlay technology, the foundation for interact...

07/05/2026

Leader prepares advanced Test and Measurement showcase fo...

Test & measurement innovator Leader Electronics of Europe will present an extensive range of Leader, PHABRIX and LeaderPhabrix test and measurement solutions at...