Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

North America Stories

18/03/2026

Net Insight Names Larissa GrnerMeeus CPO

Newly named chief product officer (CPO), Larissa G rner Meeus will return to Net Insight on May 4. Larissa G rner Meeus will become CPO of Net Insight on May 4...

18/03/2026

SVG Europe's The Football Summit 2026: Select Sessions Now Available to Watch on SVG PLAY

SVG Europe's The Football Summit 2026 explored how the sports broadcasting i...

18/03/2026

SVG's SportsTech@NABShow Blog Goes Live as Countdown to Vegas Heats Up

The 2026 NAB Show kicks off one month from today and SVG is once again set to cover the show from every angle. SVG's SportsTech@NAB Show Blog is now live - ...

18/03/2026

The Premiere of Dead Lover Showcases Grave Robbing and Sex With a Giant Finger

(L-R) The cast and crew of Dead Lover at The Ray Theater for its premiere at the 2025 Sundance Film Festival. (Photo by Robin Marshall/Shutterstock for Sundan...

18/03/2026

SMPTE Unveils 2026 NAB Show Educational Presentations

SMPTE Unveils 2026 NAB Show Educational Presentations Brie Clayton March 18, 2026 0 Comments SMPTE , the home of media professionals, technologists, a...

18/03/2026

Auditel Ad Campaign Shot on Blackmagic PYXIS 12K

Auditel Ad Campaign Shot on Blackmagic PYXIS 12K Brie Clayton March 18, 2026 0 Comments LED wall virtual production blends 12K open gate acquisition w...

18/03/2026

Brainstorm transforms productivity and sustainability with Suite 7 at NAB Show 2026

Brainstorm transforms productivity and sustainability with Suite 7 at NAB Show 2...

18/03/2026

Neutrik To Showcase opticalCON ADVANCED Connectors At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

18/03/2026

SMPTE Details 2026 NAB Show Educational Sessions

Share Copy link Facebook X Linkedin Bluesky Email...

18/03/2026

Ben Bradshaw Joins PSSI as Director, Product and Network Development

Share Copy link Facebook X Linkedin Bluesky Email...

18/03/2026

Peter Thordarson Joins ASG as Technical Account Executive

Share Copy link Facebook X Linkedin Bluesky Email...

18/03/2026

Survey: Voters Trust TV News Over AI, Social and Search

Share Copy link Facebook X Linkedin Bluesky Email...

18/03/2026

2026 NAB Show Exhibitor Insight: Amazon Web Services (AWS)

Share Copy link Facebook X Linkedin Bluesky Email...

18/03/2026

SMPTE Unveils 2026 NAB Show Educational Presentations

SMPTE , the home of media professionals, technologists, and engineers, today unveiled its educational presentations for the 2026 NAB Show. This year SMPTE will ...

18/03/2026

Maxon Marks Its Official Entry Into the AEC Market With I...

Maxon, maker of powerful, approachable software solutions for creators working in 2D and 3D design, motion graphics, visual effects, gaming, and more, today ann...

18/03/2026

Digital Alert Systems NAB Preview 2026

Digital Alert Systems Preview 2026 NAB Show April 19 - 22 Booth C3452 At the 2026 NAB Show, Digital Alert Systems will showcase Version 6.0 of its DASDEC ...

18/03/2026

Setplex Transforms Video Streaming with AI and Super Aggr...

Setplex today announced that it will showcase its complete, fully integrated Zapflex platform for the first time at the 2026 NAB Show, introducing powerful new ...

18/03/2026

COW Jobs: Seeking DP for Low Budget Dramedy - Chicago

COW Jobs: Seeking DP for Low Budget Dramedy - Chicago Brie Clayton March 17, 2026 0 Comments Seeking Director of Photography for Low Budget Dramedy Fe...

18/03/2026

COW Jobs: Seeking Gaffer for Low Budget Dramedy - Chicago

COW Jobs: Seeking Gaffer for Low Budget Dramedy - Chicago Brie Clayton March 17, 2026 0 Comments Seeking Gaffer for Low Budget Dramedy Feature Film- I...

18/03/2026

COW Jobs: Seeking Location, Sound for Low Budget Dramedy - Chicago

COW Jobs: Seeking Location, Sound for Low Budget Dramedy - Chicago Brie Clayton March 17, 2026 0 Comments Seeking Location/Sound for Low Budget Dramed...

18/03/2026

COW Jobs: Seeking Child Wrangler for Low Budget Film - Chicago

COW Jobs: Seeking Child Wrangler for Low Budget Film - Chicago Brie Clayton March 17, 2026 0 Comments Seeking Child Wrangler for Low Budget Dramedy Fe...

18/03/2026

Calrec Redefines Broadcast Workflows at NAB 2026 with its Most Powerful Hardware, Virtual and Hybrid Audio Lineup Yet

Calrec Redefines Broadcast Workflows at NAB 2026 with its Most Powerful Hardware...

18/03/2026

Oscar Nominated Two People Exchanging Saliva Posted with DaVinci Resolve Studio

Oscar Nominated Two People Exchanging Saliva Posted with DaVinci Resolve Studio Brie Clayton March 17, 2026 0 Comments DaVinci Resolve Studio handle...

18/03/2026

Boston Conservatory Presents Celebrated Musical Satire Urinetown

Boston Conservatory Presents Celebrated Musical Satire Urinetown Performances for this Center Stage production will take place at Boston Conservatory Theater ...

18/03/2026

Charlie Puth Joins Switched On Pop at Berklee NYC

Charlie Puth Joins Switched on Pop at Berklee NYC The Berklee alum spoke with host and Berklee NYC professor Charlie Harding for a live taping, answering audi...

18/03/2026

Preparatevi Alla Terza E Ultima Stagione De 'La Legge di Lidia Pot' In Arrivo Solo Su Netflix Dal 15 Aprile

Back to All News Preparatevi Alla Terza E Ultima Stagione De La Legge di Lidia ...

18/03/2026

Netflix Drops Teaser and First Look Images for The Chestnut Man: Hide and Seek'

Back to All News Netflix Drops Teaser and First Look Images for The Chestnut M...

17/03/2026

NASA+ Prepares To Live Stream Historic Artemis II Mission, Bringing Deep-Space Exploration to Global Audiences

NASA+'s Rebecca Sirmons and Brittany Brown offer unique look at live streami...

17/03/2026

BBright's TTML & SMPTE ST 2110-43: One Single Stream For the Whole World

The transition to IP has fundamentally reshaped professional media infrastructures. Video, audio, and increasingly metadata now circulate as independent, precis...

17/03/2026

Op-Ed: How Generative AI Is Transforming Live Sports Streaming Optimization

Live sports streaming can push every element in your video delivery chain to its limit, exposing every potential weakness in seconds. When the Super Bowl, the O...

17/03/2026

Dell Case Study: Powering the Future of Sports Media One Experience at a Time at UT Austin

Texas Athletics sought to modernize its media production, enhance fan experience...

17/03/2026

NAB 2026: Ikegami to Showcase Latest Generation TV Production Cameras, Controllers and Monitors

Ikegami USA will demonstrate the latest additions to its wide range of broadcast...

17/03/2026

TNA Wrestling and iHeartMedia Announce Major Multi-Platform Collaboration

TNA Wrestling and iHeartMedia announces a new multi-platform collaboration that will integrate iHeartMedia across TNA's premium live events, weekly televisi...

17/03/2026

The Miami Dolphins and Dell Boost Fan Experience, Safety, and Efficiency at Hard Rock Stadium

The goal was to transform Hard Rock Stadium into a global leader in sports and e...

17/03/2026

Spectrum Launches Multiview for NCAA Basketball Tournaments

Spectrum has announced the launch of its new Multiview feature in the Spectrum TV App, giving customers the ability to watch up to four NCAA men's or women&...

17/03/2026

Pac-12 Inks Integrity/Data Deals With Genius Sports, IC360

Genius Sports deal also covers data technology, AI, fan engagement, and performance analysis....

17/03/2026

Rede Massa Chooses Net Insight to Enable State-Wide Centralized Operations

Net Insight is supporting the rollout of a new state-wide centralized operation with Rede Massa, which is an SBT affiliate, the Brazilian regional television ne...

17/03/2026

F1 The Movie' Wins the Academy Award for Best Sound

Featuring audio from practice sessions, qualifying races, and Grand Prix races, the film represents Apple's sports-media ambitions At Sunday night's Ac...

17/03/2026

SVG New Sponsor Spotlight: Oracle's Mark Ramberg on the Future of Live Broadcast in the Cloud with OCI

Live broadcast has always been one of the most demanding environments in media a...

17/03/2026

DIRECTV Adds Multiview and Sports Central Features Ahead of NCAA Tournament

DirecTV is introducing several new viewing features, including a multi-screen March Madness Mix channel and an updated Sports Central mobile app hub, ahead of...

17/03/2026

Deltatre and ATP Media Announce Multi-Year Broadcast Graphics/Data Partnership

Deltatre has announced a multi-year partnership with ATP Media, the media arm of the ATP Tour, covering broadcast graphics, data, and production across the 2026...

17/03/2026

Detroit Pistons, Scripps Sports To Air Five Games Free Over the Air on TV-20 Detroit

The Detroit Pistons have announced a third consecutive season partnering with Sc...

17/03/2026

NAB Appoints Two New Members to Television Board of Directors

Share Copy link Facebook X Linkedin Bluesky Email...

17/03/2026

PMVG's TechConnect Goes Virtual for 2026

Share Copy link Facebook X Linkedin Bluesky Email...

17/03/2026

Miris unlocks high-fidelity 3D asset streaming at scale

3D streaming infrastructure provider Miris today announced the launch of a public beta for its new 3D asset streaming platform. Miris is building the infrastruc...

17/03/2026

Tedial Powers the Future of Media Operations at NAB Show...

As media organizations face mounting pressure to produce more content, faster, while maximizing value and operational efficiency, Tedial, a leading provider of ...

17/03/2026

Brainstorm transforms productivity and sustainability wit...

Brainstorm, a leading manufacturer of real-time graphics, augmented and virtual production, is launching the newest version of its platform, Brainstorm Suite 7,...

17/03/2026

Limecraft Introduces New Platform Update Adding Greater C...

Limecraft today announces the release of Limecraft 2026.2, the second platform update in its 2026 release cycle. Limecraft is an AI-powered production platform ...

17/03/2026

Pioneering the Next Era of Sports Broadcasting - Broadcas...

Broadcast Solutions, a leading system integrator and provider of innovative solutions for the broadcast and media industry, showcased its latest broadcast and V...