Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

North America Stories

20/05/2026

DSPolitical and FreeWheel Partner on Political CTV Ads

Share Copy link Facebook X Linkedin Bluesky Email...

20/05/2026

Amazon, Netflix and Google to Capture Half of CTV Ad Market by 2030

Share Copy link Facebook X Linkedin Bluesky Email...

20/05/2026

Roku Introduces 'Creator Hub

Share Copy link Facebook X Linkedin Bluesky Email...

19/05/2026

Cisco and USGA Announce Multiyear Partnership Extension

Cisco and the USGA have announced a multiyear extension of their partnership, which began in 2018. Cisco serves as the Official Technology Partner of the USGA, ...

19/05/2026

Urban Edge Network and Spiideo Partner to Enhance NAIA Sports Streaming

Urban Edge Network (UEN), a streaming platform for NAIA sports, has announced a partnership with Spiideo to provide streaming and production tools to UEN's ...

19/05/2026

Warner Bros. Discovery Announces Expanded Roland-Garros 2026 Coverage

Warner Bros. Discovery (WBD) will provide live coverage of all 900 Roland-Garros matches across its platforms beginning with qualifiers on May 18. In Europe, 21...

19/05/2026

Tubi Launches FIFA World Cup 2026 FOX Hub

Tubi, Fox Corporation's free streaming service, has announced the launch of the FIFA World Cup 2026 FOX Hub, a dedicated destination for World Cup programmi...

19/05/2026

Telefnica and Sony Conduct 5G mmWave Broadcast Trial at Movistar Arena

Telef nica, in collaboration with Sony, has conducted a 5G connectivity trial at the Movistar Arena in Spain using the 26 GHz millimetre wave (mmWave) band. The...

19/05/2026

Ross Production Services Integrates Calrec Argo M into New Hypermax-1 Truck

Ross Production Services (RPS) has installed a Calrec Argo M console into its new Hypermax-1 remote production truck, replacing one of three Argo S consoles pre...

19/05/2026

Panasonic Projector and Display Acquires HIVE Media Control

Panasonic Projector and Display Corporation has announced the acquisition of 100% of the shares of UK-based media technology company Hive Media Control Ltd. (HI...

19/05/2026

Globecast Completes Major Renovation of Singapore Facility

Globecast has announced the completion of a nine-month renovation of its Singapore facility, converting it from a traditional linear broadcast operation into a ...

19/05/2026

Grass Valley Signs Three-Year Enterprise Agreement with Phoenix Broadcast Solutions

Grass Valley has announced a three-year enterprise agreement with Phoenix Broadc...

19/05/2026

Watch Brasil Selects Bitmovin for Streaming Player, Encoder, and Observability

Bitmovin has announced that Watch Brasil, a streaming platform operating across Brazil and Europe since 2018, has replaced its legacy systems with Bitmovin'...

19/05/2026

Ateme and Akamai Power Nepal Premier League Live Streaming

Ateme has announced the migration of Dish Home Nepal's Nepal Premier League (NPL) streaming infrastructure to Ateme's TITAN Live solution deployed on Ak...

19/05/2026

CMSI Supports ESPNs NCAA Gymnastics Championships Coverage

CMSI provided workflow, media management, and HDR support for ESPN during coverage of the NCAA Gymnastics Semifinals and Championships. The company supported fi...

19/05/2026

SVG Rewind: TGLs SmartPin Cam Offers a Whole New Perspective for Golf

In advance of this year's Sports Emmy Awards, SVG is taking a deep dive into the six production-technologies nominated for this year's George Wensel Tec...

19/05/2026

SVG Rewind: FOX IndyCar Takes In-Car AR Graphics to New Heights

In advance of this year's Sports Emmy Awards, SVG is taking a deep dive into the six production-technologies nominated for this year's George Wensel Tec...

19/05/2026

NEP Supershooter 11 and 65 Roll Out To Meet Different Production Needs

Featuring a fully IP infrastructure, Supershooter 11 is intended for large-scale events. Enabling remote and distributed workflows, Supershooter 65 joins the RE...

19/05/2026

Boots Riley Unpacks His Vision of Power, Style, and Time at Sundance Institute's I Love Boosters Denver Screening

By Jessica Herndon The line wrapped around the building outside Denver's La...

19/05/2026

To Deliver for the Warfighter, Acquisition Reform Must Provide a Level Playing Field

When all companies in the market are allowed to compete under the same set of ru...

19/05/2026

Unjammable Lifeline: L3Harris Fiber-Optic Tethers Keep Warfighters Connected

Battle-proven technology withstands electronic warfare threats across air, maritime surface and subsea domains....

19/05/2026

WESCAM MX-Series: Defining the Moment Before the Target Moves

WESCAM MX-Series EO/IR systems provide high-precision targeting across domains, including counter-UAS applications...

19/05/2026

Cobalt Digital and SineSix Media Partner to Transform Accessibility Compliance into a More Engaging Viewer Experience at NAB Show 2026

CHAMPAIGN, Ill. April 16, 2026 - Cobalt Digital today announced a partnership ...

19/05/2026

Cobalt Digital Announces Launch of blueCORE at NAB Show 2026: Compact Powerhouse Signal Processors for SDI and ST 2110 Workflows

Las Vegas - April 18, 2026 Cobalt Digital, the leading designer and manufactur...

19/05/2026

Advanced HDR by Technicolor and Cobalt Digital Address Live Production Gap as HDR Adoption Collides With Legacy Workflows

LAS VEGAS April 18, 2026 - Advanced HDR by Technicolor and Cobalt Digital are ...

19/05/2026

Cobalt Digital Wins Two Future Best of Show Awards at 2026 NAB Show

LAS VEGAS - May 4, 2026 - Cobalt Digital, the leading designer and manufacturer of signal processing products, and a founding partner in the openGear initiativ...

19/05/2026

Cobalt Digital Brings Flexible End-to-End IPMX/ST 2110 Ecosystem to MPTS

London - 11 May 2026 Cobalt Digital, the leading designer and manufacturer of award-winning signal processing products, and a founding partner in the openGear...

19/05/2026

NFL Pushes Back Against Critics of Games on Streaming

Share Copy link Facebook X Linkedin Bluesky Email...

19/05/2026

NEP Group Rolls Out Two New Mobile Production Units In U.S.

Share Copy link Facebook X Linkedin Bluesky Email...

19/05/2026

Phoenix Broadcast Solutions Raises its Live Production Ga...

Grass Valley has entered into a three-year enterprise agreement with Phoenix Broadcast Solutions as the Singapore-based company expands its live production oper...

19/05/2026

Clear-Com Enhances Musikal Perahu Kertas with Eclipse HX...

Clear-Com supported Musikal Perahu Kertas, a large-scale musical theater production in Indonesia adapted from the acclaimed novel by Dee Lestari, through the d...

19/05/2026

METEO CONSULT and La Chaine Meteo Select Actus Digital fo...

Actus Digital, a LiveU company, and a leading provider of intelligent media monitoring and broadcast compliance platforms, announced today that METEO CONSULT ha...

19/05/2026

Transforming Broadcast Operations - Globecast Unveils Sta...

Globecast, a leading provider of managed services for the broadcast and media industry, today announced the completion of a major renovation of its Singapore fa...

19/05/2026

Calrec Console Switch Helps Ross Production Services Fle...

Following a major upgrade to its Connecticut remote production centre with the installation of three 60-fader Argo S consoles in 2023, RPS has switched one of t...

19/05/2026

PlayBox Technology Launches a Modular Media Orchestration...

PlayBox Technology today announced the launch of Celebro Play, a browser-based media orchestration platform designed specifically for broadcast facilities and p...

19/05/2026

Publicis Groupe to Buy LiveRamp for $2.2 Billion

Share Copy link Facebook X Linkedin Bluesky Email...

19/05/2026

Tubi Launches 2026 FIFA World Cup Fox Hub

Share Copy link Facebook X Linkedin Bluesky Email...

19/05/2026

Viamedia Introduces a New Geographic Intelligence Layer

Share Copy link Facebook X Linkedin Bluesky Email...

19/05/2026

NFL Pushes Back Against Shift of Games to Streaming

Share Copy link Facebook X Linkedin Bluesky Email...

19/05/2026

LYNX Technik Brings SDI IP Integration and High-Density F...

LYNX Technik, provider of modular signal processing solutions for broadcast, post, and professional AV will showcase its latest signal processing solutions at B...

19/05/2026

Media Links Integrates Xscend IP Transport Platform with...

Media Links, a leading developer and manufacturer of IP media transport solutions, has announced the integration of its Xscend platform with Skyline Communicat...

19/05/2026

Media Links Expands in Southeast Asia with Clearcast Asia...

Collaboration expands regional partner ecosystem as Media Links showcases IP media transport solutions on stand 4M4-4 and alongside Clearcast Asia on stand Me...

19/05/2026

LynTec to Deliver Scalable Power Distribution and Integra...

Company Highlights Labor-Saving Control and Custom Branding Opportunities Designed to Increase Integrator Profitability LynTec, a leading manufacturer of ele...

19/05/2026

Mago Studio Moves AI Video Beyond Prompts with New Production-Focused Tools

Mago Studio Moves AI Video Beyond Prompts with New Production-Focused Tools Brie Clayton May 18, 2026 0 Comments New Capabilities Bring Controllable, ...

19/05/2026

It May be Time to Stop and Take an AI Breather

It May be Time to Stop and Take an AI Breather Brie Clayton May 19, 2026 0 Comments If the rock you stay on starts to roll, jump clean. Or you'll...

19/05/2026

NVIDIA and Google Cloud Empower the Next Wave of AI Builders

At this year's Google I/O conference, NVIDIA and Google Cloud are accelerating the work of more than 100,000 developers in the companies' joint develope...

19/05/2026

Netflix Unveils the Trailer for 'RAFA', the Documentary Series About Rafael Nadal

Back to All News Netflix Unveils the Trailer for RAFA, the Documentary Series A...

19/05/2026

Netflix Expands Consumer Products Footprint with Strategic Confectionery and Toy Partnerships

Back to All News Netflix Expands Consumer Products Footprint with Strategic Con...