Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

09/10/2026

Dolby Expands Dolby OptiView Platform with New Capabilities at IBC 2026

September 10 2026, 06:00 (PDT) Dolby Expands Dolby OptiView Platform with New Capabilities at IBC 2026 New Sports Intelligence helps providers better unders...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

23/09/2026

University of Washington's New Sundodger Studios Leverages ST 2110 to Bring Broadcast, Videoboard, and Creative Under One Roof

Four control rooms support B1G+ streams, videoboard shows, creative content, and...

23/09/2026

Mountain West Conference Moves Replay Operations West, Cutting Late-Night Hours

By Andrew Cohen, SVG Coorespondant The Mountain West debuted its new Replay Operations Center at the start of this football season, bringing real-time collabor...

23/09/2026

SVG Students to Watch: Brian Short, Ball State University

The Taylorville, IL, native is pushing creative boundaries at Ball State, gaining reps across directing, technical directing, tape, and more than 100 ESPN+ broa...

23/09/2026

The Musical and the Power of Spiteful Thinking

Alexander Heller, Nevada Jose, Will Brill, and Giselle Bonilla pose with cast members at The Musical premiere during the 2026 Sundance Film Festival at Eccles...

23/09/2026

The Crow Hill Company release Children's Choir

Includes natural and processed patches The Crow Hill Company's latest sample library captures a children's choir comprising a troupe of boys perform...

23/09/2026

Kreuzberg Audio reveal Oberton

Plug-in offers six bands of saturation Kreuzberg Audio's product range includes a selection of plug-ins and virtual instruments aimed primarily at hard-...

23/09/2026

SoundBridge 3.2.0 arrives

Latest DAW update now live SoundBridge have announced that the latest update for the remote-collaboration-focused DAW is now available. The release of Sound...

23/09/2026

Koori Knockout returns home exclusively to NITV and SBS On Demand in 2026

Koori Knockout returns home exclusively to NITV and SBS On Demand in 2026 11 September, 2026 Media releases Fans can catch all the live action exclusively ...

23/09/2026

Rohde & Schwarz acquires NEOSAT and PHYTRONIC and strengthens its competencies in sovereign satellite communications

Rohde & Schwarz acquires NEOSAT and PHYTRONIC and strengthens its competencies i...

23/09/2026

NFVF SEDIBA CREATIVE ENTERPRISE PROGRAMME CALL FOR APPLICATIONS

1. ABOUT THE PROGRAMME The National Film and Video Foundation (NFVF) invites applications from experienced producers who own or manage a South African product...

23/09/2026

CALL FOR THE SEVENTH PRESIDENTIAL EMPLOYMENT STIMULUS PROGRAMME(PESP 7) FOCUSED ON THE AUDIO-VISUAL INDUSTRY IS OPEN

The National Film and Video Foundation (NFVF) is pleased to announce the opening...

23/09/2026

NHK Technologies Inc. Chooses Calrec for Sports Production

Download Now! To receive a short case study showcasing NHK Technologies, Inc. (NT) s audio setup, please enter the following details: -- First Name* Fir...

23/09/2026

McDonald's takes top spot as New Zealand's fast-food tastes evolve, according to Nielsen

Nielsen CMI data reveals a changing fast-food landscape, with value, convenience...

23/09/2026

AFP, Dalet To Collaborate On News Video Provenance, Authenticity

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

Spectrum News NECN Launches Across New England

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

COW Jobs: Blender on Windows - Generalist - Remote

COW Jobs: Blender on Windows - Generalist - Remote Brie Clayton September 23, 2026 0 Comments Application Users - Blender on Windows - Generalist Se...

23/09/2026

Bitfocus wins IBC2026 Best of Show Award for latest Butto...

Version 1.8 recognised for new workflow tools, boosted control panel flexibility and ease of integration into IP networks...

23/09/2026

Hiltron Reports a Successful IBC2026

Hiltron Communications reports a successful 2026 International Broadcasting Convention, held at the RAI Amsterdam. Exhibiting on stand A37 in Hall 1, Hiltron ex...

23/09/2026

LiveU Showcases AI Driven Production Alongside Resilient...

Reinforcing its long-term commitment to the Middle East and North Africa (MENA) broadcast market, LiveU is presenting the Q Era at CABSAT 2026, on Stand H1-D35 ...

23/09/2026

AFP and Dalet Partner to Strengthen the Provenance and Au...

Agence France-Presse (AFP), a global news agency, and Dalet, a leading technology and service provider for media-rich organizations, today announced a partnersh...

23/09/2026

Berklee Connect Unlocks the Power of a Global Network

Berklee Connect Unlocks the Power of a Global Network Katie Vitolins BM '04 knows firsthand what Berklee's alumni community can make possible. Now she...

23/09/2026

Study: Renewals for Streaming Shows Continue to Decline

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

Study: More Than 4 in 10 U.S. Adults Own or Are Interested in Smart Glasses

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

Television Critics Association Revises Bylaws to Address AI

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

eX Celebrates Completion of Holland Mich. FTTP Network

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

Gray Media Renews Ohio Valley Conference Deal

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

QuickLink StudioPro Powers Next Generation Student Media...

Westminster Schools, a K through 12 school in Atlanta, Georgia with approximately 2,000 students, has transformed its student broadcast and digital storytelling...

23/09/2026

UK Pavilion Brings Leading British Technology to CABSAT a...

The 2026 edition of CABSAT and SATExpo may have a new home and dates in the diary, but it still retains a strong presence from UK companies in the GREAT Britain...

23/09/2026

Glensound marks 60 years with new audio developments at C...

Glensound will return to CABSAT with new developments in professional monitoring and Dante interfacing, alongside its established commentary, esports and parlia...

23/09/2026

Guy Martin's Proper Jobs returns for third series

A great watch - Radio Times TV's most likeable proper bloke - Daily Telegraph 23rd September 2026, London: Guy Martin returns to free streaming service...

23/09/2026

Sky Sports to show England Mens tour of South Africa as part of Cricket South Africa rights deal

Wednesday 23 September 2026 Sky Sports to show England Men's tour of South ...

23/09/2026

Comscore Partners with datafuelX to Bring Seamless Currency Adoption Into a Comprehensive Planning and Stewardship Workflow

Comscore Partners with datafuelX to Bring Seamless Currency Adoption Into a Comp...

23/09/2026

SES Appoints Michel Scholer to Board of Directors

Luxembourg, 23 September 2026 - SES today announced the appointment of Mr. Michel Scholer, Secretary General of the Luxembourg Government and Chief of Staff to ...

23/09/2026

New RT series Ceol le Chile amplifies the multicultural voices of Ireland today

RT One and RT Player, Friday 25 September Ceol le Ch ile, a new series that follows Irish artists as they collaborate with musicians from diverse background...

23/09/2026

RT true crime podcast Unsolved reveals significant new evidence in the case of missing person Philip Cairns

Available now on RT Listen New information has come to light in the case of mi...

23/09/2026

Sakeena Fiza Helps NVIDIA Hardware Succeed at Scale

When Sakeena Fiza describes her work as a validation engineer at NVIDIA, she does so in terms more befitting a detective story than a world-class engineering la...

22/09/2026

Young Journalist Award 2026: Meet the Final 12

From Malaysia to Mexico and from DR Congo to Ukraine, this year's 12 finalists for the 2026 Thomson Foundation Young Journalist of the Year award tell extra...

22/09/2026

ESPN, Precision Lasers Light Up NHL Player Media Tour in Las Vegas

The annual preseason content-gathering event brought 36 NHL players to Las Vegas, where ESPN used 2.5 kilowatts of laser power and a motion-control camera syste...

22/09/2026

Neumann Introduces KK 104 A and KK 105 A Wireless Capsule Heads

Neumann has introduced the KK 104 A (cardioid) and KK 105 A (supercardioid) wireless capsule heads, succeeding the KK 204 and KK 205. The new capsules are compa...

22/09/2026

NFL and TMRW Sports Announce Pro Flag Showcase During Super Bowl LXI Week

The NFL and TMRW Sports have announced a new annual Pro Flag Showcase to be held during Super Bowl LXI week in Los Angeles. The event will feature top flag foot...

22/09/2026

Washington State University Unveils New Daktronics Video Board and Audio System at Gesa Field

Washington State University has unveiled a new end zone video display and audio ...

22/09/2026

SVG Sit-Down: Disney's Dave Johnson on ESPN's New 4K HDR Distribution Pipeline and What It Means for Super Bowl LXI and Beyond

Disney's SVP of Live Operations and Engineering details the technology behin...

22/09/2026

Cosm Promotes Mazen Alawar to SVP of Marketing, Alexis Scalice to VP of Cosm Studios

Cosm has announced two senior promotions. Mazen Alawar has been promoted to SVP,...

22/09/2026

Mississippi State University Extends Camera Relationship with KOKUSAI DENKI

Mississippi State University's University Television Center has purchased four KOKUSAI DENKI SK-UHD7000-S2 cameras since 2024, with plans to add two more, a...

22/09/2026

Net Insight Appoints Christine Kjellvard as Interim CFO

Net Insight has announced that Christine Kjellvard will assume the role of interim Chief Financial Officer on November 1, 2026. Kjellvard has previously held in...

22/09/2026

Net Insight Launches Zyntai TimeNode Access PTP Grandmaster for Edge Sites

Net Insight has announced Zyntai TimeNode Access, a compact PTP Grandmaster designed for distributed edge sites. The product carries secure, precise time across...

22/09/2026

The Weight Explores the High Cost of Freedom in the Depression Era

Sam Hazeldine, Austin Amelio, Padraic McKinley, Julia Jones, and Ethan Hawke attend the premiere of The Weight at The Ray Theatre on January 26, 2026, in Park...