Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

28/01/2026

Daniel Finn Joins LABF in Philanthropy Role

Share Share by: Copy link Facebook X Linkedin Bluesky Email...

28/01/2026

Tegna Expands Local News Offering with Revamped Mobile App

Share Share by: Copy link Facebook X Linkedin Bluesky Email...

28/01/2026

Marshall Electronics Unveils CV420 27X UHD Camera at ISE...

Marshall Electronics launches the CV420-27X, its next-generation ultra-high-definition (UHD) IP camera, at ISE 2026 (Stand 4N900). Engineered for modern IP-base...

28/01/2026

TVM Selects Grass Valley Technology for OB Truck Refurbis...

Grass Valley has announced that Television Mobiles Ltd. (TVM), one of Europe's leading independent outside broadcast providers, has carried out a major refu...

28/01/2026

FOR-A to show cutting edge technology at FOMEX 2026

AI, graphics and virtual software power new production capabilities FOR-A is bringing remarkable new technologies to FOMEX, the Future of Media Exhibition (ex...

28/01/2026

Riedel and Media Tailor Deliver Unified Broadcast and AV...

Continuing a longstanding collaboration, Riedel Communications and Nordic media technology company Media Tailor have once again joined forces to deliver a state...

28/01/2026

Pebble appoints Paul Nagle-Smith to drive fulfilment

Pebble has appointed Paul Nagle-Smith as vice president for customer fulfilment, strengthening its senior leadership focus on customer delivery and operational ...

28/01/2026

TV Azteca Strengthens Disaster Recovery Capabilities with...

Cloud playout solutions provider, Veset has announced that leading Mexican broadcaster, TV Azteca is using Veset Nimbus on AWS as a disaster recovery (DR) playo...

28/01/2026

MVP kicks off major football tournament with a tactical s...

Ensuring it can keep pace with a rapidly evolving live sports market, Balkan broadcast facility provider MVP Most Valuable Production has upgraded its flags...

28/01/2026

Akamai and Yospace Deliver Seamless Personalized Ad Exper...

Akamai Technologies, Inc. (NASDAQ: AKAM), the cloud solutions provider that powers and protects life online, and Yospace, the leader in dynamic ad insertion tec...

28/01/2026

Clear-Com Empowers Reykjavik City Theatre with New Upgrad...

The renowned Reykjavik City Theatre (RCT) recently underwent a major intercom system upgrade using Clear-Com solutions. This milestone project utilizes Clear-C...

28/01/2026

SES Acknowledges Fitch's Rating Action and Reiterates Deleveraging Plan

Luxembourg, January 26, 2026 - SES S.A. ( SES or the Company ), a leading space solutions company, acknowledges the credit rating action announced by Fitch to...

28/01/2026

OpenDrives Announces New Funding, Appoints Trevor Morgan CEO

Share Share by: Copy link Facebook X Linkedin Bluesky Email...

28/01/2026

AWARN Alliance Backs ATSC Sunset, NextGen TV Security Measures

Share Share by: Copy link Facebook X Linkedin Bluesky Email...

28/01/2026

More Than Two Dozen Groups Tell FCC to Reject Nexstar-Tegna Deal

Share Share by: Copy link Facebook X Linkedin Bluesky Email...

28/01/2026

Screen Australia refreshes Market & Audience approach to increase the impact of local content

28 01 2026 - Media release Screen Australia refreshes Market & Audience approach...

28/01/2026

Boston Conservatory Orchestra Premieres a New Piano Concerto by Peter and Leonardo Dugan

Boston Conservatory Orchestra Premieres a New Piano Concerto by Peter and Leonar...

27/01/2026

Akamai and Yospace Deliver Seamless, Personalised Ad Experiences to Tens of Millions of ABEMA Users

Click for Japanese version Tokyo, Japan - January 27, 2026 - Akamai Technolo...

27/01/2026

Alfred P. Sloan Foundation and Sundance Institute Name Winners of Science-In-Film Initiative's Feature Film Prize and Artist Grants

L-R: Jonathan Cuchacovich, Sonia Kennebeck, Alan Fischer, Daeil Kim, Andrew Sta...

27/01/2026

B-LINE Is Here to Help: Backline Launches 24/7 Music Industry Mental Health & Crisis Support Line in Partnership with Spotify

Today, Spotify is proud to support our partner Backline, an industry-leading men...

27/01/2026

Netn Vega's Genre-Bending Sound Hits a Crossover Moment on Spotify's RADAR

With nearly 29 million monthly listeners and clear momentum on Spotify, Net n Ve...

27/01/2026

25 Ontario Named One of Torontos Best New Buildings

January 14, 2026 We are proud to share that 25 Ontario, First Gulf's commercial project located just two minutes from our head office, has been recognized ...

27/01/2026

Full-Building Lease Secured at Speers Road

January 22, 2026 First Gulf is excited to share that a full-building lease has been secured at 625 Bronte Rd in Oakville, part of Bronte Station Business Park,...

27/01/2026

Another Gold for First Gulf and 351 King Street East

January 23, 2026 First Gulf continues to demonstrate its commitment to high-performance, sustainable real estate with 351 King Street East achieving BOMA BEST ...

27/01/2026

Clear-Com Empowers Reykjavik City Theatre with New Upgrades

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({ content_source:......

27/01/2026

Hubbard Broadcasting Renews Partnership With Nielsen in Albany and Albuquerque-Santa Fe Markets

Nielsen will continue to drive growth and provide local audience measurement in...

27/01/2026

Pebble Adds Paul Nagle-Smith as VP of Customer Fulfillment

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

Study: Top Reason for Cancelling Streaming Service Is Programming

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

U.K.'s Digital TV Group Taps Leslie Mackenzie as Chair

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

Amagi becomes the first cloud-native SaaS company offerin...

Amagi Media Labs Limited, the only cloud-native SaaS provider offering end-to-end solutions across live production, content preparation, distribution, and monet...

27/01/2026

Miri Technologies Launches V410 Live 4K Video Encoder-Dec...

Miri Technologies Inc. today unveiled its V410 live 4K video encoder/decoder for streaming, IP-based production workflows and AV-over-IP distribution. Designed ...

27/01/2026

UML Tsongas Center Upgrades with Ikegami UHK-X600 Cameras

The Paul E. Tsongas Center at UMass Lowell in Massachusetts has chosen Ikegami cameras for incorporation into its broadcast-quality television production facili...

27/01/2026

SipRadius provides simplicity and security for AV content...

SipRadius, specialists in secure, low latency media transport, will show the pro AV community its end-to-end solution for high quality content distribution and ...

27/01/2026

Hiltron to Showcase its Full Range of Satcom Solutions at...

Hiltron will highlight the fruits of its 45-plus years of experience in satellite communications system integration and product development at the annual GovSat...

27/01/2026

Broadcast and pro AV professionals cut costs and single-u...

Klvr will demonstrate how its Charger Pro solution is delivering cost reduction and performance benefits for live entertainment and pro AV markets at ISE 2026 (...

27/01/2026

Amino Showcases Single Player Digital Signage and Enterpr...

Amino, a global media technology leader in enterprise video and digital signage, today announced that its Integrated Systems Europe (ISE) 2026 booth (4N700) wil...

27/01/2026

ANI Expands Strategic Partnership with LiveU

ANI, India's leading news agency and a major content provider across the Asia Pacific region, today announced a significant expansion of its long standing p...

27/01/2026

VisualOn and The Channel Store Demonstrate Up to 65 Perce...

Independent analysis confirms faster encoding, lower CPU usage, and improved perceptual quality compared to CBR and CRF VisualOn, a leader in multimedia playba...

27/01/2026

EVS Officially Launches eShop

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

Miri to Showcase New V410 Video Encoder/Decoder at ISE 2026

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

FCC Settles Two Investigations, Renews Three TV Station Licenses

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

UML Tsongas Center Upgrades to Ikegami UHK-X600 Cameras

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

27/01/2026

Australian Defence Force Secures Satellite Communications on SES IS-22

New agreement for uninterrupted UHF connectivity for Australian Defence Force through 2033, With Options Extending to 2041 Luxembourg, January 13, 2026 - Satel...

27/01/2026

SNL UK creative team expands as Pauli Lovejoy, Annie Hardinge and Kevin Fortune join series

Tuesday 27 January 2026 SNL UK creative team expands as Pauli Lovejoy, Annie Ha...

27/01/2026

When It Feels Real, Is It Still a Lie? The Art of Sarah' Premieres February 13

Back to All News When It Feels Real, Is It Still a Lie? The Art of Sarah' ...