Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

08/07/2026

Aqueduct Racetrack Goes Silent

The end of the Big A' marks the demise of one more iconic sports venue and its signature sound After 132 years as one of the preeminent horse-racing track...

08/07/2026

Memphis Athletics, Learfield Continue Long-Term Partnership

Memphis Athletics and Learfield have signed a long-term renewal to their 25-year relationship, which sees Learfield's Tiger Sports Properties continue repre...

08/07/2026

T-Mobile Brings ABS Challenge System, 5G Batting Practice Cams to 2026 MLB All-Star Game in Philadelphia

MLB All-Star Week returns to Philadelphia for the first time in 30 years as Amer...

08/07/2026

SVG Enterprise Launches Inaugural Enterprise Video Summit, Oct. 19 in New York

Former Wall Street Video event expanded to include all Enterprise sectors SVG Enterprise has announced the inaugural Enterprise Video Summit, taking place Oct....

08/07/2026

Wrigley Field Revamp: Chicago Cubs Upgrade LED Videoboards To Match New 1080p Control Room

An offseason project enhanced eight video displays around the venue in time for ...

08/07/2026

A Beacon on Biscayne Bay: Miami Heat Reflect on Ambitious Renovation at SVG Venues & Teams Summit

Kaseya Center has a ton of new features that weren't available to the indust...

08/07/2026

FIPRESCI Establishes Jury at Sundance Film Festival

The International Federation of Film Critics (FIPRESCI) and the Sundance Film Festival are pleased to announce the establishment of a FIPRESCI jury beginning wi...

08/07/2026

Universal Audio introduce Black Box HG-2 & elysia karacter

Latest analogue-emulation plug-ins revealed Universal Audio have just added two new analogue-emulation plug-ins to their DSP-powered UAD line-up. Developed ...

08/07/2026

Pitch Innovations release Loop2Kit

Turn any audio loop into a playable kit Pitch Innovations have recently introduced an interesting new tool that's capable of creating playable kits from...

08/07/2026

MonkeyC update Rando sampler plug-in

Adds new keyboard-split function MonkeyC have recently launched an update that overhauls their powerful software sampler. Described as the sampler that mad...

08/07/2026

Strymon unveil the TimeLine MX

Four new delay algorithms & dual-engine functionality Strymon have just announced that the much-anticipated MX version of their hugely popular delay pedal i...

08/07/2026

Well over half of Australia have tuned in to watch the FIFA World Cup on SBS

Well over half of Australia have tuned in to watch the FIFA World Cup on SBS 7 July, 2026 Media releases Australia's Round of 32 match against Egypt a...

08/07/2026

SBS and Koorie Heritage Trust announce three-year partnership to celebrate First Nations cultures and communities

SBS and Koorie Heritage Trust announce three-year partnership to celebrate First...

08/07/2026

Rohde & Schwarz and China Mobile Research Institute jointly demonstrate generative AI user experience testing

Rohde & Schwarz and China Mobile Research Institute jointly demonstrate generati...

08/07/2026

Fox TV Stations Bypass Satellite for Live Game Show Run

Share Copy link Facebook X Linkedin Bluesky Email...

08/07/2026

Fox Television Stations Extreme Reach and LTN Bring Live...

Collaboration demonstrates how managed IP distribution and cloud-based syndication workflows can support live television at national scale Fox Television Stati...

08/07/2026

New ARRI ALEXA 35 Live Xtreme camera seamlessly delivers up to 8x HFR via SDI for live productions

New ARRI ALEXA 35 Live Xtreme camera seamlessly delivers up to 8x HFR via SDI fo...

08/07/2026

Did You Know? Redshift for Cinema 4D: The Power of the Distance Node

Did You Know? Redshift for Cinema 4D: The Power of the Distance Node Marc Potocnik July 8, 2026 0 Comments The Did You Know? Redshift for Cinema 4D ...

08/07/2026

LARA Wizard Touchpanel gets major configuration upgrade

Lightware is introducing the biggest update yet to the LARA Wizard Touchpanel configuration experience, designed to help integrators work faster while deliverin...

08/07/2026

DMC Production Reimagines Live OB with Calrec

DMC Production has turned to Calrec in its quest to redefine outside broadcasting with a flexible approach to live production that reverse engineers the convent...

08/07/2026

FCC Probe of 'The View' Racks Up 77,611 Comments

Share Copy link Facebook X Linkedin Bluesky Email...

08/07/2026

NAB Releases New Radio and TV Station Self-Inspection Guides

Share Copy link Facebook X Linkedin Bluesky Email...

08/07/2026

Ross Production Services Standardizes on Clear-Com Intercom Systems

Share Copy link Facebook X Linkedin Bluesky Email...

08/07/2026

Reel FX Animation Adopts Foundrys Flix on Doctor Dolittle...

Foundry today announced that Reel FX Animation Studios has adopted Flix to standardize their pipeline and streamline processes across departments. Flix is a vis...

08/07/2026

The right team. The right time.' How The Killings at Parrish Station stuck to their creative guns

The right team. The right time.' How The Killings at Parrish Station stuck t...

08/07/2026

Australians in Film and Screen Australia launch FUTURE VISION Forward to elevate Australian television to the global stage

Australians in Film and Screen Australia launch FUTURE VISION Forward to elevate...

08/07/2026

RT to air award-winning documentary TESTIMONY

Film follows the courageous fight for justice by survivors of Ireland's Magdalene Laundries & Mother and Baby Homes TESTIMONY airs Wednesday 22 July at 9...

08/07/2026

NVIDIA Nemotron Achieves Benchmark-Leading Performance With LangChain Deep Agents Harness

NVIDIA Nemotron 3 Ultra is offering leading performance at lower cost than top c...

07/07/2026

Open call: Apply for the Media Innovation Europe Validation Booster

Applications are now open for the Validation Booster, a six-month acceleration programme supporting journalists, media organisations, content creators, media in...

07/07/2026

HAND Issues Persistent Digital IDs to 50 Top 2026 MLB Draft Prospects

The Universal Talent Identifier (HAND) has issued HAND IDs to 50 top prospects ahead of the 2026 MLB Draft, including consensus top-ranked players Roch Cholowsk...

07/07/2026

Cisco and McLaren Mastercard Formula 1 Team Extend Technology Partnership

Cisco and the McLaren Mastercard Formula 1 Team have announced an extension of their partnership, expanding the scope from networking and communications to incl...

07/07/2026

LucidLink Announces Enhanced Integration with Adobe Frame.io

LucidLink has announced a workflow integration with Adobe Frame.io that gives production teams unified access to Frame.io projects alongside supporting producti...

07/07/2026

Chyron Appoints Ahmed Magd as Middle East Sales Director

Chyron has announced the appointment of Ahmed Magd as Sales Director for the Middle East. Magd joins from previous senior commercial roles at Riedel Communicati...

07/07/2026

Leader Electronics T&M Equipment Used in Live Broadcast of Pope Leo XIVs Visit to Sagrada Famlia

Leader Electronics of Europe has announced that its PHABRIX Qx rasterizer and Le...

07/07/2026

eCLUTCH Secures EMEA and MENA Broadcast Rights for Esports World Cup 2026

eCLUTCH has announced it has acquired broadcast rights to the Esports World Cup 2026 for EMEA and MENA, covering the Middle East, Africa, the Balkans, Sweden, a...

07/07/2026

Riedel and ARRI Support Cinematic Live Production at Eurovision Song Contest 2026

Riedel Communications and ARRI supported ORF's host broadcast of the Eurovis...

07/07/2026

ARRI Releases ALEXA 35 Live Xtreme With High-Frame-Rate SDI Output for Live Production

ARRI has announced the ALEXA 35 Live Xtreme, a camera that adds high frame rate ...

07/07/2026

IBC 2026: LiveU To Exhibit With LU900Q, Sony Integration, EcoSystem Expansion

LiveU will exhibit at IBC 2026 (Stand 7.C24), showcasing the LU900Q Intelligent Production Unit, expanded Sony collaboration, and additions to its open IP-video...

07/07/2026

Peacock Advances Spanish-Language World Cup Streaming With Visin de Campo, Dolby Vision, Real-Time Highlights

Telemundo's coverage is packaged with Multiview, Catch Up with Key Plays, in...

07/07/2026

Australian Rugby League Commission Secures $5.3 Billion Media Rights Deal with Foxtel, Nine, and Sky NZ

The Australian Rugby League Commission (ARLC) has announced a seven-year media r...

07/07/2026

BYUtv Partners With Big 12 for Football Media Days Production

Produced onsite at The Star in Frisco, TX, live coverage of the two-day event airs on ESPN platforms The season for Football Media Days is here, and the Big ...

07/07/2026

Rogers To Become 100% Owner of Maple Leaf Sports & Entertainment

Rogers Communications Inc. has announced an agreement to acquire the remaining 25% ownership stake in Maple Leaf Sports and Entertainment (MLSE) from Kilmer Spo...

07/07/2026

Mobile TV Groups Nick Garvin on Success of MTVG Production Platform at New Mountain Media Center

End-to-end broadcast solution includes fleet of mobile units, Cloud Control, and...

07/07/2026

SVG Executive Perspectives 2026: Industry Leaders Discuss Biz and Tech Trends for the Year Ahead

Sports-production-tech experts sound off on the state of the industry...

07/07/2026

Mix Buddy instrument mixer from Heritage Audio

New mixer aimed squarely at instrument-level sources Heritage Audio have clearly been busy recently, as alongside their all-new Tubestrip, they've also ...

07/07/2026

Heritage Audio unveil the Tubestrip

New channel strip combines popular hardware designs The latest arrival to the Heritage Audio line-up brings together a selection of their praised hardware d...

07/07/2026

Clear-Com Powers Ross Production Services Mobile Productions

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({ content_source:......