Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

16/07/2026

Complete spectrum of innovation from British exhibitors a...

33 leading exhibitors are taking part in the GREAT Britain and Northern Ireland Pavilions and linked locations across IBC2026 (Amsterdam RAI, 11 14 September)...

16/07/2026

Hitomi Broadcast to showcase HDR colour verification and...

Hitomi Broadcast will introduce Spectra, a new HDR colour verification solution measured in picture, not using VPID, at IBC2026 (Hall 10, Stand A40). Expanding ...

16/07/2026

SMPTE Kicks Off 110th Anniversary Campaign

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

Cox Media Group Taps Waymark for AI Ad Creation Tools

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

Parks: Cable Broadband Sub Losses Slow Down

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

CatchLight, Sony Launch Initiative to Help Local Newsrooms

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

XR Launches XR ONE Ad Ops Platform

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

Zixi and Comcast Technology Solutions Partner to Provide...

Zixi, a leader in live video delivery and workflow orchestration, and Comcast Technology Solutions (CTS) today announced a strategic new integration to deliver ...

16/07/2026

Blackmagic Design Cameras Capture Immersive Flight for New Cirrus App

Blackmagic Design Cameras Capture Immersive Flight for New Cirrus App Brie Clayton July 15, 2026 0 Comments Leader in Personal Aviation taps URSA Cin...

16/07/2026

XenData Announces LTO Archive Appliances with both File and S3 Object Storage Interfaces

XenData Announces LTO Archive Appliances with both File and S3 Object Storage In...

16/07/2026

FIFA World Cup semi-finals deliver whopper audiences as RT Player surpasses 21 million streams across the tournament to date

Together the semi-finals had a combined reach of 1.98 million unique individuals...

15/07/2026

Inside DAZNs Multi-Market World Cup Effort With CTO Sandeep Tiku

The 2026 FIFA World Cup is a big deal for DAZN, which has distribution rights in three markets: Italy, Spain, and Japan. Each requires a different editorial app...

15/07/2026

SVG New Sponsor Spotlight: Qumulo's Chris Horn on Powering Global Media Workflows Across Edge, Core, and Cloud

As sports-media organizations manage rapidly growing content libraries and incre...

15/07/2026

FIFA World Cup 2026: Inside the HBS Non-Live London Production Hub

HBS director of digital Johannes Franken talks SVG Europe through the operation built to serve FIFA World Cup 2026 rights holders remotely from Stratford, east ...

15/07/2026

IBC2026 to Spotlight AI, Live Sports Production, Creator Economy, and Trust in Media

IBC has unveiled the conference program for IBC2026, with this year's event ...

15/07/2026

Vanessa Paradis Tour Adopts Sennheiser Spectera Wireless Ecosystem for 2026 Production

Software-defined wideband RF platform supports 96-kHz monitoring, streamlined fr...

15/07/2026

Telos Alliance Integrates AudioShake Source Separation Into Minnetonka AudioTools Server

New AI-powered module enables broadcasters and content owners to extract dialogu...

15/07/2026

AIMS Schedules Second IPMX Product Certification Event for August

Weeklong gathering at EBU headquarters will test new IPMX capabilities and expand certified product ecosystem...

15/07/2026

Force Tech Appoints Bryan Adams as Vice President of Integration

Force Tech has announced the appointment of Bryan Adams, CTS, as Vice President of Integration. Adams will oversee AV and broadcast integration project delivery...

15/07/2026

USA Sports and Bundesliga Announce Exclusive Multi-Year U.S. Media Rights Agreement

USA Sports has announced an exclusive English-language agreement with the Bundes...

15/07/2026

Inside ITV Sport's Stunning FIFA World Cup Studio in New York City With Paul McNamara

ITV Sport's FIFA World Cup 2026 coverage has been produced from one of the m...

15/07/2026

CatchLight, Sony Launch Equipment Initiative for Local Visual Journalists

Program will provide professional camera kits, training, and technical support to participating local newsrooms...

15/07/2026

CornerstoneSF Installs FOR-A HVS-Q12 Switcher in New UHD Production Facility

San Francisco church becomes first U.S. deployment of the 2 M/E production switcher...

15/07/2026

Steve Bornstein Joins Advisory Board of Scott Cokers New Global MMA Promotion

Former ESPN and NFL Network executive to advise startup league on media, distribution, technology, and fan engagement...

15/07/2026

Space City Home Network Executive Producer John Valek on Leaning Into Innovation, Access, and Opportunity

The Houston-based RSN is the live-game home of the Astros and Rockets...

15/07/2026

Freqport unveil the Dual FreqInOut FO1-2RK

Launched alongside new FreqSoft installer Designed to bridge the gap between hardware and software, Freqport's FreqInOut FO1 unit provides a simple way ...

15/07/2026

Expressive E's Summer Offers

Save up to 55% on software until 23 July 2026 Expressive E have just launched their Summer Offer sale, with discounts of up to 55% now applied across their ...

15/07/2026

Peel Stems 2 from zplane

Real-time stem-separation tool updated zplane have recently released a new and improved version of their real-time stem-separation plug-in which they say ma...

15/07/2026

Baby Audio SubCulture

A new benchmark in bass enhancement Described as a new benchmark in bass enhancement , Baby Audio's latest release has been designed to augment the lo...

15/07/2026

DMC Production Reimagines Live OB with Calrec

DMC Production's AR 1 and AR 2 units reverse the conventional OB workflow, keeping all production-critical processing at the venue while operators work remo...

15/07/2026

FCC to Vote on Replacing National Broadcast Ownership Cap

Share Copy link Facebook X Linkedin Bluesky Email...

15/07/2026

ATSC Updates Industry Audio Loudness Guidance

Share Copy link Facebook X Linkedin Bluesky Email...

15/07/2026

GigaCloud and Cubbit partner to deliver sovereign and res...

GigaCloud, a leading Ukrainian cloud service provider, and Cubbit, the first geo-distributed cloud storage enabler, today announced a new Business Alliance Part...

15/07/2026

SMPTE Launches 110th Anniversary Campaign to Advance Its...

SMPTE, the global society for media professionals, technologists, and engineers, has announced the launch of the SMPTE 110th Anniversary Campaign, a fundraising...

15/07/2026

STUDIO TECHNOLOGIES UNVEILS MODEL 762 STEREO LEVEL CONTRO...

Studio Technologies, a leading manufacturer of high-quality audio, video, and fiber-optic solutions, unveils the new StudioComm Model 762 Stereo Level Control, ...

15/07/2026

COW Jobs: Producer, Virtual Events - Hybrid, Full Time, Salaried

COW Jobs: Producer, Virtual Events - Hybrid, Full Time, Salaried Brie Clayton July 15, 2026 0 Comments Producer, Virtual Events (Hybrid | Full-time | ...

15/07/2026

SIMBA powers new streaming delivery network with Broadpea...

Brazilian media organization builds nationwide CDN with Broadpeak's high-performance software for low-latency streaming and new content distribution service...

15/07/2026

DHD Introduces Discovery Utility for Network-Connected Au...

DHD announces a new utility designed to increase the efficiency of its audio mixing systems during initial installation and subsequent configuration. Anyone w...

15/07/2026

BCNEXXT Brings Next-Generation Vipe to IBC2026 Breaking t...

BCNEXXT Brings Next-Generation Vipe to IBC2026, Breaking the 1:1 Channel-to-Server Ratio New innovations liberate broadcasters from legacy playout infrastructur...

15/07/2026

S&P Analysis: Three Quarters of Americans Watch Live Sports

Share Copy link Facebook X Linkedin Bluesky Email...

15/07/2026

Scripps Sports, Ion Score Women's Volleyball Rights

Share Copy link Facebook X Linkedin Bluesky Email...

15/07/2026

NVIDIA Introduces New Jetson Thor Computers to Advance Mainstream Robotics and Edge AI

General-purpose robots and autonomous machines are moving from research labs to ...

15/07/2026

Screeners and press pack now available: Gomorrah: The Origins premieres on Sky and NOW on 28 July

Wednesday 15 July 2026 Screeners and press pack now available: Gomorrah: The Or...

15/07/2026

Audiences to get behind-the-scenes access to Trinity College Dublin's Old Library in new RT series

Trinity's Treasures features Ruth Negga, Eleanor McEvoy, and David Norris, a...

15/07/2026

Building Ireland explores the River Shannon region from Leitrim to Limerick in a new series

Join geographer Dr Susan Hegarty, Engineer Tim Joyce and Architect Orla Murphy a...