Sony Pixel Power calrec Sony

NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools

06/11/2024

www.1x.tech

Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.

The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.

Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.

Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.

Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.

Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:

GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments

GR00T-Mimic for robot motion and trajectory generation

GR00T-Dexterity for robot dexterous manipulation

GR00T-Control for whole-body control

GR00T-Mobility for robot locomotion and navigation

GR00T-Perception for multimodal sensing

Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.

New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.

NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.

NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.

Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.

NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.

Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: https://blogs.nvidia.com/blog/robot-learning-humanoid-development/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

21/01/2026

Wisycom Introduces New Wideband Antenna Matrix MATF for C...

Wisycom, a global leader in advanced wireless RF solutions, launches its new wideband antenna matrix, MATF, which supports RF and fiber for demanding multi-zone...

21/01/2026

Grass Valley Showcases Scalable Future-Ready Live Product...

Grass Valley will demonstrate how it is powering scalable, future-ready live production at FOMEX 2026, taking place February 2 4 in Riyadh, Saudi Arabia. Exhibi...

21/01/2026

OKAST Implements BCNEXXT Vipe to Accelerate Monetization...

BCNEXXT, the developers of the advanced playout platform Vipe, today announced that OKAST, the monetization-first OTT platform provider, is using BCNEXXT's ...

21/01/2026

Magewell Introduces New IP to HDMI Decoder at ISE 2026 an...

Revamped design enables advanced capabilities, leading with powerful IP to HDMI conversion Magewell, developer of innovative, high-performance video I/O and I...

21/01/2026

Kiloview Expands Its AV-over-IP Ecosystem with AVX24-4 Me...

Jan 20th 2026, Changsha Kiloview today announced the launch of two major additions to its AV-over-IP ecosystem: the AVX24-4 Media HUB and KiloLink Station, ma...

21/01/2026

Bitfocus delivers high availability high security and hig...

Latest version of enterprise-class Buttons brings simple, coherent control to more than 700 professional devices and applications Bitfocus, the specialist in ...

21/01/2026

Clear-Com Announces Kari Eythorsson as New Regional Sale...

Clear-Com is pleased to announce the appointment of Kari Eythorsson as the new Regional Sales Manager (RSM) for Southeast Asia & Australia, based in Singapore,...

21/01/2026

Netflix Amends Warner Bros Discovery Bid to Offer All Cash

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

21/01/2026

Advanced Systems Group Inks Exclusive Partnership With Nu-Studio

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

21/01/2026

SWIT Ships New Powercell Camera Battery Series

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

21/01/2026

Netflix Q4 Earnings Exceed Expectations as Subs Top 325 Million

Share Share by: Copy link Facebook X Linkedin Pinterest Bluesky Email...

21/01/2026

RT News announces new Washington Correspondent

RT News is pleased to announce the appointment of Jackie Fox as its new Washington Correspondent. Jackie is a multimedia journalist with RT News and has repor...

21/01/2026

Largest Infrastructure Buildout In Human History: Jensen Huang on AI's Five-Layer Cake at Davos

From skilled trades to startups, AI's rapid expansion is the beginning of th...

20/01/2026

SVG Sit-Down: BitFire's Jim Akimchuk on the Future of Cloud, Remote, Software-Defined Live Production

SVG Sit-Down: BitFire's Jim Akimchuk on the Future of Cloud, Remote, Softwar...

20/01/2026

From Architecture to Reality: What CBC's Milano Cortina 2026 Workflows Tell Us About DMF and MXL in Live Sport

From architecture to reality: What CBC's Milano Cortina 2026 workflows tell ...

20/01/2026

Milano Cortina 2026: Making the Winter Olympics and Paralympics Workflow Work for France Tlvisions

Milano Cortina 2026: Making the Winter Olympics and Paralympics workflow work fo...

20/01/2026

Countdown to Milano Cortina 2026: SVG Launches SportsTechLive Blog in Lead-up to Winter Games

Countdown to Milano Cortina 2026: SVG Launches SportsTechLive Blog in Lead-up to...

20/01/2026

Grand Finale: Rally Saudi Arabia Pushes WRC to Make the Most of Cameras and Drones for Sandy Spectacle

Grand finale: Rally Saudi Arabia pushes WRC to make the most of cameras and dron...

20/01/2026

State of Change: What's Happening in the Remote-Production Business?

State of Change: What's Happening in the Remote-Production Business?NEP Group, Game Creek Video, Dome Productions, Mobile TV Group, Program Productions exec...

20/01/2026

CFP National Championship 2026: A Look Back at SVG's Complete Coverage

CFP National Championship 2026: A Look Back at SVG's Complete CoverageGo behind the scenes with ESPN, Game Creek Video, Van Wagner, and the champion Indiana...

20/01/2026

Centering the Artist: Why We're Launching the AI Literacy Initiative

The Gotham, Film Independent, and Creators Coalition on AI Join Alliance by Michelle Satter, Founding Senior Director, Sundance Institute's Artist Programs...

20/01/2026

19th Annual South African Film and Television Awards (SAFTAs) open the call for Discretionary Award Nominations

The National Film and Video Foundation (NFVF), an agency of the Department of Sp...

20/01/2026

The Gauge: Poland | December 2025

December, the first month of winter and the holiday season, has traditionally encouraged families to spend time together in front of the television. During this...

20/01/2026

The Gauge: Mexico December 2025

During December, streaming's share of TV viewing in Mexico settled at 24.3%, an increase of 0.1 share points from the previous month. Disclaimer: YUMI TV,...

20/01/2026

Streaming Shatters Multiple Records in December 2025 with 47.5% of TV Viewing, according to Nielsen's The Gauge

Christmas Day is Most-Streamed Day Ever with 55 Billion Viewing Minutes, led by...

20/01/2026

Nielsen: Streaming Shatters Multiple Records in December 2025

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/01/2026

Disguise and ASB GlassFloor Announce Official Partnership...

Disguise and ASB GlassFloor today announced a strategic partnership that turns the arena floor into a fully customizable interactive digital surface. This will ...

20/01/2026

LiveU Enables Cloud Production Workflow for First Ever IT...

Last summer, ITV celebrated the tenth anniversary of its popular reality dating show, Love Island. To ramp up the excitement of series 12 (launched in the UK on...

20/01/2026

Berklee's Mark Ethier Named One of SPIN's Most Influential People in Music for 2026

Berklee's Mark Ethier Named One of SPIN's Most Influential People in Mus...

20/01/2026

Sports Illustrated Launches New Streaming Channel SI TV

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/01/2026

S&P: Funding for GenAI Applications Nearly Tripled in 2025

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/01/2026

Emergent to Showcase AI-First Broadcast Graphics Platform...

Emergent, a provider of AI-powered media production solutions, today announced its presence at ISE 2026 in Barcelona, taking place February 3 6, where it will u...

20/01/2026

Netflix and Warner Bros. Discovery Amend Agreement to All-Cash Transaction

Back to All News Netflix and Warner Bros. Discovery Amend Agreement to All-Cash Transaction Business 20 January 2026 Global Link copied to clipboard All-C...

20/01/2026

Now You Can Play a Role in Our Live Events With Real-Time Voting

Back to All News Now You Can Play a Role in Our Live Events With Real-Time Voting Elmar Nubbemeyer VP, Member Product Product 20 January 2026 Global Link...

19/01/2026

CFP National Championship 2026: Indiana Football's Digital Crew Generate Hype Around Undefeated Run to Miami

CFP National Championship 2026: Indiana Football's Digital Crew Generate Hyp...

19/01/2026

CFP National Championship 2026: Van Wagner Relies on Experience for Indiana-Miami Matchup at Hard Rock Stadium

CFP National Championship 2026: Van Wagner Relies on Experience for Indiana-Miam...

19/01/2026

CFP National Championship 2026: Game Creek Video Fields Premiere IP Compound for ESPN Game, Studio Coverage

CFP National Championship 2026: Game Creek Video Fields Premiere IP Compound for...

19/01/2026

CFP National Championship 2026: For a Slew of Studio Shows, ESPN Turns Hard Rock Stadium Into a Production Campus

CFP National Championship 2026: For a Slew of Studio Shows, ESPN Turns Hard Rock...

19/01/2026

CFP National Championship 2026: ESPN's Sweeping Live Game Production Leverages Deep Specialty Camera Lineup, IP-Connected Compound

CFP National Championship 2026: ESPN's Sweeping Live Game Production Leverag...

19/01/2026

The NOVA Binocular System: The Future of Night Vision

The new L3Harris NOVA binocular system is the every-soldier goggle' - and the beginning of a new era for night-vison capability....

19/01/2026

Nielsen: Two-Thirds of Black Consumers Pay More Attention To Ads That Reflect Their Culture

The Majority of Those Surveyed Prefer to Buy Brands Who Advertise in Content Tha...

19/01/2026

Straight to Hell' Sets April 27 Premiere: Teaser Trailer, Art, and Additional Cast Revealed

Back to All News Straight to Hell' Sets April 27 Premiere: Teaser Trailer,...

19/01/2026

Broadcast Pix Releases Version 10.3 Software

New Workflow Enhancements for All Broadcast Pix Systems, Plus 4-Channel ISO Recording for Hybrid and Roadie Tyngsboro, Mass. - January 2025 - Broadcast Pix ann...

19/01/2026

Sky News reinvents News at Ten for the modern audience with The Wrap

Monday 19 January 2026 Sky News reinvents News at Ten for the modern audience with The Wrap Sky News today announces The Wrap, a bold evolution of its 10pm ou...

19/01/2026

Netflix Announces the Series 'Nights & Days,' Directed by Kamila Tarabura

Back to All News Netflix Announces the Series Nights & Days, Directed by Kamila...