Sony Pixel Power calrec Sony

Hugging Face and NVIDIA to Accelerate Open-Source AI Robotics Research and Development

06/11/2024

At the Conference for Robot Learning (CoRL) in Munich, Germany, Hugging Face and NVIDIA announced a collaboration to accelerate robotics research and development by bringing together their open-source robotics communities.

Hugging Face's LeRobot open AI platform combined with NVIDIA AI, Omniverse and Isaac robotics technology will enable researchers and developers to drive advances across a wide range of industries, including manufacturing, healthcare and logistics.

Open-Source Robotics for the Era of Physical AI The era of physical AI - robots understanding physical properties of environments - is here, and it's rapidly transforming the world's industries.

To drive and sustain this rapid innovation, robotics researchers and developers need access to open-source, extensible frameworks that span the development process of robot training, simulation and inference. With models, datasets and workflows released under shared frameworks, the latest advances are readily available for use without the need to recreate code.

Hugging Face's leading open AI platform serves more than 5 million machine learning researchers and developers, offering tools and resources to streamline AI development. Hugging Face users can access and fine-tune the latest pretrained models and build AI pipelines on common APIs with over 1.5 million models, datasets and applications freely accessible on the Hugging Face Hub.

LeRobot, developed by Hugging Face, extends the successful paradigms from its Transformers and Diffusers libraries into the robotics domain. LeRobot offers a comprehensive suite of tools for sharing data collection, model training and simulation environments along with designs for low-cost manipulator kits.

NVIDIA's AI technology, simulation and open-source robot learning modular framework such as NVIDIA Isaac Lab can accelerate the LeRobot's data collection, training and verification workflow. Researchers and developers can share their models and datasets built with LeRobot and Isaac Lab, creating a data flywheel for the robotics community.

Scaling Robot Development With Simulation Developing physical AI is challenging. Unlike language models that use extensive internet text data, physics-based robotics relies on physical interaction data along with vision sensors, which is harder to gather at scale. Collecting real-world robot data for dexterous manipulation across a large number of tasks and environments is time-consuming and labor-intensive.

Making this easier, Isaac Lab, built on NVIDIA Isaac Sim, enables robot training by demonstration or trial-and-error in simulation using high-fidelity rendering and physics simulation to create realistic synthetic environments and data. By combining GPU-accelerated physics simulations and parallel environment execution, Isaac Lab provides the ability to generate vast amounts of training data - equivalent to thousands of real-world experiences - from a single demonstration.

Generated motion data is then used to train a policy with imitation learning. After successful training and validation in simulation, the policies are deployed on a real robot, where they are further tested and tuned to achieve optimal performance.

This iterative process leverages real-world data's accuracy and the scalability of simulated synthetic data, ensuring robust and reliable robotic systems.

By sharing these datasets, policies and models on Hugging Face, a robot data flywheel is created that enables developers and researchers to build upon each other's work, accelerating progress in the field.

The robotics community thrives when we build together, said Animesh Garg, assistant professor at Georgia Tech. By embracing open-source frameworks such as Hugging Face's LeRobot and NVIDIA Isaac Lab, we accelerate the pace of research and innovation in AI-powered robotics.

Fostering Collaboration and Community Engagement The planned collaborative workflow involves collecting data through teleoperation and simulation in Isaac Lab, storing it in the standard LeRobotDataset format. Data generated using GR00T-Mimic, will then be used to train a robot policy with imitation learning, which is subsequently evaluated in simulation. Finally, the validated policy is deployed on real-world robots with NVIDIA Jetson for real-time inference.

The initial steps in this collaboration have already been taken, having shown a physical picking setup with LeRobot software running on NVIDIA Jetson Orin Nano, providing a powerful, compact compute platform for deployment.

Combining Hugging Face open-source community with NVIDIA's hardware and Isaac Lab simulation has the potential to accelerate innovation in AI for robotics, said Remi Cadene, principal research scientist at LeRobot.

This work builds on NVIDIA's community contributions in generative AI at the edge, supporting the latest open models and libraries, such as Hugging Face Transformers, optimizing inference for large language models (LLMs), small language models (SLMs) and multimodal vision-language models (VLMs), along with VLM's action-based variants of vision language action models (VLAs), diffusion policies and speech models - all with strong, community-driven support.

Together, Hugging Face and NVIDIA aim to accelerate the work of the global ecosystem of robotics researchers and developers transforming industries ranging from transportation to manufacturing and logistics.

Learn about NVIDIA's robotics research papers at CoRL, including VLM integration for better environmental understanding, temporal navigation and long-horizon planning. Check out workshops at CoRL with NVIDIA researchers.
LINK: https://blogs.nvidia.com/blog/hugging-face-lerobot-open-source-robotic...
See more stories from nvidia

North America Stories

05/05/2026

ARRI unveils Omnibar: compact, modular, battery-powered IP65 LED bars with precise pixel control

ARRI unveils Omnibar: compact, modular, battery-powered IP65 LED bars with preci...

05/05/2026

NBC Sports' NBA Playoff Viewership Up 58%

Share Copy link Facebook X Linkedin Bluesky Email...

05/05/2026

U.S. Court Upholds Some Patents in LG ATSC 3.0 Infringement Case

Share Copy link Facebook X Linkedin Bluesky Email...

05/05/2026

Gray Media and Allen Media Group Close Station Transactions

Share Copy link Facebook X Linkedin Bluesky Email...

05/05/2026

Digital Domain Welcomes Award-Nominated VFX Supervisor Jelmer Boskma

Digital Domain Welcomes Award-Nominated VFX Supervisor Jelmer Boskma Brie Clayton May 4, 2026 0 Comments Digital Domain, a global leader in visual eff...

04/05/2026

SVG Sit-Down: NEP Americas Mike Werteen on How Great Tech, Better People Drive Success

Hardware is still an emphasis - Supershooter 11 is new, and REMI-based 65 is in ...

04/05/2026

Beyond 90 Minutes: How K League's Soccer Blueprint for Growth Has Lessons for Everyone

Head of International Business Development Min Joo Kim explores the league's...

04/05/2026

Audio-Technica ATND1061 and ATUC Discussion Systems Certified for Crestron Automate VX

Audio-Technica has announced that its ATND1061 ceiling array microphone and ATUC...

04/05/2026

Triple B Media Launches Bowling TV, a 24/7 FAST Channel Dedicated to Bowling

Triple B Media has launched Bowling TV, a free ad-supported television (FAST) channel dedicated to bowling. The channel is available on Prime Video, LG Channels...

04/05/2026

PlayMetrics Acquires SportsEngine from Versant

PlayMetrics, a provider of operations management software for youth sports organizations, has announced the completion of its acquisition of substantially all t...

04/05/2026

IHSE GmbH Appoints Dr. Thomas Niessen as CEO

IHSE GmbH has announced that Dr. Thomas Niessen has joined as CEO and Managing Director, effective May 1, 2026. He joins Frank Breitenfelder, who has served as ...

04/05/2026

PMY Group Deploys Optic Crowd Intelligence Platform at Australian Formula 1 Grand Prix

PMY Group deployed its AI-powered crowd intelligence platform, Optic, at the For...

04/05/2026

Behind The Mic: Stephen A. Smith and Skip Bayless to Reunite for First Take Episode; Donna Brothers Worked Final Kentucky Derby

Behind The Mic provides a roundup of recent news regarding on-air talent, includ...

04/05/2026

SAG-AFTRA, Studios Reach Tentative Agreement

Share Copy link Facebook X Linkedin Bluesky Email...

04/05/2026

Study: Paramount-WBD Deal Signals New Era of Streaming Scale

Share Copy link Facebook X Linkedin Bluesky Email...

04/05/2026

Student Spotlight: Joshua Griffin

Student Spotlight: Joshua Griffin The New Orleans native, who was named the 2026 student commencement speaker for Boston Conservatory at Berklee, talks about ...

03/05/2026

Introducing the new Mistika Workflows Suite: transformative and cost-effective for every user

Introducing the new Mistika Workflows Suite: transformative and cost-effective f...

03/05/2026

Introducing the new Mistake Workflows Suite: transformative and cost-effective for every user

Introducing the new Mistake Workflows Suite: transformative and cost-effective f...

03/05/2026

Filming begins on the third and final season of Breathless

Back to All News Filming begins on the third and final season of Breathless Entertainment 03 May 2026 GlobalSpain Link copied to clipboard Discover the vi...

02/05/2026

Release Rundown: What to Watch in May, From Saccharine to Tuner

(L-R) Dustin Hoffman and Leo Woodall appear in Tuner by Daniel Roher, an official selection of the 2026 Sundance Film Festival. (Photo courtesy of Sundance In...

02/05/2026

FCC Releases Tentative Agenda for May Open Meeting

Share Copy link Facebook X Linkedin Bluesky Email...

02/05/2026

Sinclair Remains Bullish on Station M&A

Share Copy link Facebook X Linkedin Bluesky Email...

02/05/2026

NABLF Announces 2026 Broadcast Leadership Training Award Winners

Share Copy link Facebook X Linkedin Bluesky Email...

02/05/2026

Gravity Media Taps Custom Consoles for Work on Production Center

Share Copy link Facebook X Linkedin Bluesky Email...

02/05/2026

May 01, 2026

Scripps Research immunologist Dennis Burton elected to American Academy of Arts and Sciences A leader in broadly neutralizing antibodies, Burton has helped driv...

01/05/2026

Ratings Roundup: NBA Playoffs Return to NBC Sports up 38%; NFL Draft Down 12% Overall From 2025

Ratings Roundup is a rundown of recent rating news and is derived from press rel...

01/05/2026

BKB Bare Knuckle Boxing Appoints Will Wright as Chief Operating Officer to Drive Global Growth and Operational Excellence

BKB Bare Knuckle Boxing ( BKB ), today announced the appointment of Will Wright ...

01/05/2026

NAB Rewind: Lawo's Andreas Hilmer on the Power of the Edge One AV Stagebox

Lawo has been at the center of the industry's transition to IP and other next-generation technologies. At NAB 2026, its story was the Edge One AV stagebox, ...

01/05/2026

Kentucky Derby 152 to Air Across 19 Networks in 170-Plus Territories

HBA Media, acting on behalf of NBC Sports and Churchill Downs Incorporated, has announced broadcast and streaming distribution for Kentucky Derby 152, taking pl...

01/05/2026

Give Me the Backstory: Get to Know Barbara Kopple, the Director of American Dream

By Bailey Pennick One of the most exciting things about the Sundance Film Festi...

01/05/2026

UPDATED: Republican AGs Join Nexstar-Tegna Antitrust Suit

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

Broadcaster Draper Media Names Bill Vernon President

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

Analysts: 'Hollywood's Vertical Video Strategy Is Dead Wrong'

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

Lightware UK celebrates new London showroom with launch e...

To celebrate the opening of its new showroom and office, Lightware UK hosted a dedicated launch event at the new London location. The event welcomed partners, c...

01/05/2026

Calrec Puts Broadcaster Choice Centre Stage at MPTS 2026

Choice without compromise The broadcast industrys transformation is accelerating, and traditional broadcasters are having to fundamentally reinvent how they o...

01/05/2026

Beam Dynamics Showcases its Asset Intelligence Platform a...

Beam Dynamics will return to MPTS 2026 with its asset intelligence platform, helping systems integrators, live production teams, media facilities and profession...

01/05/2026

Synamedia and FX Digital collaborate to bring GO Shorts a...

Best-in-class UX design and rapid, scalable delivery for next-generation viewing experiences Leading video software provider, Synamedia, today announced a coll...

01/05/2026

Compact new cforce MAX lens motor brings unrivaled speed and responsiveness to the Hi-5 ecosystem

Compact new cforce MAX lens motor brings unrivaled speed and responsiveness to t...

01/05/2026

Panavision welcomes Fritz Heinzle as Vice President of Sales

Panavision welcomes Fritz Heinzle as Vice President of Sales Brie Clayton May 1, 2026 0 Comments Heinzle will support Panavision's global growth s...

01/05/2026

NAB Hires FCC Staffer Ben Arden as SVP, Deputy General Counsel

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

ARRI Introduces Compact New cforce MAX Lens Motor

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

CPI Media Deploys QuickLink StudioCall

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

FCC Proposes to Amend Audible Crawl Rule to Preserve Accessibility

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

Republican AGs Join Nexstar/Tegna Antitrust Suit

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

Dan Johnson Elevates Precision Editing With NUGEN Audio D...

LONDON, APRIL 30, 2026 The Post Republic London's Re-recording Mixer and Dialogue Editor Dan Johnson has built a reputation for clean, emotionally resonan...

01/05/2026

Adobe Unveils Powerful New Innovations in Photoshop & Lightroom

Adobe Unveils Powerful New Innovations in Photoshop & Lightroom Deepa Subramaniam April 30, 2026 0 Comments Your most tedious creative tasks just got ea...

01/05/2026

Berklee Partners with Santander US to Establish Global Opportunity Fund

Berklee Partners with Santander US to Establish Global Opportunity Fund The $400,000 grant offers students access to experiential learning opportunities withi...

01/05/2026

Student Spotlight: Keziah Thomas

Student Spotlight: Keziah Thomas The Indian composer, who was named the 2026 student commencement speaker for Berklee College of Music, talks about how shes p...

01/05/2026

NIAJ Fest Gets Los Angeles In on the Joke With Free Pop-Up Events

Back to All News NIAJ Fest Gets Los Angeles In on the Joke With Free Pop-Up Events Entertainment 01 May 2026 GlobalUnited States Link copied to clipboard ...

30/04/2026

PWHL Reports Record Growth in Third Regular Season as Playoffs Begin April 30

The Professional Women's Hockey League (PWHL) concluded its third regular season on Saturday, reporting growth across attendance, viewership, digital engage...