Sony Pixel Power calrec Sony

Hugging Face and NVIDIA to Accelerate Open-Source AI Robotics Research and Development

06/11/2024

At the Conference for Robot Learning (CoRL) in Munich, Germany, Hugging Face and NVIDIA announced a collaboration to accelerate robotics research and development by bringing together their open-source robotics communities.

Hugging Face's LeRobot open AI platform combined with NVIDIA AI, Omniverse and Isaac robotics technology will enable researchers and developers to drive advances across a wide range of industries, including manufacturing, healthcare and logistics.

Open-Source Robotics for the Era of Physical AI The era of physical AI - robots understanding physical properties of environments - is here, and it's rapidly transforming the world's industries.

To drive and sustain this rapid innovation, robotics researchers and developers need access to open-source, extensible frameworks that span the development process of robot training, simulation and inference. With models, datasets and workflows released under shared frameworks, the latest advances are readily available for use without the need to recreate code.

Hugging Face's leading open AI platform serves more than 5 million machine learning researchers and developers, offering tools and resources to streamline AI development. Hugging Face users can access and fine-tune the latest pretrained models and build AI pipelines on common APIs with over 1.5 million models, datasets and applications freely accessible on the Hugging Face Hub.

LeRobot, developed by Hugging Face, extends the successful paradigms from its Transformers and Diffusers libraries into the robotics domain. LeRobot offers a comprehensive suite of tools for sharing data collection, model training and simulation environments along with designs for low-cost manipulator kits.

NVIDIA's AI technology, simulation and open-source robot learning modular framework such as NVIDIA Isaac Lab can accelerate the LeRobot's data collection, training and verification workflow. Researchers and developers can share their models and datasets built with LeRobot and Isaac Lab, creating a data flywheel for the robotics community.

Scaling Robot Development With Simulation Developing physical AI is challenging. Unlike language models that use extensive internet text data, physics-based robotics relies on physical interaction data along with vision sensors, which is harder to gather at scale. Collecting real-world robot data for dexterous manipulation across a large number of tasks and environments is time-consuming and labor-intensive.

Making this easier, Isaac Lab, built on NVIDIA Isaac Sim, enables robot training by demonstration or trial-and-error in simulation using high-fidelity rendering and physics simulation to create realistic synthetic environments and data. By combining GPU-accelerated physics simulations and parallel environment execution, Isaac Lab provides the ability to generate vast amounts of training data - equivalent to thousands of real-world experiences - from a single demonstration.

Generated motion data is then used to train a policy with imitation learning. After successful training and validation in simulation, the policies are deployed on a real robot, where they are further tested and tuned to achieve optimal performance.

This iterative process leverages real-world data's accuracy and the scalability of simulated synthetic data, ensuring robust and reliable robotic systems.

By sharing these datasets, policies and models on Hugging Face, a robot data flywheel is created that enables developers and researchers to build upon each other's work, accelerating progress in the field.

The robotics community thrives when we build together, said Animesh Garg, assistant professor at Georgia Tech. By embracing open-source frameworks such as Hugging Face's LeRobot and NVIDIA Isaac Lab, we accelerate the pace of research and innovation in AI-powered robotics.

Fostering Collaboration and Community Engagement The planned collaborative workflow involves collecting data through teleoperation and simulation in Isaac Lab, storing it in the standard LeRobotDataset format. Data generated using GR00T-Mimic, will then be used to train a robot policy with imitation learning, which is subsequently evaluated in simulation. Finally, the validated policy is deployed on real-world robots with NVIDIA Jetson for real-time inference.

The initial steps in this collaboration have already been taken, having shown a physical picking setup with LeRobot software running on NVIDIA Jetson Orin Nano, providing a powerful, compact compute platform for deployment.

Combining Hugging Face open-source community with NVIDIA's hardware and Isaac Lab simulation has the potential to accelerate innovation in AI for robotics, said Remi Cadene, principal research scientist at LeRobot.

This work builds on NVIDIA's community contributions in generative AI at the edge, supporting the latest open models and libraries, such as Hugging Face Transformers, optimizing inference for large language models (LLMs), small language models (SLMs) and multimodal vision-language models (VLMs), along with VLM's action-based variants of vision language action models (VLAs), diffusion policies and speech models - all with strong, community-driven support.

Together, Hugging Face and NVIDIA aim to accelerate the work of the global ecosystem of robotics researchers and developers transforming industries ranging from transportation to manufacturing and logistics.

Learn about NVIDIA's robotics research papers at CoRL, including VLM integration for better environmental understanding, temporal navigation and long-horizon planning. Check out workshops at CoRL with NVIDIA researchers.
LINK: https://blogs.nvidia.com/blog/hugging-face-lerobot-open-source-robotic...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

16/03/2026

DAZN to Stream NCAA Men's and Women's Basketball Tourneys Free in Select International Markets

DAZN will allow fans in select international territories to watch the NCAA men&#...

16/03/2026

IDM and Skate Board Association Announce Arena and Training Complex Planned for Big Bear Lake

IDM and The Skate Board Association (SBA) have announced a partnership with Coop...

16/03/2026

NAB 2026: Solid State Logic Introduces ST 2110-to-Dante Converter

Solid State Logic (SSL) will debut the Net I/O ST 2110 Bridge at NAB 2026 (booth C6907), a standalone unit that converts between ST 2110 and Dante audio formats...

16/03/2026

NAB 2026: Marshall Electronics Launches First 4K All-IP Weatherproof NDI Camera

Marshall Electronics (Booth C8339) is introducing its first all-IP 4K POV camera, the CV574-WP, at NAB 2026. The camera carries an IP67 weatherproof rating for ...

16/03/2026

Sony Expands Camera Authenticity Solution to Support Video

Sony Electronics' Camera Verify (beta), a feature of its Camera Authenticity Solution which enables news organizations to share content authenticity informa...

16/03/2026

FloSports and Storied Sports Partner on Women's and College Sports Content

FloSports has announced a partnership with Storied Sports, a content and IP studio founded by former espnW and The Players' Tribune executives, to develop s...

16/03/2026

Montreux Jazz Festival Names Gravity Media as A/V Production Provider

Montreux Jazz Festival has announced a multi-year collaboration with Gravity Media, who will become the Festival's Audio Visual Production Provider followin...

16/03/2026

USSI Global Names Ralph Annunziata Senior Vice President of Operations

USSI Global, a provider of customized network, broadcast and digital signage systems and services, has announced Ralph Annunziata joined the company on Jan. 5 a...

16/03/2026

NAB 2026: Boland Communications to Show New OLED Displays and Video Wall Applications

Boland Communications (booth C3519) will exhibit at NAB Show 2026 in Las Vegas, ...

16/03/2026

ST 2110 On The Go? A Peek Inside BRISK, FOX Sports' Broadcast Remote IP Studio Kit

Built in partnership with Diversified, the system As the sports broadcast indus...

16/03/2026

Amagi Report: FAST Viewership Up 21%, AI Adoption Growing Across Media Operations

Global FAST (Free Ad-supported Streaming TV) viewership grew 21% year-over-year ...

16/03/2026

Behind The Mic: Netflix, NBC Tap Matt Vasgersian to Call MLB and Tony Dungy Is Out at NBC

Behind The Mic provides a roundup of recent news regarding on-air talent, includ...

16/03/2026

Cloudvocal launch the SonoFlex instrument mic

Promises studio-grade fidelity for the stage Cloudvocal have announced the launch of a new instrument mic designed for professional live performers and engi...

16/03/2026

Kenton reveal the USB Solo Mk2

Popular MIDI/CV converter & interface overhauled Kenton have announced the launch of the USB Solo Mk2, a new and improved version of their compact MIDI to C...

16/03/2026

Sonarworks Spring Sale

Running from 16-29 March 2026 Starting from today (16 March) and running until 29 March 2026, Sonarworks are offering discounts of up to 40% across their ra...

16/03/2026

L3Harris Carries Goddard's Legacy Into a New Era

Dr. Robert H. Goddard and a liquid oxygen-gasoline rocket in the frame from which it was fired on March 16, 1926, at Auburn, Massachusetts. Credit: NASA....

16/03/2026

L3Harris Military GPS Receiver Deliveries Surpass 100,000 Units

Precision-guided munitions shown in production illustrate one of many operational systems benefiting from modernized M-Code GPS, supporting assured positioning,...

16/03/2026

A+E Global Media Signs New Multiyear Deal With Nielsen Covering Audience Measurement and Media Intelligence

NEW YORK - March 16, 2026 - A E Global Media and Nielsen today announced a new,...

16/03/2026

aconnic ramping up delivery of commercial 100-gigabit system

aconnic AG (ISIN: DE000A0LBKW6), Munich, is delivering the first commercial 100-Gigabit systems following successful validation and certification for customer n...

16/03/2026

Spectrum Launches Multiview for March Madness

Share Copy link Facebook X Linkedin Bluesky Email...

16/03/2026

Ikegami To Spotlight Latest UNICAM 4K-UHD Cameras At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

16/03/2026

A+E Global Media Signs New Agreement With Nielsen

Share Copy link Facebook X Linkedin Bluesky Email...

16/03/2026

Shotoku Brings Broadcast-Grade Control to PTZ with New Au...

Shotoku USA, Shotoku Broadcast Systems' North American operation, will unveil significant additions to its platform at NAB 2026. Topping the list is the wor...

16/03/2026

Ikegami to Showcase Latest Generation TV Production Camer...

Ikegami USA will demonstrate the latest additions to its wide range of broadcast-quality cameras, controllers and monitors on Central Hall booth C3819 during th...

16/03/2026

[Updated] Carr Threatens Broadcast Licenses Over Iran War Coverage

Share Copy link Facebook X Linkedin Bluesky Email...

16/03/2026

ELEMENTS launches GRID at NAB Show 2026

ELEMENTS launches GRID at NAB Show 2026 Brie Clayton March 15, 2026 0 Comments North Hall, Booth N1717 ELEMENTS returns to NAB Show 2026, with an exp...

16/03/2026

Blackmagic Design Cameras Capture Artist Salavat Fidai's Micro Sculptures

Blackmagic Design Cameras Capture Artist Salavat Fidai's Micro Sculptures Brie Clayton March 15, 2026 0 Comments 6K sensor and open gate capabilit...

16/03/2026

DHD to Introduce Latest Generation Broadcast Audio Mixers at NAB 2026, Las Vegas

DHD to Introduce Latest Generation Broadcast Audio Mixers at NAB 2026, Las Vegas Brie Clayton March 15, 2026 0 Comments Hero image: Front of DHD RM1 P...

16/03/2026

VEON Files its 2025 Annual Report on Form 20-F

16 Mar 2026 VEON Files its 2025 Annual Report on Form 20-F Dubai and New York, March 16, 2026 - VEON Ltd. (Nasdaq: VEON), a global digital operator ( VEON'...

16/03/2026

Sky Commissions The 100 Day Split, A New Relationship Series Exploring What Time Apart Reveals About Lifelong Love

Six Couples. 100 Days Apart. One Question: Does Absence Make the Heart Grow Fond...

16/03/2026

Tina Fey, Jamie Dornan and Riz Ahmed announced as first three hosts of Saturday Night Live UK

Monday 16 March 2026 Tina Fey, Jamie Dornan and Riz Ahmed announced as first th...

16/03/2026

Netflix Has Released the Trailer for 'Love at Last,' Starring Eda Ece and Kaan Yildirim

Back to All News Netflix Has Released the Trailer for Love at Last, Starring Ed...

16/03/2026

All UK national newspapers move to private circulation reporting while remaining audited by ABC

The reporting option was introduced following extensive consultation with publis...

16/03/2026

Rose of Tralee Katelyn Cummins wins Dancing with the Stars 2026

After a nail-biting Grand Finale, Rose of Tralee Katelyn Cummins has been announced as the winner of Dancing with the Stars 2026. The four finalists each dance...

16/03/2026

New RT series Welcome to Moore Street gives a glimpse into life on iconic Dublin street

Welcome to Moore Street will begin on RT One and RT Player on Thursday 19 Marc...

15/03/2026

Visit ToolsOnAir at NAB Las Vegas 2026

Visit ToolsOnAir at NAB Las Vegas 2026 More Details:From April 19-22, join us at NAB Show Las Vegas in the North Hall, Booth N1258, for an exclusive preview of...

15/03/2026

Outland III from The Very Loud Indeed Co.

Latest dark drama, thrillers & tension library announced The Very Loud Indeed Co.'s latest Kontakt library has just arrived, delivering a third instalme...

15/03/2026

19TH ANNUAL SOUTH AFRICAN FILM AND TELEVISION AWARDS (SAFTAs) CELEBRATED THE BEST OF THE SOUTH AFRICAN FILM AND TELEVISION INDUSTRY

Johannesburg, 14 March 2026 - On 13 and 14 March 2026, the 19th Annual South Afr...

15/03/2026

Carr Threatens Broadcast Licenses Over Iran War Coverage

Share Copy link Facebook X Linkedin Bluesky Email...

14/03/2026

HISONG announce the AirStudio S1

Combines mic, USB interface & wireless IEMs Following a successful Kickstarter campaign, HISONG have announced that their innovative AirStudio S1 device is ...

14/03/2026

Study: Applied AI Moving Into Core Media Workflows

Share Copy link Facebook X Linkedin Bluesky Email...

14/03/2026

GlobalM to Showcase Live IP Media Transport at 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

14/03/2026

Milano Cortina 2026 - Yospace helps ad-funded rights-hold...

Yospace, the trusted leader in Dynamic Ad Insertion (DAI), stitched 5.4 billion one-to-one addressable OTT advertisements across the 17 days of Milano Cortina 2...

14/03/2026

Telestream Advances Production-Ready AI Across Its Product Portfolio

Telestream Advances Production-Ready AI Across Its Product Portfolio Brie Clayton March 13, 2026 0 Comments New AI capabilities drive smarter automati...