Sony Pixel Power calrec Sony

Hugging Face and NVIDIA to Accelerate Open-Source AI Robotics Research and Development

06/11/2024

At the Conference for Robot Learning (CoRL) in Munich, Germany, Hugging Face and NVIDIA announced a collaboration to accelerate robotics research and development by bringing together their open-source robotics communities.

Hugging Face's LeRobot open AI platform combined with NVIDIA AI, Omniverse and Isaac robotics technology will enable researchers and developers to drive advances across a wide range of industries, including manufacturing, healthcare and logistics.

Open-Source Robotics for the Era of Physical AI The era of physical AI - robots understanding physical properties of environments - is here, and it's rapidly transforming the world's industries.

To drive and sustain this rapid innovation, robotics researchers and developers need access to open-source, extensible frameworks that span the development process of robot training, simulation and inference. With models, datasets and workflows released under shared frameworks, the latest advances are readily available for use without the need to recreate code.

Hugging Face's leading open AI platform serves more than 5 million machine learning researchers and developers, offering tools and resources to streamline AI development. Hugging Face users can access and fine-tune the latest pretrained models and build AI pipelines on common APIs with over 1.5 million models, datasets and applications freely accessible on the Hugging Face Hub.

LeRobot, developed by Hugging Face, extends the successful paradigms from its Transformers and Diffusers libraries into the robotics domain. LeRobot offers a comprehensive suite of tools for sharing data collection, model training and simulation environments along with designs for low-cost manipulator kits.

NVIDIA's AI technology, simulation and open-source robot learning modular framework such as NVIDIA Isaac Lab can accelerate the LeRobot's data collection, training and verification workflow. Researchers and developers can share their models and datasets built with LeRobot and Isaac Lab, creating a data flywheel for the robotics community.

Scaling Robot Development With Simulation Developing physical AI is challenging. Unlike language models that use extensive internet text data, physics-based robotics relies on physical interaction data along with vision sensors, which is harder to gather at scale. Collecting real-world robot data for dexterous manipulation across a large number of tasks and environments is time-consuming and labor-intensive.

Making this easier, Isaac Lab, built on NVIDIA Isaac Sim, enables robot training by demonstration or trial-and-error in simulation using high-fidelity rendering and physics simulation to create realistic synthetic environments and data. By combining GPU-accelerated physics simulations and parallel environment execution, Isaac Lab provides the ability to generate vast amounts of training data - equivalent to thousands of real-world experiences - from a single demonstration.

Generated motion data is then used to train a policy with imitation learning. After successful training and validation in simulation, the policies are deployed on a real robot, where they are further tested and tuned to achieve optimal performance.

This iterative process leverages real-world data's accuracy and the scalability of simulated synthetic data, ensuring robust and reliable robotic systems.

By sharing these datasets, policies and models on Hugging Face, a robot data flywheel is created that enables developers and researchers to build upon each other's work, accelerating progress in the field.

The robotics community thrives when we build together, said Animesh Garg, assistant professor at Georgia Tech. By embracing open-source frameworks such as Hugging Face's LeRobot and NVIDIA Isaac Lab, we accelerate the pace of research and innovation in AI-powered robotics.

Fostering Collaboration and Community Engagement The planned collaborative workflow involves collecting data through teleoperation and simulation in Isaac Lab, storing it in the standard LeRobotDataset format. Data generated using GR00T-Mimic, will then be used to train a robot policy with imitation learning, which is subsequently evaluated in simulation. Finally, the validated policy is deployed on real-world robots with NVIDIA Jetson for real-time inference.

The initial steps in this collaboration have already been taken, having shown a physical picking setup with LeRobot software running on NVIDIA Jetson Orin Nano, providing a powerful, compact compute platform for deployment.

Combining Hugging Face open-source community with NVIDIA's hardware and Isaac Lab simulation has the potential to accelerate innovation in AI for robotics, said Remi Cadene, principal research scientist at LeRobot.

This work builds on NVIDIA's community contributions in generative AI at the edge, supporting the latest open models and libraries, such as Hugging Face Transformers, optimizing inference for large language models (LLMs), small language models (SLMs) and multimodal vision-language models (VLMs), along with VLM's action-based variants of vision language action models (VLAs), diffusion policies and speech models - all with strong, community-driven support.

Together, Hugging Face and NVIDIA aim to accelerate the work of the global ecosystem of robotics researchers and developers transforming industries ranging from transportation to manufacturing and logistics.

Learn about NVIDIA's robotics research papers at CoRL, including VLM integration for better environmental understanding, temporal navigation and long-horizon planning. Check out workshops at CoRL with NVIDIA researchers.
LINK: https://blogs.nvidia.com/blog/hugging-face-lerobot-open-source-robotic...
See more stories from nvidia

North America Stories

01/09/2026

IBC 2026: Bridge Technologies Adds Blackmagic IP10 Support to VB440 Production Probe

Bridge Technologies will announce at IBC2026 (Booth 1.A71) that its VB440 produc...

01/09/2026

DAZN to Acquire EverPass Media to Expand U.S. Commercial Venue Distribution

DAZN has announced an agreement to acquire EverPass Media, a commercial sports distribution platform serving bars, restaurants, hotels, and other venues in the ...

01/09/2026

Kalshi Named Official Prediction Market Partner of the US Open

The USTA and Kalshi have announced a multi-year partnership making Kalshi the Official Prediction Market Partner of the US Open, beginning with the 2026 US Open...

01/09/2026

IBC 2026: MASV to Showcase Sports Video Transfer Workflows

MASV will exhibit at IBC2026 (Booth 5.B40, RAI Amsterdam, September 11-14). CTO Majed Alhajry will appear on a panel session, A Tale of Two Feeds: Automation o...

01/09/2026

IBC 2026: FingerWorks Telestrators Partners with Broadcast Rental for European Distribution

FingerWorks Telestrators has announced a partnership with Broadcast Rental to ex...

01/09/2026

IBC 2026: FOR-A Expands viztrick AiDi into Two Dedicated Products

FOR-A has announced the expansion of viztrick AiDi into two dedicated products: Go Vertical! AiDi and AiDi Sports Hub. Developed by Nippon TV and delivered glob...

01/09/2026

Sony Launches Live TV on PS5 with Free Ad-Supported Streaming in the U.S.

Sony Pictures Television has launched Live TV on PS5, a free ad-supported streaming experience available to PS5 players in the United States. The service is acc...

01/09/2026

Friedrichstadt-Palast Installs Riedel Production Infrastructure

Friedrichstadt-Palast in Berlin has installed a Riedel production infrastructure spanning video, intercom, control, and I/O across the venue, implemented in col...

01/09/2026

IBC 2026: Arkona Partners with BFE for Arkona-Branded Control Panels

Arkona technologies GmbH has announced an OEM partnership with BFE Studio und Medien Systeme GmbH to offer arkona-branded control panels for its SDI-IP routing ...

01/09/2026

Cricket Australia Launches Cricket TV Streaming Service on Eluvio Platform

Cricket Australia has launched Cricket TV, a direct-to-consumer streaming service built on Eluvio's Content Fabric platform. The service is currently in ear...

01/09/2026

Eluvio Selected to Power Pasifika+ Streaming Service for Pacific Island Communities

Eluvio has announced it was selected to power Pasifika , a new direct-to-consume...

01/09/2026

Layercake Names Jon Marquard as Director/Chairperson

Layercake, the company behind the intelligent media workflow platform Streamcake, has appointed Jon Marquard as Director and Chairperson. The appointment marks ...

01/09/2026

Gravity Media Extends Powercor Stawell Gift Broadcast Agreement Through 2031

Gravity Media has announced a multi-year renewal of its agreement with Stawell Gift Event Management to produce the complete broadcast of the Powercor Stawell G...

01/09/2026

IBC 2026: Beam Dynamics Makes Debut with License Intelligence Module

Beam Dynamics will exhibit at IBC2026 for the first time (Stand 14.FT29, Hall 14), introducing License Intelligence, a new module for managing software licenses...

01/09/2026

ZOO Digital Selects Interra Systems BATON for File-Based Quality Control

Interra Systems has announced that ZOO Digital has selected BATON for file-based quality control across its media workflows. ZOO Digital delivers localized ente...

01/09/2026

IBC 2026: Bridge Technologies Deploys VB330 Monitoring at Sunrise

Bridge Technologies has announced the deployment of its VB330 monitoring probe at Sunrise, supporting the Swiss telecommunications provider's IP-based telev...

01/09/2026

Riedel Expands Coach Communications Partnership with Scottish Rugby Union

Riedel Communications has announced the continued expansion of its managed coach communications partnership with Scottish Rugby Union, now entering its third se...

01/09/2026

Behind The Mic: Football Season Approaches, as Networks Name On Air Talent for NFL, College Broadcasts

Behind The Mic provides a roundup of recent news regarding on-air talent, includ...

01/09/2026

IBC 2026: SMPTE Brings Full-Day Software-Defined Infrastructure Roadshow

SMPTE today detailed the centerpiece of its IBC2026 program: the SMPTE Roadshow: Future of Media Infrastructure, a full-day workshop on Monday, 14 September, th...

01/09/2026

Nielsen Announces New Local Methodology Enhancements

New York August 31, 2026 Nielsen is announcing four new local methodology enhancements that will be activated into Local TV currency data starting today, Au...

01/09/2026

Broadcasters Gain Flexibility in Pursuit of 5G Multicasting

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

Winning the Fight Against Churn

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

How Multiviewers Are Evolving Into Information Hubs'

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

IBC2026: New Audio Tools on Tap from Telos Alliance

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

SMPTE Roadshow to Examine Media Infrastructure at IBC2026

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

Roku Debuts Its First 4K OLED TVs

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

Mark Vinten Joins ASG's Audio and Postproduction Team

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

Pronology Introduces New StreamFile Core Capabilities at...

Pronology, a developer of cutting-edge broadcast workflow solutions, showcases the newest capabilities for its StreamFile Core platform at the 2026 IBC Show (S...

01/09/2026

PROTON Unveils Next-Generation FLEXY Camera at IBC 2026

PROTON Unveils Next-Generation FLEXY Camera at IBC 2026 Brie Clayton September 1, 2026 0 Comments 10-metre serialised coax unlocks new perspectives an...

01/09/2026

Hulu Drama The Season Graded with DaVinci Resolve Studio

Hulu Drama The Season Graded with DaVinci Resolve Studio Brie Clayton September 1, 2026 0 Comments Senior Colorist Mark Kueper of Picture Shop crafts ...

01/09/2026

Pluxbox Launches CoreOC to Transform Mission-Critical Media Operations

Pluxbox Launches CoreOC to Transform Mission-Critical Media Operations Brie Clayton September 1, 2026 0 Comments A vendor-agnostic operational control l...

01/09/2026

Layercake Appoints Former FOX SPORTS and Network Ten COO...

Layercake, the company behind the intelligent media workflow platform Streamcake, today announced the appointment of Jon Marquard as Director and Chairperson. T...

01/09/2026

Actus Digital Unveils the Future of Broadcast Monitoring...

Actus Digital, a LiveU company, will unveil the next generation of its Actus X platform at IBC2026 on LiveUs Stand 7.C24, demonstrating how broadcasters can sim...

01/09/2026

Studio Automated and Egripment Power the Tactical Feed fo...

Broadcast Solutions, a leading systems integrator and provider of innovative solutions for the broadcast media industry, delivered transformational technology a...

01/09/2026

OTV turns to Quortex to replace legacy video hardware

Quortex, the intelligent video technology company carved out from Synamedia by Lumine Group, today announced that Odisha Television Network (OTV), the leading r...

01/09/2026

Zixi and Ateme Deepen Strategic Alliance to Accelerate Hy...

Zixi and Ateme today announced an expanded strategic partnership to help broadcasters and content owners accelerate the transition from satellite to resilient I...

01/09/2026

CBS Boston Debuts New AR/VR Technology-Driven Studio

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

FCC Raises Regulatory Fees for TV Stations

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

For-A Introduces Go Vertical! AiDi and AiDi Sports Hub

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

T-Mobile Continues to Dominate Ooklas Fixed Wireless Speed Tests

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

Carr Announces New FCC Enforcement Bureau Chief

Share Copy link Facebook X Linkedin Bluesky Email...

01/09/2026

New Products, New Workflows: Telos Alliance Heads to IBC 2026

New Products, New Workflows: Telos Alliance Heads to IBC 2026 Search CLEVELAND, Ohio, September 1, 2026 Telos Alliance , trusted global leader in broadca...

01/09/2026

FOR-A Introduces Go Vertical! AiDi and AiDi Sports Hub Under Expanded viztrick AiDi AI Technology Brand

On-device AI platform gives broadcasters and content creators purpose-built tool...

31/08/2026

Pixotope Consolidates Virtual Production Tools into Unified Platform

Pixotope Technologies has announced a unified platform offering all Pixotope capabilities across three editions, available with the Pixotope 26.3 release in Q3 ...

31/08/2026

FingerWorks Telestrators Expands European Presence Throug...

FingerWorks Telestrators, the industry leader in software-based telestration and live sports analysis solutions, announces its new partnership with Broadcast R...

31/08/2026

ARISDA Bridge Launches: DaVinci Resolve Timecode, Studio Clock, and Work-Time Tracking in One Windows App

ARISDA Bridge Launches: DaVinci Resolve Timecode, Studio Clock, and Work-Time Tr...

31/08/2026

Short Film All That's Left Shot with URSA Cine 12K LF

Short Film All That's Left Shot with URSA Cine 12K LF Brie Clayton August 31, 2026 0 Comments Intimate drama edited and graded in DaVinci Resolve ...

31/08/2026

Bridge Technologies Announces Support for Blackmagic IP10 Codec

Bridge Technologies Announces Support for Blackmagic IP10 Codec Brie Clayton August 31, 2026 0 Comments Both booths will showcase each other's tec...

31/08/2026

Creative COW Roundtable, August 23, 2026- AEAI

Creative COW Roundtable, August 23, 2026- AEAI Brie Clayton August 31, 2026 0 Comments COWs from around the world assemble to discuss Using AI to Beat...

31/08/2026

Homecoming Artist Spotlight: Susan Tedeschi

Homecoming Artist Spotlight: Susan Tedeschi The Grammy-winning singer and guitarist talks about finding her voice at Berklee ahead of her Homecoming performan...