Sony Pixel Power calrec Sony

Hugging Face and NVIDIA to Accelerate Open-Source AI Robotics Research and Development

06/11/2024

At the Conference for Robot Learning (CoRL) in Munich, Germany, Hugging Face and NVIDIA announced a collaboration to accelerate robotics research and development by bringing together their open-source robotics communities.

Hugging Face's LeRobot open AI platform combined with NVIDIA AI, Omniverse and Isaac robotics technology will enable researchers and developers to drive advances across a wide range of industries, including manufacturing, healthcare and logistics.

Open-Source Robotics for the Era of Physical AI The era of physical AI - robots understanding physical properties of environments - is here, and it's rapidly transforming the world's industries.

To drive and sustain this rapid innovation, robotics researchers and developers need access to open-source, extensible frameworks that span the development process of robot training, simulation and inference. With models, datasets and workflows released under shared frameworks, the latest advances are readily available for use without the need to recreate code.

Hugging Face's leading open AI platform serves more than 5 million machine learning researchers and developers, offering tools and resources to streamline AI development. Hugging Face users can access and fine-tune the latest pretrained models and build AI pipelines on common APIs with over 1.5 million models, datasets and applications freely accessible on the Hugging Face Hub.

LeRobot, developed by Hugging Face, extends the successful paradigms from its Transformers and Diffusers libraries into the robotics domain. LeRobot offers a comprehensive suite of tools for sharing data collection, model training and simulation environments along with designs for low-cost manipulator kits.

NVIDIA's AI technology, simulation and open-source robot learning modular framework such as NVIDIA Isaac Lab can accelerate the LeRobot's data collection, training and verification workflow. Researchers and developers can share their models and datasets built with LeRobot and Isaac Lab, creating a data flywheel for the robotics community.

Scaling Robot Development With Simulation Developing physical AI is challenging. Unlike language models that use extensive internet text data, physics-based robotics relies on physical interaction data along with vision sensors, which is harder to gather at scale. Collecting real-world robot data for dexterous manipulation across a large number of tasks and environments is time-consuming and labor-intensive.

Making this easier, Isaac Lab, built on NVIDIA Isaac Sim, enables robot training by demonstration or trial-and-error in simulation using high-fidelity rendering and physics simulation to create realistic synthetic environments and data. By combining GPU-accelerated physics simulations and parallel environment execution, Isaac Lab provides the ability to generate vast amounts of training data - equivalent to thousands of real-world experiences - from a single demonstration.

Generated motion data is then used to train a policy with imitation learning. After successful training and validation in simulation, the policies are deployed on a real robot, where they are further tested and tuned to achieve optimal performance.

This iterative process leverages real-world data's accuracy and the scalability of simulated synthetic data, ensuring robust and reliable robotic systems.

By sharing these datasets, policies and models on Hugging Face, a robot data flywheel is created that enables developers and researchers to build upon each other's work, accelerating progress in the field.

The robotics community thrives when we build together, said Animesh Garg, assistant professor at Georgia Tech. By embracing open-source frameworks such as Hugging Face's LeRobot and NVIDIA Isaac Lab, we accelerate the pace of research and innovation in AI-powered robotics.

Fostering Collaboration and Community Engagement The planned collaborative workflow involves collecting data through teleoperation and simulation in Isaac Lab, storing it in the standard LeRobotDataset format. Data generated using GR00T-Mimic, will then be used to train a robot policy with imitation learning, which is subsequently evaluated in simulation. Finally, the validated policy is deployed on real-world robots with NVIDIA Jetson for real-time inference.

The initial steps in this collaboration have already been taken, having shown a physical picking setup with LeRobot software running on NVIDIA Jetson Orin Nano, providing a powerful, compact compute platform for deployment.

Combining Hugging Face open-source community with NVIDIA's hardware and Isaac Lab simulation has the potential to accelerate innovation in AI for robotics, said Remi Cadene, principal research scientist at LeRobot.

This work builds on NVIDIA's community contributions in generative AI at the edge, supporting the latest open models and libraries, such as Hugging Face Transformers, optimizing inference for large language models (LLMs), small language models (SLMs) and multimodal vision-language models (VLMs), along with VLM's action-based variants of vision language action models (VLAs), diffusion policies and speech models - all with strong, community-driven support.

Together, Hugging Face and NVIDIA aim to accelerate the work of the global ecosystem of robotics researchers and developers transforming industries ranging from transportation to manufacturing and logistics.

Learn about NVIDIA's robotics research papers at CoRL, including VLM integration for better environmental understanding, temporal navigation and long-horizon planning. Check out workshops at CoRL with NVIDIA researchers.
LINK: https://blogs.nvidia.com/blog/hugging-face-lerobot-open-source-robotic...
See more stories from nvidia

North America Stories

12/08/2026

Op-Ed: The Versioning Playbook - How Rights Holders are Getting More From Live Events

Traditional live sports production has focused on creating the best possible pro...

12/08/2026

IBC 2026: FOR-A to Demonstrate Production and Processing Solutions

FOR-A will showcase a range of broadcast and production products at IBC2026 (Stand 2.B53, Amsterdam RAI, September 11-14), including software-based switching, A...

12/08/2026

IBC 2026: AMWA and EBU to Demo NMOS BCP-008 Multi-Vendor IP Status Reporting

The Advanced Media Workflow Association (AMWA) and EBU will host live demonstrations of NMOS BCP-008 Minimum Status Reporting at IBC2026 (September 11-14) on th...

12/08/2026

IBC 2026: Dalet Launches Solution Suites for Media Production and Archive Management

Dalet has announced Dalet Solution Suites, a portfolio of cloud-native solutions...

12/08/2026

Net Insight Enables Centralized Remote Production of Blinkfestivalen 2026 for NRK

Net Insight has announced that DMC Production Norway has selected its Open Live ...

12/08/2026

Zixi Appoints Alyson Stewart as Senior Director of Sales

Zixi has appointed Alyson Stewart as Senior Director of Sales. Stewart will be responsible for direct sales and partnerships, with a focus on broadcasters, medi...

12/08/2026

LiveU Powers Host Broadcast Operations at 2026 Glasgow Games

LiveU's LU900Q intelligent production unit was deployed as part of the host broadcast operation for the Games in Glasgow (July 23-August 2, 2026). Host broa...

12/08/2026

Disney+ and ESPN Acquire Multi-Year Formula E Rights Beginning 2026/27 Season

Disney , ESPN, and Formula E have announced a multi-year deal bringing all Formula E races to Disney and ESPN in the U.S. and to Disney in most international...

12/08/2026

PSSI Launches X-LIVE Managed Service for Live Event Transmission

PSSI has launched X-LIVE, a managed service offering dedicated engineering resources and Appear encoding infrastructure for live event signal processing and tra...

12/08/2026

MoonPay X Games League Draws 18.3 Million Viewers in Inaugural Summer Season

The inaugural MoonPay X Games League Summer Season reached 18.3 million unique live viewers across three stops on ABC and ESPN, a 78.5% increase compared to the...

12/08/2026

IBC 2026: BBright Adds RIST Support to IP-Gateway

BBright will showcase an expanded IP-Gateway at IBC2026, adding support for all RIST profiles to its existing support for MPEG Transport Stream over IP with SMP...

12/08/2026

IBC 2026: BBright Adds EBU OGraf Graphics Support to UHD and ST 2110 Playout Platform

BBright will demonstrate EBU OGraf animated graphics support in its software-bas...

12/08/2026

Interra Systems to Showcase QC and Monitoring Solutions at SET Expo 2026

Interra Systems will demonstrate its ORION, BATON, BATON Captions, and VEGA solutions at SET Expo 2026 (August 17-20, Distrito Anhembi, S o Paulo) at partner Li...

12/08/2026

DPA Microphones Appoints Raul Contreras as West Coast Regional Sales Manager

DPA Microphones has appointed Raul Contreras as West Coast Regional Sales Manager. In the role, Contreras will oversee sales and customer relationships througho...

12/08/2026

Save the Date: SVG LIVE! Returns to LA Nov. 10 to Spotlight Sports & Entertainment Production

Live spectacles ranging from the Super Bowl and FIFA World Cup to the Oscars and...

12/08/2026

Love, Separation, and the Cost of Comfort in How to Divorce During the War

Writer-Director Andrius Bla evi ius and crew attend the premiere of How to Divorce During the War at The Ray Theatre on January 26, 2026, in Park City, UT. (P...

12/08/2026

Nielsen Introduces Groundbreaking Streaming Ad Measurement in Brazil

Nielsen is cementing Digital Ad Intel as the new go-to currency for digital ad intelligence in Brazil-now with streaming monitoring in the mix. S o Paulo, Augu...

12/08/2026

LPTV Advocates Petition FCC for Voluntary 5G Broadcasts

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Tubi Media Group Inks Content Discovery Deal with Gracenote

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Matrox Video Showcases IP-Based and Software-Defined Tech...

As Matrox Video celebrates 50 years of trusted innovation, the company will showcase its portfolio of interoperable IP technologies and software-defined innovat...

12/08/2026

SMPTE Showcases AI and Open Media Workflows and Industry...

SMPTE, the home of media professionals, technologists and engineers, today announced a program of expert-led sessions, professional workshops and networking eve...

12/08/2026

Berklee Announces Full Details for Inaugural Block Party, a Music Festival Next to Fenway Park

Berklee Announces Full Details for Inaugural Block Party, a Music Festival Next ...

12/08/2026

ACLU Requests ABC Station Probe Documents from FCC

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Roger Keating To Retire from Hearst Television

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Netflix Closes U.S. Upfronts, Nearly Doubles Ad Commitments

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

CBS Pittsburgh Debuts New AR/VR News Studio

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Matthews Intros T-Go Extreme Rigging Solutions

Matthews Studio Equipment introduces T-Go Mounting Solutions extreme camera rigging systems, engineered for the most demanding environments in motion picture pr...

12/08/2026

Anton Bauers EDEN Becomes Only Mobile Power Station in it...

Anton/Bauer, a Videndum brand and the industry leader in mobile power solutions for broadcast and cinematic production, announces that the EDEN Mobile Power Sta...

12/08/2026

TAG Video Systems Cuts Hardware Costs for Customers with...

TAG Video Systems, the global leader in software-based IP monitoring and multiviewer solutions, today announced a memory usage optimization that lowers the tota...

12/08/2026

Unified Streaming Launches Trusted Media - C2PA in Dynami...

Unified Streaming announces Trusted Media, a new solution that enables broadcasters, streaming platforms, and premium content owners to attach verifiable authen...

12/08/2026

Tedial Powers the AI-Driven Media Supply Chain at IBC 202...

Combining intelligent automation, composable business services and user-centric experiences The media industry is entering a new phase where AI intelligence is...

12/08/2026

DPA Microphones Appoints Raul Contreras as West Coast Reg...

DPA Microphones welcomes Raul Contreras as its new West Coast Regional Sales Manager. In this role, Contreras will oversee sales and customer relationships thro...

12/08/2026

Weather is Not a Factor for Marshall Electronics New CV57...

Marshall Electronics' first all-weather, all-IP 4K POV camera, the CV574-WP, makes its European debut at IBC 2026 (Stand 10.C11). Built for the elements, th...

12/08/2026

A double off-switch helps cells respond to stress

LA JOLLA, CA-Scientists at Scripps Research have discovered that two seemingly unrelated cellular enzymes become locked together during times of stress, simulta...

12/08/2026

NVIDIA CEO Tops Glassdoor's 2026 List of Best CEOs

NVIDIA founder and CEO Jensen Huang is ranked No. 1 on Glassdoor's Best CEOs list for 2026. In the just-released ranking, recognition is earned directly fr...

11/08/2026

Apple To Bring Friday Night Baseball Live to Apple Immersive on Vision Pro Beginning Aug. 28

Weekly immersive broadcasts will feature 8K 3D video, a 180-degree field of view...

11/08/2026

Looking for a New Cameraperson? ProductionHUB Guide Lays Out Considerations, Pitfalls

By Briana Brady, Social Media Manager, Production Hub Sports content is having ...

11/08/2026

SVG Sit-Down: Backlight CEO Kathleen Barrett on Building Media Infrastructure for a Global, AI-Driven Sports Landscape

She addresses why media organizations need more than asset storage, how sports t...

11/08/2026

FloSports Dennis LeMire on Implementing Cloud-Based Workflows on Lower Tier Sports

Remote commentary has improved significantly on these broadcasts At a clip of ...

11/08/2026

Miami HEAT's Revamped IP-Based Control Room Coordinates Workflows With Meinberg Intelligent Modular Synchronization System

A pair of LANTIME IMS M2000 PTP Grandmasters deliver resilient timing infrastruc...

11/08/2026

LTN Expands Live Event Master Control Capabilities for Sports Organizations

LTN has announced expanded capabilities for its Live Event Master Control offering, providing sports broadcasters, leagues, rights holders, and content owners w...

11/08/2026

WTA Ventures Appoints IMG as Managed Production Services Partner

WTA Ventures has appointed IMG as its managed production services partner in a multi-year agreement commencing with the 2027 season. The partnership is the late...

11/08/2026

New York Jets and CBS New York Extend Multi-Year Partnership

The New York Jets and CBS New York have agreed to a multi-year extension of their partnership, keeping CBS New York as the official broadcast home of the team. ...

11/08/2026

Nielsen Acquires DoubleVerify to Create New Combined Media Intelligence Platform

Nielsen Holdings and DoubleVerify have announced a definitive agreement under which Nielsen will acquire DoubleVerify in an all-cash transaction with an enterpr...

11/08/2026

AES Elects David v.R. Bowles as President-Elect for 2027, Unveils Other Election Results

The Audio Engineering Society has announced the results of its 2026 Board of Dir...

11/08/2026

SVG Sit-Down Creator Sports Network's Barrick Prince on Building the Next Layer of the Live-Sports Rights Stack

The CSN co-founder details its network and affiliate' model, how creator wi...

11/08/2026

Gravity Media Providing Host Broadcast Operations for Esports World Cup 2026

Gravity Media is providing host broadcast operations for the Esports World Cup (EWC) 2026, supporting ESL FACEIT Group (EFG) in delivering the tournament to glo...

11/08/2026

TAG Video Systems Reduces Memory Requirements for Compressed Workflows

TAG Video Systems has released a memory usage optimization for its MCM software that reduces the cost per channel by up to two-thirds for compressed workflows. ...

11/08/2026

SMT Expands LED Production and Scoring Services at 2026 Cincinnati Open

SMT is expanding its technology partnership with the Cincinnati Open this year, adding new LED displays, production capabilities, and fan-facing experiences at ...

11/08/2026

Pac-12 to Stream Fall Olympic Sports Home Events Free on YouTube

The Pac-12 has announced that home Olympic sports broadcasts throughout the fall season will be available free on the conference's YouTube channel at youtub...