Sony Pixel Power calrec Sony

Hugging Face and NVIDIA to Accelerate Open-Source AI Robotics Research and Development

06/11/2024

At the Conference for Robot Learning (CoRL) in Munich, Germany, Hugging Face and NVIDIA announced a collaboration to accelerate robotics research and development by bringing together their open-source robotics communities.

Hugging Face's LeRobot open AI platform combined with NVIDIA AI, Omniverse and Isaac robotics technology will enable researchers and developers to drive advances across a wide range of industries, including manufacturing, healthcare and logistics.

Open-Source Robotics for the Era of Physical AI The era of physical AI - robots understanding physical properties of environments - is here, and it's rapidly transforming the world's industries.

To drive and sustain this rapid innovation, robotics researchers and developers need access to open-source, extensible frameworks that span the development process of robot training, simulation and inference. With models, datasets and workflows released under shared frameworks, the latest advances are readily available for use without the need to recreate code.

Hugging Face's leading open AI platform serves more than 5 million machine learning researchers and developers, offering tools and resources to streamline AI development. Hugging Face users can access and fine-tune the latest pretrained models and build AI pipelines on common APIs with over 1.5 million models, datasets and applications freely accessible on the Hugging Face Hub.

LeRobot, developed by Hugging Face, extends the successful paradigms from its Transformers and Diffusers libraries into the robotics domain. LeRobot offers a comprehensive suite of tools for sharing data collection, model training and simulation environments along with designs for low-cost manipulator kits.

NVIDIA's AI technology, simulation and open-source robot learning modular framework such as NVIDIA Isaac Lab can accelerate the LeRobot's data collection, training and verification workflow. Researchers and developers can share their models and datasets built with LeRobot and Isaac Lab, creating a data flywheel for the robotics community.

Scaling Robot Development With Simulation Developing physical AI is challenging. Unlike language models that use extensive internet text data, physics-based robotics relies on physical interaction data along with vision sensors, which is harder to gather at scale. Collecting real-world robot data for dexterous manipulation across a large number of tasks and environments is time-consuming and labor-intensive.

Making this easier, Isaac Lab, built on NVIDIA Isaac Sim, enables robot training by demonstration or trial-and-error in simulation using high-fidelity rendering and physics simulation to create realistic synthetic environments and data. By combining GPU-accelerated physics simulations and parallel environment execution, Isaac Lab provides the ability to generate vast amounts of training data - equivalent to thousands of real-world experiences - from a single demonstration.

Generated motion data is then used to train a policy with imitation learning. After successful training and validation in simulation, the policies are deployed on a real robot, where they are further tested and tuned to achieve optimal performance.

This iterative process leverages real-world data's accuracy and the scalability of simulated synthetic data, ensuring robust and reliable robotic systems.

By sharing these datasets, policies and models on Hugging Face, a robot data flywheel is created that enables developers and researchers to build upon each other's work, accelerating progress in the field.

The robotics community thrives when we build together, said Animesh Garg, assistant professor at Georgia Tech. By embracing open-source frameworks such as Hugging Face's LeRobot and NVIDIA Isaac Lab, we accelerate the pace of research and innovation in AI-powered robotics.

Fostering Collaboration and Community Engagement The planned collaborative workflow involves collecting data through teleoperation and simulation in Isaac Lab, storing it in the standard LeRobotDataset format. Data generated using GR00T-Mimic, will then be used to train a robot policy with imitation learning, which is subsequently evaluated in simulation. Finally, the validated policy is deployed on real-world robots with NVIDIA Jetson for real-time inference.

The initial steps in this collaboration have already been taken, having shown a physical picking setup with LeRobot software running on NVIDIA Jetson Orin Nano, providing a powerful, compact compute platform for deployment.

Combining Hugging Face open-source community with NVIDIA's hardware and Isaac Lab simulation has the potential to accelerate innovation in AI for robotics, said Remi Cadene, principal research scientist at LeRobot.

This work builds on NVIDIA's community contributions in generative AI at the edge, supporting the latest open models and libraries, such as Hugging Face Transformers, optimizing inference for large language models (LLMs), small language models (SLMs) and multimodal vision-language models (VLMs), along with VLM's action-based variants of vision language action models (VLAs), diffusion policies and speech models - all with strong, community-driven support.

Together, Hugging Face and NVIDIA aim to accelerate the work of the global ecosystem of robotics researchers and developers transforming industries ranging from transportation to manufacturing and logistics.

Learn about NVIDIA's robotics research papers at CoRL, including VLM integration for better environmental understanding, temporal navigation and long-horizon planning. Check out workshops at CoRL with NVIDIA researchers.
LINK: https://blogs.nvidia.com/blog/hugging-face-lerobot-open-source-robotic...
See more stories from nvidia

North America Stories

06/08/2026

Hisense Adds Dolby Vision 2 to Select Models

Share Copy link Facebook X Linkedin Bluesky Email...

06/08/2026

MediaKind to showcase one unified portfolio at IBC2026

At IBC2026, MediaKind will make its first major appearance as a unified global powerhouse in video, showcasing one of the world's most comprehensive video i...

06/08/2026

Big Blue Marble to unveil AI-assisted piracy detection an...

Big Blue Marble (#5.A63) will demonstrate how its integrated technology and operational expertise help media companies scale premium services with less complexi...

06/08/2026

NIH expected to award Scripps Research nearly $4.2 million over 5 years to advance tools for vaccine design

LA JOLLA, CA-Scripps Research has received more than $500,000 in first-year fund...

06/08/2026

Improving vaccine design for Ebola, HIV and more

LA JOLLA, CA-Viruses are masters at invading our cells thanks to specialized proteins that coat their surfaces. When scientists design vaccines, they often crea...

06/08/2026

How a chemical reaction triggers brain inflammation in Alzheimer's disease

LA JOLLA, CA-The brain has its own immune system, which detects threats and mounts a defense. A growing body of evidence has shown that in Alzheimer's disea...

06/08/2026

Jin-Quan Yu elected to the National Academy of Sciences

LA JOLLA, CA-Scripps Research chemist Jin-Quan Yu has been elected to the National Academy of Sciences (NAS), one of the highest honors a scientist can achieve....

06/08/2026

Scripps Research ranks third in 2026 Cure Innovation Index

LA JOLLA, CA-Scripps Research ranked third in the inaugural 2026 Cure Innovation Index recognizing the top-performing institutes and centers across the United S...

06/08/2026

Scripps Research chemist Benjamin Cravatt elected to American Philosophical Society

Benjamin Cravatt, the Gilula Chair of Chemical Biology and a professor of chemis...

06/08/2026

Scripps Research immunologist Dennis Burton elected to American Academy of Arts and Sciences

LA JOLLA, CA-Dennis Burton, professor and the James & Jessie Minor Chair in Immu...

06/08/2026

How changes to proteins can alter drug interactions for new precision therapies

LA JOLLA, CA-Inside every human cell, proteins are constantly being tagged with small chemical modifications after they're produced. Known as post-translati...

06/08/2026

Scripps Research establishes endowed chair honoring renowned structural biologist Ian Wilson

LA JOLLA-Scripps Research has established the Ian Wilson Endowed Chair, a new fa...

06/08/2026

Scripps Research's Skaggs Graduate School awards doctoral degrees to 34th graduating class

Scripps Research's Skaggs Graduate School of Chemical and Biological Science...

06/08/2026

Scripps Research chemist Jin-Quan Yu is named a Fellow of the Royal Society

LA JOLLA, CA-Professor Jin-Quan Yu of Scripps Research has been elected to the Fellowship of the Royal Society, the U.K.'s national academy of sciences and ...

06/08/2026

Experimental HIV vaccine achieves a long-sought goal

LA JOLLA, CA-For years, researchers have been hoping for vaccines that protect people against not just one strain of HIV, but every strain of the quickly mutati...

06/08/2026

Calibr-Skaggs advances CLF065, a regenerative GLP-2 therapy, into two Phase 2 IBD studies

LA JOLLA, CA-The Calibr-Skaggs Institute for Innovative Medicines, the nonprofit...

06/08/2026

Chemists snap together complex 3D molecules from highly reactive radicals'-without losing their shape

LA JOLLA, CA-Building the complex 3D molecules needed for new medicines has alwa...

06/08/2026

A fentanyl countermeasure that adapts to combat future black-market drugs

LA JOLLA, CA-Fentanyl and related variants of the synthetic opioid kill more Americans each year than car accidents and gun violence combined. In too-high doses...

06/08/2026

Two Scripps Research assistant professors named 2026 Baxter Young Investigators

LA JOLLA, CA-What do decoding communication between organs and reimagining the future of genome editing have in common? They're among the scientific questio...

06/08/2026

Calibr-Skaggs awarded $5.1M by NIH to develop long-acting hepatitis B virus therapy

LA JOLLA, CA-Of the 1.2 million people living with HIV in the United States, app...

06/08/2026

Lab studies explain how new cancer drug works as it enters patient testing

LA JOLLA, CA-For some people, cancer immunotherapies are life-changing. These treatments can turn the body's own immune system against a tumor, either elimi...

06/08/2026

Newly identified molecule strengthens the eye's response to damage in retinal disease

LA JOLLA, CA-Many conditions that cause vision loss share a common feature: the ...

06/08/2026

Molecular scissors caught in action: A structural blueprint for RNA therapeutics

LA JOLLA, CA-RNA interference is a natural mechanism for living cells to control whether specific genes are being used or not. Crowned with the 2006 Nobel Priz...

06/08/2026

Immune molecule may drive excessive drinking in alcohol use disorder

LA JOLLA, CA-The drugs that keep rheumatoid arthritis in check may one day help people stop drinking. A new Scripps Research study shows that an anti-inflammato...

06/08/2026

Back in action: Researchers make drug-resistant bacteria vulnerable again

LA JOLLA, CA-Antibiotic resistance is one of the most urgent threats to global health, linked to an estimated 4.7 million deaths worldwide in 2019 alone. As mor...

06/08/2026

Scripps Research scientists demonstrate a faster, cheaper route to making critical drugs using common table sugar

LA JOLLA, CA-Some of the world's best-selling diabetes drugs depend on a che...

06/08/2026

Scripps Research scientists awarded $2M to advance global disease surveillance

LA JOLLA, CA-Detecting infectious disease threats early and responding quickly can dramatically alter the course of an infectious outbreak. Technologies such as...

06/08/2026

Joan Pulupa joins Scripps Research faculty to study the organization of DNA in brain cells and its links to neurodegeneration

LA JOLLA, CA-Molecular biophysicist Joan Pulupa will join Scripps Research in Ja...

06/08/2026

Scripps Research scientists train the immune system to make antibodies against numerous HIV strains

LA JOLLA, CA-HIV is globally so diverse, consisting of hundreds of thousands of ...

06/08/2026

ASG Ups Michele Ferreira to Chief Business Officer

Share Copy link Facebook X Linkedin Bluesky Email...

06/08/2026

WNBC and WNJU Expand New York Giants Deal

Share Copy link Facebook X Linkedin Bluesky Email...

06/08/2026

Utah Scientific to Highlight NBOSS at IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

06/08/2026

Disney, TikTok Ink Global Short-Form Content-Sharing Deal

Share Copy link Facebook X Linkedin Bluesky Email...

06/08/2026

Kane Peterson Joins QuickLink's North America Team

Share Copy link Facebook X Linkedin Bluesky Email...

06/08/2026

FCC Returns $881 Million in Unused TV Broadcaster Relocation Funds

Share Copy link Facebook X Linkedin Bluesky Email...

06/08/2026

PlayBox Neo to highlight secure and scalable workflow del...

At this year's SET EXPO, PlayBox Neo will present recent innovations across its PlayBox Neo Suite and integrated range of broadcast media solutions. By show...

06/08/2026

Modern Streaming Solutions Private Limited Partners with...

Live demo at IBC2026 VisualOn Booth, Hall 5, Stand A55 Amsterdam, Netherlands Modern Streaming Solutions Private Limited, a rising force in India's dig...

06/08/2026

How Karukera Studio Built a Media Production Hub with SNS EVO

How Karukera Studio Built a Media Production Hub with SNS EVO Melanie Ciotti August 5, 2026 0 Comments Hero images displays Karukera Studio in Sainte-...

06/08/2026

NHK drama series Rosanjin no Kamado shot on PYXIS 6K

NHK drama series Rosanjin no Kamado shot on PYXIS 6K Brie Clayton August 5, 2026 0 Comments Blackmagic PYXIS 6K and DaVinci Resolve Studio capture the...

06/08/2026

Camera Match: AutoSetup 3d Plugin / script for Photomontages- Cinema 4D

Camera Match: AutoSetup 3d Plugin / script for Photomontages- Cinema 4D Jamie Cardoso August 5, 2026 0 Comments If you do any kind of Architectural Vi...

06/08/2026

Into the Omniverse: How Open World Models Push the Frontier of Physical AI

Editor's note: This post is part of Into the Omniverse, a series focused on how developers, 3D practitioners and enterprises can transform their workflows u...

06/08/2026

GeForce NOW Shakes Up August With 26 New Games

August is here, bringing 26 new games for GeForce NOW members. Command the seas in World of Warships: Legends and discover what's next in the GeForce NOW ...

05/08/2026

SVG Regional Sports Production Summit 2026: All Sessions Now Available to Watch on SVG PLAY

RSNs, teams, leagues, and streamers explore the present and future of local spor...

05/08/2026

SVG New Sponsor Spotlight: Hitachi Vantara's Lenny Khaitov on Building Resilient Data Infrastructure for Sports Production

As sports-production workflows generate larger volumes of unstructured data and ...

05/08/2026

NESN's Women's Celebration Night Returns With Expanded Focus on the Next Generation

Three aspiring broadcasters will contribute on-camera and inside the production ...

05/08/2026

Union County Paints an Authentic Picture of Recovery and Community

Adam Meeks attends the premiere of his film Union County, an official selection of the 2026 Sundance Film Festival. (Photo by Jemal Countess/Sundance Institut...

05/08/2026

Australia's retirement sector advertising rises more than 10% as millions plan for life after work according to Nielsen

Advertisers invested $53.6 million in the retirement sector in the last 12 month...

05/08/2026

New Nielsen data shows IKEA's New Zealand launch attracted more than 243,000 shoppers in the last month

Nielsen CMI and Ad Intel data reveals who's visiting IKEA, where else they s...

05/08/2026

Nielsen expands Four-Screen Ad Deduplication across CTV in Japan

Media buyers and sellers can now compare deduplicated campaign reach across computer, mobile, connected TV and linear TV across all measurable CTV publishers in...

05/08/2026

Stranger Things,' Bluey,' The Pitt' Top Nielsen's Streaming Charts Through First Half of 2026

Stranger Things' Leads All Streaming Titles with 23.3 Billion Minutes in Fir...