Sony Pixel Power calrec Sony

Hugging Face and NVIDIA to Accelerate Open-Source AI Robotics Research and Development

06/11/2024

At the Conference for Robot Learning (CoRL) in Munich, Germany, Hugging Face and NVIDIA announced a collaboration to accelerate robotics research and development by bringing together their open-source robotics communities.

Hugging Face's LeRobot open AI platform combined with NVIDIA AI, Omniverse and Isaac robotics technology will enable researchers and developers to drive advances across a wide range of industries, including manufacturing, healthcare and logistics.

Open-Source Robotics for the Era of Physical AI The era of physical AI - robots understanding physical properties of environments - is here, and it's rapidly transforming the world's industries.

To drive and sustain this rapid innovation, robotics researchers and developers need access to open-source, extensible frameworks that span the development process of robot training, simulation and inference. With models, datasets and workflows released under shared frameworks, the latest advances are readily available for use without the need to recreate code.

Hugging Face's leading open AI platform serves more than 5 million machine learning researchers and developers, offering tools and resources to streamline AI development. Hugging Face users can access and fine-tune the latest pretrained models and build AI pipelines on common APIs with over 1.5 million models, datasets and applications freely accessible on the Hugging Face Hub.

LeRobot, developed by Hugging Face, extends the successful paradigms from its Transformers and Diffusers libraries into the robotics domain. LeRobot offers a comprehensive suite of tools for sharing data collection, model training and simulation environments along with designs for low-cost manipulator kits.

NVIDIA's AI technology, simulation and open-source robot learning modular framework such as NVIDIA Isaac Lab can accelerate the LeRobot's data collection, training and verification workflow. Researchers and developers can share their models and datasets built with LeRobot and Isaac Lab, creating a data flywheel for the robotics community.

Scaling Robot Development With Simulation Developing physical AI is challenging. Unlike language models that use extensive internet text data, physics-based robotics relies on physical interaction data along with vision sensors, which is harder to gather at scale. Collecting real-world robot data for dexterous manipulation across a large number of tasks and environments is time-consuming and labor-intensive.

Making this easier, Isaac Lab, built on NVIDIA Isaac Sim, enables robot training by demonstration or trial-and-error in simulation using high-fidelity rendering and physics simulation to create realistic synthetic environments and data. By combining GPU-accelerated physics simulations and parallel environment execution, Isaac Lab provides the ability to generate vast amounts of training data - equivalent to thousands of real-world experiences - from a single demonstration.

Generated motion data is then used to train a policy with imitation learning. After successful training and validation in simulation, the policies are deployed on a real robot, where they are further tested and tuned to achieve optimal performance.

This iterative process leverages real-world data's accuracy and the scalability of simulated synthetic data, ensuring robust and reliable robotic systems.

By sharing these datasets, policies and models on Hugging Face, a robot data flywheel is created that enables developers and researchers to build upon each other's work, accelerating progress in the field.

The robotics community thrives when we build together, said Animesh Garg, assistant professor at Georgia Tech. By embracing open-source frameworks such as Hugging Face's LeRobot and NVIDIA Isaac Lab, we accelerate the pace of research and innovation in AI-powered robotics.

Fostering Collaboration and Community Engagement The planned collaborative workflow involves collecting data through teleoperation and simulation in Isaac Lab, storing it in the standard LeRobotDataset format. Data generated using GR00T-Mimic, will then be used to train a robot policy with imitation learning, which is subsequently evaluated in simulation. Finally, the validated policy is deployed on real-world robots with NVIDIA Jetson for real-time inference.

The initial steps in this collaboration have already been taken, having shown a physical picking setup with LeRobot software running on NVIDIA Jetson Orin Nano, providing a powerful, compact compute platform for deployment.

Combining Hugging Face open-source community with NVIDIA's hardware and Isaac Lab simulation has the potential to accelerate innovation in AI for robotics, said Remi Cadene, principal research scientist at LeRobot.

This work builds on NVIDIA's community contributions in generative AI at the edge, supporting the latest open models and libraries, such as Hugging Face Transformers, optimizing inference for large language models (LLMs), small language models (SLMs) and multimodal vision-language models (VLMs), along with VLM's action-based variants of vision language action models (VLAs), diffusion policies and speech models - all with strong, community-driven support.

Together, Hugging Face and NVIDIA aim to accelerate the work of the global ecosystem of robotics researchers and developers transforming industries ranging from transportation to manufacturing and logistics.

Learn about NVIDIA's robotics research papers at CoRL, including VLM integration for better environmental understanding, temporal navigation and long-horizon planning. Check out workshops at CoRL with NVIDIA researchers.
LINK: https://blogs.nvidia.com/blog/hugging-face-lerobot-open-source-robotic...
See more stories from nvidia

North America Stories

02/04/2026

Evergent showcases innovations in sports streaming and mo...

Evergent will showcase new innovations in subscriber lifecycle management and monetization at NAB Show 2026 (Las Vegas, April 18 22), including: New advances i...

02/04/2026

Binghamton University Strengthens Student Run Productions...

WUPPERTAL, Germany April 1, 2026 Riedel Communications is proud to be part of Binghamton University, State University of New York, Athletics' milestone...

02/04/2026

Techex and Encompass Launch Industry-Leading Cloud-Based...

Encompass Digital Media and Techex have today announced new, fully managed, cloud-native Master Control services designed to meet the growing operational demand...

02/04/2026

Winning in the new media economy - Avid debuts fully avai...

Avid today announced it will showcase new innovations designed to help media companies win in the new media economy at NAB Show 2026 (April 18 22, Las Vegas Co...

02/04/2026

PlayBox Neo reinforces MIMO Tech with new Playout capabil...

PlayBox Neo helps AIS PLAY kick-off premier football content direct to fans PlayBox Neo has provided MIMO Tech with a brand-new major installation to extend it...

02/04/2026

Globo transitions primary distribution to SRT over IP wit...

Globo has transitioned its primary content distribution to Secure Reliable Transport over a fully IP-based managed backbone using Synamedia's Quortex PowerV...

02/04/2026

Nexstar Says Pausing Tegna Merger Creates 'Impossible' Challenges

Share Copy link Facebook X Linkedin Bluesky Email...

02/04/2026

FCC Launches Efforts to Strengthen U.S. Drone Ecosystem

Share Copy link Facebook X Linkedin Bluesky Email...

02/04/2026

WAPA+ to Launch on Dish, DishLatino, Sling TV and Sling Freestream

Share Copy link Facebook X Linkedin Bluesky Email...

02/04/2026

Student Spotlight: Al-Fadl Salem

Student Spotlight: Al-Fadl Salem The Danish singer recently performed for the queen of Denmark. April 1, 2026 By Editorial Staff Image by Junia Morrow Wh...

02/04/2026

Taku Hirano's Career Is Defined by Identity

Taku Hirano's Career Is Defined by Identity Whether he's performing, composing, teaching, or developing instruments, the do-it-all percussionist sees ...

02/04/2026

Press Start on April: GeForce NOW Brings 10 Games to the Cloud

No joke - GFN Thursday is skipping the tricks and heading straight into the games. April kicks off with ten new titles, bringing fresh adventures to GeForce NOW...

01/04/2026

SVG New Sponsor Spotlight: Flowstate AI's Sahil Shah on Transforming Video Content with Intelligent AI Agents

As sports media organization continue to seek out new ways to streamline their p...

01/04/2026

SVG GFX Forum 2026: Sessions Now Available to Watch on SVG PLAY

The SVG GFX Forum hit New York City earlier this month for a day packed with sessions focused on the creative strategy and technology behind today's cutting...

01/04/2026

Green Hippo Debuts Hands on Hippotizer Media Server Train...

Green Hippo is excited to announce the launch of its new Hippotizer Media Server training courses at Pixel Academy, a purpose built AV learning hub combining ha...

01/04/2026

TAG Video Systems and Oracle Cloud Infrastructure Partner...

TAG Video Systems, a global leader in IP-native broadcast monitoring, multiviewing, and quality control, today announced a collaboration with Oracle Cloud Infra...

01/04/2026

Professional Wireless Systems PWS Takes on Intercom and R...

Professional Wireless Systems (PWS), a leading provider of wireless audio solutions and RF management, was on site at the Caesars Superdome in New Orleans, wher...

01/04/2026

AgileTV powers Liwest next generation TV experience with...

AgileTV, a leading provider of end-to-end TV technology solutions, has deployed next , the new IPTV platform of the Austrian telco LIWEST, marking the first st...

01/04/2026

LTN and Ateme partner to deliver integrated video process...

LTN, a leader in fully managed IP video transport, and Ateme, a global leader in video compression and delivery solutions, today announced a collaboration integ...

01/04/2026

Adobe Unveils Powerful New Innovations for Creative Pros in Adobe Illustrator

Adobe Unveils Powerful New Innovations for Creative Pros in Adobe Illustrator Deepa Subramaniam April 1, 2026 0 Comments I'm excited to share that...

01/04/2026

Boland Communications Introduces QD4K315HDR10 QD-OLED Series Monitors for Live Production, Film, Post, and Broadcast

Boland Communications Introduces QD4K315HDR10 QD-OLED Series Monitors for Live P...

01/04/2026

2026 NAB Show Exhibitor Insight: Evertz

Share Copy link Facebook X Linkedin Bluesky Email...

01/04/2026

Judge Blocks Order Barring NPR and PBS From Funding

Share Copy link Facebook X Linkedin Bluesky Email...

01/04/2026

Nikon to Sell Mark Roberts Motion Control

Share Copy link Facebook X Linkedin Bluesky Email...

01/04/2026

Mediagenix Showcases Semantic Intelligence-Powered Title Management, Schedule Optimization, and Personalization at NAB 2026

Mediagenix Showcases Semantic Intelligence-Powered Title Management, Schedule Op...

01/04/2026

FCC Approves WJAX-TV License Transfer to Cox

Share Copy link Facebook X Linkedin Bluesky Email...

01/04/2026

Scripps Sports Ink Deal for Ion to Air 2026 Teal Rising Cup

Share Copy link Facebook X Linkedin Bluesky Email...

01/04/2026

UK Group Companies Unveil NAB Show Plans

Share Copy link Facebook X Linkedin Bluesky Email...

01/04/2026

Victoria Mont Brings the Multi-Hyphenate Mindset to Career Jam 2026

Victoria Mon t Brings the Multi-Hyphenate Mindset to Career Jam 2026 The Grammy-winning singer, songwriter, and producer shared how versatility and self-inves...

01/04/2026

Rite of Spring - Save on Ivory 3 German D and American Concert D

A New Season of Expression: Save 30% On Ivory 3There are moments in music when everything changes-when new ideas break through and redefine what's possible....

01/04/2026

A Demon Falls for a Human in 'Long Vacation': Netflix Greenlights New Romance Series from Director of 'Crash Landing on You'

Back to All News A Demon Falls for a Human in Long Vacation: Netflix Greenlight...

01/04/2026

All WWE Is Now Live Only on Netflix in Italy

Back to All News All WWE Is Now Live Only on Netflix in Italy Entertainment 01 April 2026 GlobalItaly Link copied to clipboard ALL WWE IS NOW LIVE ON NET...

31/03/2026

MeyerPro Delivers LED Production for Microsoft Ignite 2025

MeyerPro delivered the LED production for the Microsoft Ignite 2025 keynote at Chase Center in San Francisco, using more than 2,700 ROE Visual panels across mul...

31/03/2026

MC-IF Expands Membership to Include Smaller Technology Companies to Drive VVC Adoption

The Media Coding Industry Forum (MC-IF) is expanding membership outreach in 2026...

31/03/2026

ViewLift and MyOutdoorTV Launch Conversational AI Search

ViewLift and MyOutdoorTV (MOTV) have announced the launch of ViewLift Conversational AI Search on the MyOutdoorTV platform. The feature enables natural language...

31/03/2026

Atlanta Braves and Gray Media to Simulcast 25 Regular Season Games on Over-the-Air Television

The Atlanta Braves and Gray Media have announced a multi-year agreement to simul...

31/03/2026

Gravity Media to Deliver Broadcast Coverage of the Sydney Autumn Racing Carnival

Gravity Media has delivered broadcast services for TAB Golden Slipper Day at Rosehill Gardens on March 21 and will now cover The Star Championships at Royal Ran...

31/03/2026

Mileto Tecnologia Selects Synamedia Go for OTT Expansion

Synamedia has announced that Mileto Tecnologia, a Brazilian pay-TV operator, has selected the Synamedia Go platform for its OTT expansion. Mileto is also deploy...

31/03/2026

NAB 2026: KMH Integration to Present AV Casting Philosophy

KMH Integration will attend NAB Show in Las Vegas with a focus on what the company calls AV Casting - an approach built on the idea that professional AV and bro...

31/03/2026

Prime Video to Air First and Second Rounds of 2026 Masters Tournament

Prime Video will carry live coverage of the first and second rounds of the 2026 Masters Tournament, beginning at 1:00 p.m. ET on Thursday, April 9 and Friday, A...

31/03/2026

NHL, ESPN, TNT Sports and Sportsnet Launch Joint Marketing Campaign for 2026 Stanley Cup Playoffs

The NHL and its U.S. and Canadian broadcast partners have announced a collaborat...

31/03/2026

Daktronics to Install New Video Display and Show Control System at Wake Forest's Allegacy Stadium

Daktronics has partnered with Wake Forest to manufacture and install a new end z...

31/03/2026

NFL Announces Las Vegas to Host Super Bowl LXIII in 2029

The National Football League has announced that Las Vegas will host Super Bowl LXIII in 2029 at Allegiant Stadium. The announcement was made at the NFL Annual M...

31/03/2026

NFL Partners with TMRW Sports to Launch Professional Flag Football League

The NFL has announced a partnership with TMRW Sports to develop and operate a professional flag football league for women and men. The league will launch with i...

31/03/2026

Zac Brown Band and Blake Shelton Deploy Sennheiser Spectera at Las Vegas Residencies

Sennheiser's Spectera wideband bidirectional wireless system was used by bot...

31/03/2026

Sports Studio Forms Advisory Board for Free Live Sports Platform

Sports Studio Inc., the company behind Free Live Sports, has announced the formation of an Advisory Board with five media and technology executives. Each of o...

31/03/2026

PSSI Deploys Appear X Platform for Mobile Fleet and Teleport Upgrade

PSSI and Appear have announced a new phase in their collaboration, with PSSI deploying Appear's X Platform to upgrade its mobile fleet and PSSI Internationa...

31/03/2026

SVG New Sponsor Spotlight: Flowstate AI's Sahiah Shah on Transforming Video Content with Intelligent AI Agents

As sports media organization continue to seek out new ways to streamline their p...

31/03/2026

NWSL Scales Up Production Ops to Match League's Exponential Growth

With two new expansion teams and viewership/attendance on the rise, the league's production/broadcast arm and its partner NEP Connected Solutions are busier...

31/03/2026

Welcome Young Entertainment Professionals Class of 2026

By Kiera Williams Every year, a new group of emerging media and entertainment professionals steps into the HPA community - curious, talented, and ready to grow...