Sony Pixel Power calrec Sony

Into the Omniverse: How OpenUSD-Based Simulation and Synthetic Data Generation Advance Robot Learning

11/12/2024

Editor's note: This post is part of Into the Omniverse, a series focused on how developers, 3D practitioners, and enterprises can transform their workflows using the latest advances in OpenUSD and NVIDIA Omniverse.

Scalable simulation technologies are driving the future of autonomous robotics by reducing development time and costs.

Universal Scene Description (OpenUSD) provides a scalable and interoperable data framework for developing virtual worlds where robots can learn how to be robots. With SimReady OpenUSD-based simulations, developers can create limitless scenarios based on the physical world.

And NVIDIA Isaac Sim is advancing perception AI-based robotics simulation. Isaac Sim is a reference application built on the NVIDIA Omniverse platform for developers to simulate and test AI-driven robots in physically based virtual environments.

At AWS re:Invent, NVIDIA announced that Isaac Sim is now available on Amazon EC2 G6e instances powered by NVIDIA L40S GPUs. These powerful instances enhance the performance and accessibility of Isaac Sim, making high-quality robotics simulations more scalable and efficient.

These advancements in Isaac Sim mark a significant leap for robotics development. By enabling realistic testing and AI model training in virtual environments, companies can reduce time to deployment and improve robot performance across a variety of use cases.

Advancing Robotics Simulation With Synthetic Data Generation Robotics companies like Cobot, Field AI and Vention are using Isaac Sim to simulate and validate robot performance while others, such as SoftServe and Tata Consultancy Services, use synthetic data to bootstrap AI models for diverse robotics applications.

The evolution of robot learning has been deeply intertwined with simulation technology. Early experiments in robotics relied heavily on labor-intensive, resource-heavy trials. Simulation is a crucial tool for the creation of physically accurate environments where robots can learn through trial and error, refine algorithms and even train AI models using synthetic data.

Physical AI describes AI models that can understand and interact with the physical world. It embodies the next wave of autonomous machines and robots, such as self-driving cars, industrial manipulators, mobile robots, humanoids and even robot-run infrastructure like factories and warehouses.

Robotics simulation, which forms the second computer in the three computer solution, is a cornerstone of physical AI development that lets engineers and researchers design, test and refine systems in a controlled virtual environment.

A simulation-first approach significantly reduces the cost and time associated with physical prototyping while enhancing safety by allowing robots to be tested in scenarios that might otherwise be impractical or hazardous in real life.

With a new reference workflow, developers can accelerate the generation of synthetic 3D datasets with generative AI using OpenUSD NIM microservices. This integration streamlines the pipeline from scene creation to data augmentation, enabling faster and more accurate training of perception AI models.

Synthetic data can help address the challenge of limited, restricted or unavailable data needed to train various types of AI models, especially in computer vision. Developing action recognition models is a common use case that can benefit from synthetic data generation.

To learn how to create a human action recognition video dataset with Isaac Sim, check out the technical blog on Scaling Action Recognition Models With Synthetic Data. 3D simulations offer developers precise control over image generation, eliminating hallucinations.

Robotic Simulation for Humanoids Humanoid robots are the next wave of embodied AI, but they present a challenge at the intersection of mechatronics, control theory and AI. Simulation is crucial to solving this challenge by providing a safe, cost-effective and versatile platform for training and testing humanoids.

With NVIDIA Isaac Lab, an open-source unified framework for robot learning built on top of Isaac Sim, developers can train humanoid robot policies at scale via simulations. Leading commercial robot makers are adopting Isaac Lab to handle increasingly complex movements and interactions.

NVIDIA Project GR00T, an active research initiative to enable the humanoid robot ecosystem of builders, is pioneering workflows such as GR00T-Gen to generate robot tasks and simulation-ready environments in OpenUSD. These can be used for training generalist robots to perform manipulation, locomotion and navigation.

Recently published research from Project GR00T also shows how advanced simulation can be used to train interactive humanoids. Using Isaac Sim, the researchers developed a single unified controller for physically simulated humanoids called MaskedMimic. The system is capable of generating a wide range of motions across diverse terrains from intuitive user-defined intents.

Physics-Based Digital Twins Simplify AI Training Partners across industries are using Isaac Sim, Isaac Lab, Omniverse, and OpenUSD to design, simulate and deploy smarter, more capable autonomous machines:

Agility uses Isaac Lab to create simulations that let simulated robot behaviors transfer directly to the robot, making it more intelligent, agile and robust when deployed in the real world.

Cobot uses Isaac Sim with its AI-powered cobot, Proxie, to optimize logistics in warehouses, hospitals, manufacturing sites and more.

Cohesive Robotics has integrated Isaac Sim into its software framework called Argus OS for developing and deploying robotic workcells used in high-mix manufacturing environments.

Field AI, a builder of robot foundation models, uses Isaac Sim and Isaac Lab to evaluate the performance of its models in complex, unstructured environments across industries such as construction,
LINK: https://blogs.nvidia.com/blog/openusd-sdg-advance-robot-learning/...
See more stories from nvidia

Most recent headlines

09/11/2025

Dalet Unveils Agentic AI Media Workflows at IBC2025

Dalet today announced a transformative leap forward for media operations: Agentic Artificial Intelligence (AI) that unifies the Dalet ecosystem under one natura...

21/10/2025

Globecast Appoints G Morgan as EVP of Sales for Globecast...

Globecast, the leading provider of broadcast, media and entertainment managed services, has announced the appointment of G Morgan as Executive Vice President of...

21/10/2025

Visual Data Strengthens Global Leadership with Appointmen...

Visual Data announces the appointment of Maz Al-Jumaili as Senior Vice President, Worldwide Localization, to advance client engagement, strategic partnerships, ...

21/10/2025

Grup Mediapro to Deliver Prime Video NBA Coverage to the Hispanic Market

As Amazon's Prime Video prepares to launch its coverage of NBA basketball under a major new deal, Grup Mediapro has announced that it is working with the st...

21/10/2025

ADTH to Upgrade NextGen TV Receivers With Gateway Capabilities

ATLANTA Good news for consumers using an Atlanta DTH receiver to watch ATSC 3.0: with a new software update, they will be able to blanket their homes with Wi-Fi...

21/10/2025

Study: 'More Critical Than Ever for Brands to Focus on Hispanics

While recent news has been heavily focused on Hispanic migration into the U.S., The 2025 Hispanic Market Report from Claritas highlights the fact that this gr...

21/10/2025

RWS Appoints Michael Wayne as Head of Media and Entertainment

MAIDENHEAD, UK RWS has hired Michael Wayne as its head of media and entertainment in Los Angeles where he will lead the company's media localization busines...

21/10/2025

Imagine Communications Acquires Pixel Power From Rohde an...

Imagine Communications and Rohde & Schwarz today announced a definitive agreement under which Imagine will acquire Pixel Power Limited, a wholly owned subsidiar...

21/10/2025

ADTH Announces New NEXTGEN TV Gateway Receiver Implementi...

Atlanta DTH (ADTH) today announced a major update that will expand the functionality of its NEXTGEN TV receiver by enabling gateway capabilities allowing viewer...

21/10/2025

Heartland Video Systems Partners with Zixi for Resilient...

Heartland Video Systems, Inc. (HVS), a premier video systems integration, consulting, and expert ATSC 3.0 implementation firm announces that it has partnered wi...

21/10/2025

QuickLink Appoints Austin Hinton as Solutions Enablement...

QuickLink, the leading provider of award-winning video production and remote guest integration solutions, today announced the appointment of Austin Hinton as it...

21/10/2025

Miri Technologies to Unveil New Resilient Internet Platfo...

nternet connectivity startup Miri Technologies Inc. will use this week's NAB Show New York as the launch pad for its latest ground-breaking innovation, the ...

20/10/2025

Inside TAMS: How Time-Addressable Media Stores Could Redefine Sports Workflows

Inside TAMS: How Time-Addressable Media Stores could redefine sports workflows By Paul Markham Friday, October 17, 2025 - 08:57 Print This Story A penalty...

20/10/2025

Transformational Production: Inside TVN's Remote Production Push for the DFL's Bundesliga 2

Transformational production: Inside TVN's remote production push for the DFL...

20/10/2025

How NBC Sports Transitioned Stamford Facility to One Format: 1080p HDR

How NBC Sports Transitioned Stamford Facility to One Format: 1080p HDRMulti-year plan harmonizes workflows, simplifies operationsBy Ken Kerschbaumer, Editorial ...

20/10/2025

NBA on NBC' Studio Production Team Is Ready for Tip-Off With Coast-to-Coast Tuesday'

NBA on NBC' Studio Production Team Is Ready for Tip-Off With Coast-to-Coast...

20/10/2025

Under Pressure: TVN CEO Markus Osthaus Considers the German Sports Broadcasting Market

Under pressure: TVN CEO Markus Osthaus considers the German sports broadcasting ...

20/10/2025

Carmen Emmi's Plainclothes Evokes the Rawness and Sensuality of New Queer Cinema

(L-R) Maria Dizzia, Carmen Emmi, and Russell Tovey attend the Plainclothes pre...

20/10/2025

Search and Follow Your Favorite Venues on Spotify

In March, we launched Concerts Near You to help listeners find concerts from their favorite artists. Since then, more than 3 million people have used it to disc...

20/10/2025

Arte Preta Brasileira Destaque No Dia AMPLIFIKA 2025

Em diversas cidades do Brasil, um movimento tem se fortalecido para celebrar o poder, a beleza e a profundidade da criatividade negra. O Dia AMPLIFIKA, agora em...

20/10/2025

Black Brazilian Artistry Takes Center Stage During AMPLIFIKA Day 2025

In cities across Brazil, a movement is growing that celebrates the power, beauty, and depth of Black creativity. AMPLIFIKA Day, now in its fifth edition, return...

20/10/2025

The Republic of Korea Selects L3Harris for Airborne Early Warning and Control Aircraft Program

Airborne Early Warning and Control aircraft rendering...

20/10/2025

Imagine Communications Acquires Pixel Power From Rohde & Schwarz

DENVER and MUNICH Imagine Communications today announced its plans to acquire Pixel Power Ltd., a wholly owned subsidiary of Rohde & Schwarz. Financial terms of...

20/10/2025

Globecast Appoints G Morgan as EVP of Sales, Globecast Americas

LOS ANGELES G Morgan has joined Globecast, a provider of broadcast, media and entertainment managed services, as executive vice president of sales, Globecast Am...

20/10/2025

Heartland Video Systems, Zixi Partner on IP Networking for Broadcasters

PLYMOUTH, Wisc. Heartland Video Systems and Zixi have partnered to enable broadcast-quality live video delivery over any IP network....

20/10/2025

A. R. Rahman on Facing Fear and Finding the Divine

A. R. Rahman on Facing Fear and Finding the Divine In an interview with Berklee President Jim Lucchese, the Oscar-winning composer reflects on how courage and...

20/10/2025

Sky unveils thrilling trailer of Original film Nuremberg, starring Russell Crowe, Rami Malek and Michael Shannon

Monday 20 October 2025 To view this content, please enable our use of cookies. ...

20/10/2025

Rohde & Schwarz transfers Pixel Power to Imagine Communications

Rohde & Schwarz transfers Pixel Power to Imagine Communications Companies work collaboratively to ensure continuity and ongoing support for existing customers...

20/10/2025

RT Prime Time to host final Presidential Debate

RT 's Prime Time is set to host the final Presidential Election Debate this Tuesday night, October 21, providing an opportunity to hear directly from Irelan...

20/10/2025

NVIDIA and Google Cloud Accelerate Enterprise AI and Industrial Digitalization

NVIDIA and Google Cloud are expanding access to accelerated computing to transform the full spectrum of enterprise workloads, from visual computing to agentic a...

19/10/2025

Sins of Kujo' Comes to Life in New Live-Action Series Set for Spring 2026

Back to All News Sins of Kujo' Comes to Life in New Live-Action Series Set for Spring 2026 Entertainment 19 October 2025 GlobalJapan Link copied to cl...

18/10/2025

NESN Taps Harmonic for Primary Live Sports Distribution

New England Sports Network (NESN) has chosen Harmonic, working with Astound Business Solutions, as its enterprise technology partner to transform primary distri...

18/10/2025

DirecTV Launches Gray's Gulf Coast Sports & Entertainment Network

NEW ORLEANS, La. In the run-up to the start of the NBA season, WVUE-TV and Gray Local Media have announced a deal with DirecTV that will greatly expand access t...

18/10/2025

Berklee Celebrates 40 Years of the Fall Together Concert

Berklee Celebrates 40 Years of the Fall Together Concert Faculty composers Bob Pilkington and Greg Hopkins are among the featured artists for this year's ...

17/10/2025

NEP Group Receives New Equity Investment From 26North Partners LP, Co-Investors

NEP Group Receives New Equity Investment From 26North Partners LP, Co-InvestorsCarlyle remains the largest shareholder as the company prepares for the futureBy ...

17/10/2025

Apple Lands Five-Year Deal for F1 Distribution in the U.S.

Apple Lands Five-Year Deal for F1 Distribution in the U.S.Besides airing on Apple TV, the sport will be amplified on other Apple servicesBy Ken Kerschbaumer, Ed...

17/10/2025

SVG Sit-Down: Marshall Electronics' Bernie Keach on the Future of PTZ Cameras

SVG Sit-Down: Marshall Electronics' Bernie Keach on the Future of PTZ Camera...

17/10/2025

L2 Productions' REMI Facility in Austin Can Produce Content From Anywhere

L2 Productions' REMI Facility in Austin Can Produce Content From AnywhereMusic festivals, sports events are produced via flypacks and remote control roomsBy...

17/10/2025

Give Me the Backstory: Get to Know Sarah Dowland, the Filmmaker Behind Sue Bird: In The Clutch

By Lucy Spicer One of the most exciting things about the Sundance Film Festival...

17/10/2025

Cooper Raiff Returns to the Sundance Film Festival With His Independent Series Hal & Harper

(L-R) Christopher Meyer, Addison Timlin, Cooper Raiff, Lili Reinhart, Alyah Chan...

17/10/2025

Ferramenta de arte da capa de playlists do Spotify chega ao Brasil com uma noite de autoexpresso

M sica e arte se uniram em uma noite especial na semana passada na ZIV Gallery, ...

17/10/2025

Spotify's Custom Playlist Cover Art Tool Arrives in Brazil With a Night of Self-Expression

Music and art came together for one special night last week at ZIV Gallery, an i...

17/10/2025

Spotify and FC Barcelona Extend Partnership Through 2030

Spotify and FC Barcelona are extending our partnership through 2030, continuing a collaboration that's redefining how fans, players, and artists connect. Th...

17/10/2025

Sports Fishing Championship Deploys DigitalGlue Storage Platform

MURRIETA, Calif. The Sports Fishing Championship (SFC) has deployed DigitalGlue's creative.space storage platform to streamline video production by centrali...

17/10/2025

TV Ad Impressions for Football Spiked in Q3

BELLEVUE, Wash. Football continued to cement its reputation as a bulwark of TV advertising in Q3 2025 with new data from iSpot that showed both the NFL and coll...

17/10/2025

Reeling in the Chaos Sports Fishing Championship Simplifi...

The Sports Fishing Championship (SFC), the premier competitive saltwater fishing series, has transformed its production workflow by adopting creative.space, the...

17/10/2025

QuickLink Unveils StudioPro Version 4 With Major Enhancem...

QuickLink, a leading provider of award-winning multi-camera video productions and remote contribution solutions, announces the release of StudioPro Version 4, ...

17/10/2025

Westcoast Pixel dazzles with dynamic 3D video projections

Although the annual Grammy Awards celebration is best known for recognizing achievements in the recording industry, the show often proves a visual spectacle as ...

17/10/2025

Alex Dunfey Promoted to CTO at OpenDrives

OpenDrives, Inc., a leading provider of software-defined data storage and data services, has promoted Alex Dunfey to Chief Technology Officer (CTO) from his for...