Sony Pixel Power calrec Sony

Into the Omniverse: How OpenUSD-Based Simulation and Synthetic Data Generation Advance Robot Learning

11/12/2024

Editor's note: This post is part of Into the Omniverse, a series focused on how developers, 3D practitioners, and enterprises can transform their workflows using the latest advances in OpenUSD and NVIDIA Omniverse.

Scalable simulation technologies are driving the future of autonomous robotics by reducing development time and costs.

Universal Scene Description (OpenUSD) provides a scalable and interoperable data framework for developing virtual worlds where robots can learn how to be robots. With SimReady OpenUSD-based simulations, developers can create limitless scenarios based on the physical world.

And NVIDIA Isaac Sim is advancing perception AI-based robotics simulation. Isaac Sim is a reference application built on the NVIDIA Omniverse platform for developers to simulate and test AI-driven robots in physically based virtual environments.

At AWS re:Invent, NVIDIA announced that Isaac Sim is now available on Amazon EC2 G6e instances powered by NVIDIA L40S GPUs. These powerful instances enhance the performance and accessibility of Isaac Sim, making high-quality robotics simulations more scalable and efficient.

These advancements in Isaac Sim mark a significant leap for robotics development. By enabling realistic testing and AI model training in virtual environments, companies can reduce time to deployment and improve robot performance across a variety of use cases.

Advancing Robotics Simulation With Synthetic Data Generation Robotics companies like Cobot, Field AI and Vention are using Isaac Sim to simulate and validate robot performance while others, such as SoftServe and Tata Consultancy Services, use synthetic data to bootstrap AI models for diverse robotics applications.

The evolution of robot learning has been deeply intertwined with simulation technology. Early experiments in robotics relied heavily on labor-intensive, resource-heavy trials. Simulation is a crucial tool for the creation of physically accurate environments where robots can learn through trial and error, refine algorithms and even train AI models using synthetic data.

Physical AI describes AI models that can understand and interact with the physical world. It embodies the next wave of autonomous machines and robots, such as self-driving cars, industrial manipulators, mobile robots, humanoids and even robot-run infrastructure like factories and warehouses.

Robotics simulation, which forms the second computer in the three computer solution, is a cornerstone of physical AI development that lets engineers and researchers design, test and refine systems in a controlled virtual environment.

A simulation-first approach significantly reduces the cost and time associated with physical prototyping while enhancing safety by allowing robots to be tested in scenarios that might otherwise be impractical or hazardous in real life.

With a new reference workflow, developers can accelerate the generation of synthetic 3D datasets with generative AI using OpenUSD NIM microservices. This integration streamlines the pipeline from scene creation to data augmentation, enabling faster and more accurate training of perception AI models.

Synthetic data can help address the challenge of limited, restricted or unavailable data needed to train various types of AI models, especially in computer vision. Developing action recognition models is a common use case that can benefit from synthetic data generation.

To learn how to create a human action recognition video dataset with Isaac Sim, check out the technical blog on Scaling Action Recognition Models With Synthetic Data. 3D simulations offer developers precise control over image generation, eliminating hallucinations.

Robotic Simulation for Humanoids Humanoid robots are the next wave of embodied AI, but they present a challenge at the intersection of mechatronics, control theory and AI. Simulation is crucial to solving this challenge by providing a safe, cost-effective and versatile platform for training and testing humanoids.

With NVIDIA Isaac Lab, an open-source unified framework for robot learning built on top of Isaac Sim, developers can train humanoid robot policies at scale via simulations. Leading commercial robot makers are adopting Isaac Lab to handle increasingly complex movements and interactions.

NVIDIA Project GR00T, an active research initiative to enable the humanoid robot ecosystem of builders, is pioneering workflows such as GR00T-Gen to generate robot tasks and simulation-ready environments in OpenUSD. These can be used for training generalist robots to perform manipulation, locomotion and navigation.

Recently published research from Project GR00T also shows how advanced simulation can be used to train interactive humanoids. Using Isaac Sim, the researchers developed a single unified controller for physically simulated humanoids called MaskedMimic. The system is capable of generating a wide range of motions across diverse terrains from intuitive user-defined intents.

Physics-Based Digital Twins Simplify AI Training Partners across industries are using Isaac Sim, Isaac Lab, Omniverse, and OpenUSD to design, simulate and deploy smarter, more capable autonomous machines:

Agility uses Isaac Lab to create simulations that let simulated robot behaviors transfer directly to the robot, making it more intelligent, agile and robust when deployed in the real world.

Cobot uses Isaac Sim with its AI-powered cobot, Proxie, to optimize logistics in warehouses, hospitals, manufacturing sites and more.

Cohesive Robotics has integrated Isaac Sim into its software framework called Argus OS for developing and deploying robotic workcells used in high-mix manufacturing environments.

Field AI, a builder of robot foundation models, uses Isaac Sim and Isaac Lab to evaluate the performance of its models in complex, unstructured environments across industries such as construction,
LINK: https://blogs.nvidia.com/blog/openusd-sdg-advance-robot-learning/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

05/07/2026

Fox Sports AR/XR Studio Graphics Team Shows Power of Next-Generation Operations

We've gone to another level of integrating the graphics into the set architecture -Zac Fields, Fox Sports, SVP of Graphic Technology and Integration...

05/07/2026

Free Behringer Synth Editor

Browser-based and desktop apps available Digital Sunset Studios have been busy working on an interesting project: a free software tool that provides compute...

04/07/2026

Inside OneSoccers FIFA World Cup Ops: Canadas Soccer Network Goes Behind the Scenes for CanMNTs Historic Moment

When Canada qualified for the FIFA World Cup 2026, OneSoccer knew this wasn'...

04/07/2026

Rhodes to launch the MK8/80AE

Celebrates company's 80th anniversary Rhodes have recently revealed that they will be producing a limited run of electric pianos in celebration of their...

04/07/2026

Blackmagic Design Cameras Empower Youth Broadcasting Program

Blackmagic Design Cameras Empower Youth Broadcasting Program Brie Clayton July 3, 2026 0 Comments Blackmagic Pocket Cinema Camera 6K Pro and Blackmagi...

04/07/2026

iZotope Joins Boris FX

iZotope Joins Boris FX Boris Yamnitsky July 3, 2026 0 Comments iZotope, the team behind RX and Ozone, is joining Boris FX. A letter from founder Boris...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

03/07/2026

VSL reveal Synchron Brass (muted)

New bundle & three Single Packs Continuing to expand their already sizeable orchestral collection, VSL's latest release introduces three new Single Pack...

03/07/2026

London Symphony Orchestra upgrade LSO St Luke's

Venue installs new ATC-equipped control room LSO St Luke's, a unique music venue that also serves as the home of the London Symphony Orchestra, have rec...

03/07/2026

Caden Pearson appointed new Commissioning Editor for NITV

Caden Pearson appointed new Commissioning Editor for NITV 3 July, 2026 Media releases National Indigenous Television (NITV) has strengthened its commitment...

03/07/2026

2023

1 February 2023 SHARE Facebook Twitter Linkedin Email Munich, Germany, 1st February 2023: Cinegy GmbH, the premier provider of software technology for digit...

03/07/2026

Broadcast Solutions Acquires BFE

Share Copy link Facebook X Linkedin Bluesky Email...

03/07/2026

AdClarity Expands CTV Offering to 20 Countries

Share Copy link Facebook X Linkedin Bluesky Email...

03/07/2026

Mountain West Conference Launches SVOD Service With New Revenue Model

Share Copy link Facebook X Linkedin Bluesky Email...

03/07/2026

Death Star

Death Star Andy Marken July 2, 2026 0 Comments Look, I can't get involved. I've got work to do. It's not that I like the Empire; I hate i...

03/07/2026

Berklee's New Visual Identity: Honoring Our History, Building for What's Next

Berklee's New Visual Identity: Honoring Our History, Building for What's...

03/07/2026

July 01, 2026

Scripps Research scientists awarded $2M to advance global disease surveillance Two Gates Foundation grants will expand wastewater surveillance and AI-driven dis...

03/07/2026

July 02, 2026

Joan Pulupa joins Scripps Research faculty to study the organization of DNA in brain cells and its links to neurodegeneration Using smell-sensing neurons and ad...

02/07/2026

SVG Students To Watch: Abby Finke, University of Dayton

Entering her senior year, this hometown girl is paving a career in live sports production gaining experience in replay and audio and as a TD In the live-sports...

02/07/2026

SVG GameDay, Ep. 22: Winnipeg Jets Kyle Balharry - Going to Work in the Great White North

In-venue and creative video staffers at the professional and collegiate level ha...

02/07/2026

BLAST Reports $133 Million in 2025 Revenue, Opens New York Headquarters

BLAST, a competitive entertainment company focused on esports, has announced more than $133 million in revenue for 2025, representing more than 40% year-over-ye...

02/07/2026

Riedel and SKAARHOJ Expand Collaboration with SimplyLive Integration

Riedel Communications has announced official SKAARHOJ panel support for SimplyLive production workflows, enabled through the SimplyLive 2.1 release. The integra...

02/07/2026

Czech Fire Rescue Service Deploys LiveU Video Transmission for Emergency Operations

The Fire Rescue Service of the Czech Republic has deployed LiveU video-over-bond...

02/07/2026

Gravity Media USA Appoints Brittney Boston as Head of Business Development

Gravity Media USA has announced the appointment of Brittney Boston as Head of Business Development, effective July 1, 2026. Based in Nashville, Tennessee, Bosto...

02/07/2026

TwelveLabs Raises $100 Million in Series B Funding

TwelveLabs, a video intelligence company, has announced $100 million in Series B funding co-led by NEA and NAVER Ventures, with participation from Amazon, Radic...

02/07/2026

Pro Padel League Announces Broadcast Partnership With USA Sports for 2026 Season

The Pro Padel League (PPL) has announced a broadcast partnership with USA Sports that will air five PPL championship matches on CNBC during the 2026 season, the...

02/07/2026

LiveLike Powers Eight FIFA World Cup 2026 Fan Engagement Activations Across Five Continents

LiveLike, a digital fan engagement platform, has announced eight confirmed FIFA ...

02/07/2026

InfoComm 2026: Cobalt Digitals blueCORE Wins Futures Best of Show Award

Cobalt Digital has received Future's Best of Show Award, presented by AV Technology at InfoComm 2026, for its blueCORE family of standalone signal processor...

02/07/2026

Synamedia Appoints Dr. Tzvi Gerstl as CEO

Synamedia has announced the appointment of Dr. Tzvi Gerstl as Chief Executive Officer. Paul Segre, who has served as CEO for the past six years, will transition...

02/07/2026

Esports World Cup 2026 Announces Expanded Sony Partnership for Paris Event

The Esports Foundation (EF) and Sony Group Corporation have announced an expanded collaboration for the Esports World Cup 2026 (EWC), taking place in Paris, Fra...

02/07/2026

Zee Entertainment Secures Exclusive Bundesliga Rights in India for Five Years

Zee Entertainment Enterprises Ltd. ( Z') has announced exclusive broadcast and digital rights for the Bundesliga in India for five years, beginning with the...

02/07/2026

All Hands on Deck: NBCU Comes Together to Produce Ultra-Complex Sail4th 250 Broadcast on July 4

An effort uniting News, Sports, Local, and Telemundo, the 50+-camera live produc...

02/07/2026

Release Rundown: What to Watch in July, From Gail Daughtry and the Celebrity Sex Pass to Murder 101

Zoey Deutch, John Slattery, Ken Marino, Miles Gutierrez-Riley, and Ben Wang appe...

02/07/2026

The Crow Hill Company introduce Brackish Pads

Stammering, stuttering, strangulated tones The Crow Hill Company's latest creation promises to be the most original sound set they've produced to d...

02/07/2026

Steinberg SpectraLayers 13 now available

A new era in unmixing and spectral editing The latest version of Steinberg's spectral audio-editing software has just arrived, building on the strength...

02/07/2026

Sine Machine from Melatonin

Aims to simplify additive synthesis Sine Machine is the debut launch from Melatonin, a Vienna-based developer who have spent the past six years creating wha...

02/07/2026

iZotope acquired by Boris FX

Products to remain fully active & supported Following the news of Native Instruments joining the inMusic brand line-up, Academy and Emmy Award-winning visua...

02/07/2026

GearExpo UK 2026

What you missed! Last weekend, Saturday 27 June 2026, saw the debut of Sound On Sounds new GearExpo UK event, the largest dedicated pro-audio event to take ...

02/07/2026

Imagine Communications Acquired by Lumine Group

Share Copy link Facebook X Linkedin Bluesky Email...

02/07/2026

Rise AV APAC Brings Mentoring Conversation to InfoComm As...

Following the successful launch of its inaugural APAC Mentoring Programme last month, the Rise AV APAC Regional Council will bring the conversation around mento...

02/07/2026

Blackmagic PYXIS 6K Used to Shoot Director Takahisa Zeze's Cry Out

Blackmagic PYXIS 6K Used to Shoot Director Takahisa Zeze's Cry Out Brie Clayton July 2, 2026 0 Comments Highly mobile camera supports tense and de...

02/07/2026

Broadcast Solutions acquires BFE, expanding its lead in European broadcast, media and communications infrastructure

Broadcast Solutions acquires BFE, expanding its lead in European broadcast, medi...

02/07/2026

Berklee Alum and Faculty Perform at Boston Public Library's 250th Anniversary Celebration of the Declaration of Independence

Berklee Alum and Faculty Perform at Boston Public Library's 250th Anniversar...

02/07/2026

Broadcast Solutions acquires BFE

Broadcast Solutions GmbH, a leading systems integrator and provider of innovative solutions for the broadcast media industry, is acquiring BFE Studio und Medien...

02/07/2026

LiveMode builds agile content ingest with Cinegy

Cinegy GmbH, the premier provider of software-defined television technology, has extended the ingest facility at leading Brazilian sports company LiveMode, work...

02/07/2026

Synamedia Appoints Dr Tzvi Gerstl CEO

Share Copy link Facebook X Linkedin Bluesky Email...