Sony Pixel Power calrec Sony

Into the Omniverse: How OpenUSD-Based Simulation and Synthetic Data Generation Advance Robot Learning

11/12/2024

Editor's note: This post is part of Into the Omniverse, a series focused on how developers, 3D practitioners, and enterprises can transform their workflows using the latest advances in OpenUSD and NVIDIA Omniverse.

Scalable simulation technologies are driving the future of autonomous robotics by reducing development time and costs.

Universal Scene Description (OpenUSD) provides a scalable and interoperable data framework for developing virtual worlds where robots can learn how to be robots. With SimReady OpenUSD-based simulations, developers can create limitless scenarios based on the physical world.

And NVIDIA Isaac Sim is advancing perception AI-based robotics simulation. Isaac Sim is a reference application built on the NVIDIA Omniverse platform for developers to simulate and test AI-driven robots in physically based virtual environments.

At AWS re:Invent, NVIDIA announced that Isaac Sim is now available on Amazon EC2 G6e instances powered by NVIDIA L40S GPUs. These powerful instances enhance the performance and accessibility of Isaac Sim, making high-quality robotics simulations more scalable and efficient.

These advancements in Isaac Sim mark a significant leap for robotics development. By enabling realistic testing and AI model training in virtual environments, companies can reduce time to deployment and improve robot performance across a variety of use cases.

Advancing Robotics Simulation With Synthetic Data Generation Robotics companies like Cobot, Field AI and Vention are using Isaac Sim to simulate and validate robot performance while others, such as SoftServe and Tata Consultancy Services, use synthetic data to bootstrap AI models for diverse robotics applications.

The evolution of robot learning has been deeply intertwined with simulation technology. Early experiments in robotics relied heavily on labor-intensive, resource-heavy trials. Simulation is a crucial tool for the creation of physically accurate environments where robots can learn through trial and error, refine algorithms and even train AI models using synthetic data.

Physical AI describes AI models that can understand and interact with the physical world. It embodies the next wave of autonomous machines and robots, such as self-driving cars, industrial manipulators, mobile robots, humanoids and even robot-run infrastructure like factories and warehouses.

Robotics simulation, which forms the second computer in the three computer solution, is a cornerstone of physical AI development that lets engineers and researchers design, test and refine systems in a controlled virtual environment.

A simulation-first approach significantly reduces the cost and time associated with physical prototyping while enhancing safety by allowing robots to be tested in scenarios that might otherwise be impractical or hazardous in real life.

With a new reference workflow, developers can accelerate the generation of synthetic 3D datasets with generative AI using OpenUSD NIM microservices. This integration streamlines the pipeline from scene creation to data augmentation, enabling faster and more accurate training of perception AI models.

Synthetic data can help address the challenge of limited, restricted or unavailable data needed to train various types of AI models, especially in computer vision. Developing action recognition models is a common use case that can benefit from synthetic data generation.

To learn how to create a human action recognition video dataset with Isaac Sim, check out the technical blog on Scaling Action Recognition Models With Synthetic Data. 3D simulations offer developers precise control over image generation, eliminating hallucinations.

Robotic Simulation for Humanoids Humanoid robots are the next wave of embodied AI, but they present a challenge at the intersection of mechatronics, control theory and AI. Simulation is crucial to solving this challenge by providing a safe, cost-effective and versatile platform for training and testing humanoids.

With NVIDIA Isaac Lab, an open-source unified framework for robot learning built on top of Isaac Sim, developers can train humanoid robot policies at scale via simulations. Leading commercial robot makers are adopting Isaac Lab to handle increasingly complex movements and interactions.

NVIDIA Project GR00T, an active research initiative to enable the humanoid robot ecosystem of builders, is pioneering workflows such as GR00T-Gen to generate robot tasks and simulation-ready environments in OpenUSD. These can be used for training generalist robots to perform manipulation, locomotion and navigation.

Recently published research from Project GR00T also shows how advanced simulation can be used to train interactive humanoids. Using Isaac Sim, the researchers developed a single unified controller for physically simulated humanoids called MaskedMimic. The system is capable of generating a wide range of motions across diverse terrains from intuitive user-defined intents.

Physics-Based Digital Twins Simplify AI Training Partners across industries are using Isaac Sim, Isaac Lab, Omniverse, and OpenUSD to design, simulate and deploy smarter, more capable autonomous machines:

Agility uses Isaac Lab to create simulations that let simulated robot behaviors transfer directly to the robot, making it more intelligent, agile and robust when deployed in the real world.

Cobot uses Isaac Sim with its AI-powered cobot, Proxie, to optimize logistics in warehouses, hospitals, manufacturing sites and more.

Cohesive Robotics has integrated Isaac Sim into its software framework called Argus OS for developing and deploying robotic workcells used in high-mix manufacturing environments.

Field AI, a builder of robot foundation models, uses Isaac Sim and Isaac Lab to evaluate the performance of its models in complex, unstructured environments across industries such as construction,
LINK: https://blogs.nvidia.com/blog/openusd-sdg-advance-robot-learning/...
See more stories from nvidia

Most recent headlines

28/11/2025

Brides Asks for Compassion for Our Youths

Nadia Fall attends the 2025 Sundance Film Festival premiere of Brides at the Egyptian Theatre on January 24, 2025, in Park City, Utah. (Photo by Donyale West/...

28/11/2025

4 Reasons Why Keeping Your Spotify App Updated Matters and What You Might Be Missing

It's easy to ignore those little red update available badges. But when it ...

28/11/2025

FCC to Vote on LPTV Rules at Dec. Public Meeting

WASHINGTON Federal Communications Commission has released a tentative agenda for the December Open Commission Meeting scheduled for Thursday, December 18, 2025 ...

28/11/2025

Professional Fighters League Packs a Domestic, International MMA Punch (TV Sportsplay)

The Professional Fighters League is looking to super-serve fans of mixed martial...

28/11/2025

Fubo Launches Multiview Beta on Roku

Fubo has released in beta on select Roku devices a new feature that lets users display up to four simultaneous streams at once....

28/11/2025

WNBA Playoffs Continue: What's On This Weekend in TV Sports (Sept. 28-29)

The WNBA playoffs and Week 4 of the NFL regular season highlight the list of live sports events airing on television this weekend....

28/11/2025

Freeze Frame: B+C Hall of Fame 2024

The 32nd class of honorees to the B+C Hall of Fame took to the stage at New York's Ziegfeld Ballroom on September 26 for a gala induction event. Click below...

28/11/2025

Next Text: As DirecTV and Dish Try to Seize the Remains of the Day, Does It Even Matter?

We hold in our hands the very last Next Text for Next TV, the weekly back-and-fo...

28/11/2025

DirecTV Acquires Dish, Unifying Struggling Satellite Business

DirecTV said it made a deal with EchoStar to buy EchoStar's video businesses, including satellite-TV provider Dish TV and virtual MVPD Sling TV, for $1 plus...

28/11/2025

B+C Hall of Fame Announces Its Class of 2025

The Broadcasting+Cable Hall of Fame, the premier industry event paying tribute to the influencers, innovators and shining lights of broadcast, cable and streami...

28/11/2025

Sky Sports x Slawn drop limited-edition football jersey that unlocks a month of free content from the home of sport

Friday 28 November 2025 Sky Sports x Slawn drop limited-edition football jersey...

28/11/2025

Rohde & Schwarz shows resilience in a challenging environment, revenue exceeds three billion euros for the first time

Rohde & Schwarz shows resilience in a challenging environment, revenue exceeds t...

28/11/2025

Changing children's lives for good: Donations for the RT Toy Show Appeal 2025 open tonight

Unwrapped: The Toy Show Appeal - airing this Sunday on RT One and RT Player- s...

27/11/2025

Vizrt Launches Viz One 8.1 With AI-Powered Features

LONDON Vizrt has added several AI-driven advanced features offering improved speed, intelligence and accuracy in the newest version of its media asset managemen...

27/11/2025

Prime Video Debuts AI-Powered Video Recaps

Prime Video has launched AI-powered video season recaps in a beta version for select English-language Prime Original series in the U.S., a move Amazon is callin...

27/11/2025

Netflix's 'Raat Akeli Hai: The Bansal Murders' Marks a Grand World Premiere at IFFI Ahead of Its Global Release on 19th December

Back to All News Netflix's Raat Akeli Hai: The Bansal Murders Marks a Grand...

27/11/2025

Sky unveils first look image from high-stakes action thriller Prisoner, coming 2026

Tahar Rahim and Izuka Hoyle star in the gripping six-part Sky Original from Acad...

27/11/2025

Sky Arts Reveals the Nations Greatest Basslines and Queen Reign Supreme

Thursday 27 November 2025 Sky Arts Reveals the Nation's Greatest Basslines - and Queen Reign Supreme The UK's most iconic basslines have been revealed...

27/11/2025

Stranger Things 5': Prepare for One Last Adventure With Our Final Season Coverage Guide

Back to All News Stranger Things 5': Prepare for One Last Adventure With O...

27/11/2025

Elastic Compute for a Sustainable Media Industry

The media industry has a paradox at its core. It's an industry built on light, color and imagination, yet behind the scenes, it's powered by one of the ...

27/11/2025

Arqiva Achieves Five-Star GRESB Rating

Rating reflects rating progress across areas including policies, diversity & inclusion, health & safety and Net Zero leadership Winchester, UK, 27 November 202...

27/11/2025

Retail Media Audits Explained: What Networks Need to Know

What are the industry standards for Retail Media? Kathryn explains that certification is based on the IAB Europe Retail Media Measurement Standards and the IAB ...

27/11/2025

Katie Taylor, Rachael Blackmore and Arthur Gourounlian among the guests on this week's Late Late Show

World champion boxer and Irish sporting icon Katie Taylor will be in studio this...

27/11/2025

Tonight on RT Prime Time, serious child protection concerns emerge over online gaming platform, Roblox

Roblox, one of the world's most popular online gaming platforms for primary ...

27/11/2025

The Ultimate Black Friday Deal Is Here

Black Friday is leveling up. Get ready to score one of the biggest deals of the season - 50% off the first three months of a new GeForce NOW Ultimate membership...

26/11/2025

SVG Sit-Down: Prime Video EP Mike Muriano Previews Massive Black Friday Slate Featuring NFL, NBA, and Golf

SVG Sit-Down: Prime Video EP Mike Muriano Previews Massive Black Friday Slate Fe...

26/11/2025

Inside the Archives: Winter Is in the Air and in Our Festival Films

A cinematic snow sculpture at the 1995 Sundance Film Festival. Photo by Randall Michelson...

26/11/2025

10 Book Podcasts You Can't Miss

Book podcasts are booming. On Spotify, you'll find everything from celebrity book clubs to deep dives with bestselling authors. And in markets where audiobo...

26/11/2025

JioStar and Nielsen Unveil Breakthrough Cross-Screen MeasurementStudy, Redefining Advertising Effectiveness in Live Sports

Mumbai, November 24, 2025: In a first-of-its-kind initiative, JioStar, in collab...

26/11/2025

ITN Deploys IP-Based Production Control Room

LONDON Factual content producer ITN Productions has launched a new low-latency IP gallery for news bulletins....

26/11/2025

YouTube TV, TelevisaUnivision End Lengthy Blackout

MIAMI TelevisaUnivision said it struck a new multiyear distribution agreement with YouTube TV that includes distribution of TelevisaUnivision's U.S. network...

26/11/2025

OpenDrives Bridges the Gap Between IT and Creatives with...

OpenDrives, Inc., a leader in software-defined data storage and data services, today announced the launch of the Atlas Corporate Creative Solution. This new Atl...

26/11/2025

Disguise to Showcase Future of Event Visuals at LDI 2025

Disguise, the industry-leading company powering the world's biggest live performances, is partnering with pioneering LED wall manufacturer DVS to give atten...

26/11/2025

HighField AI Expands Global Channel Partner Network to Ac...

HighField AI, the pioneer in agentic and multimodal automation for broadcast and media production, today announced the expansion of its global channel partner n...

26/11/2025

Mono Streaming selects PlayBox Neo to manage English Prem...

As high-stakes Premier League fixtures approach and additional premium content launches, with MONO positioning themselves to dominate Thailand's sports stre...

26/11/2025

Bell Centre arena in Montreal elevates fan experience wit...

Hosting a wide variety of events from high-intensity NHL games to complex live music concerts and major entertainment productions, Montreal's 21,000 capacit...

26/11/2025

Vizrt launches AI-powered advances for speed and accuracy...

Vizrt, the leader in live production technology revolutionizing viewer engagement and experience, releases AI-driven advances focusing on speed, intelligence, a...

26/11/2025

ITN Launches Low-Latency IP Control Room Powered by Teche...

ITN Productions, an award-winning factual content producer, today launched a new low-latency IP gallery for news bulletins. Responsible for delivering a leading...

26/11/2025

Ikegami Maintains Initiative in Broadcast Systems Develop...

Ikegami reports ongoing advances throughout 2025 in developing and delivering coordinated television production solutions that maximize quality, versatility and...

26/11/2025

Fubo, NBCUniversal Trade Barbs in Carriage Dispute

Following the Nov. 21 blackout of NBCUniversal channels on Fubo, the two sides have traded barbs about their inability to reach a new carriage deal....

26/11/2025

Global Sports Rights Spending to Top $78 Billion in 2030

LONDON As TV sports rights become increasingly important for both broadcasters and streamers, Ampere Analysis predicts global investment in the genre will surpa...

26/11/2025

Vubiquity Earns AWS Media & Entertainment Competency Status

LOS ANGELES Vubiquity said it has achieved the Amazon Web Services (AWS) Media & Entertainment Competency as part of the AWS Partner Network (APN). This designa...

26/11/2025

Comcast Pays $1.5 Million to Settle FCC Data Breach Probe

WASHINGTON The Federal Communications Commission's Enforcement Bureau said it has entered into a consent decree with Comcast calling for the cable company t...

26/11/2025

Berklee Named to the Hollywood Reporters Top Music Schools List

Berklee Named to the Hollywood Reporters Top Music Schools List The publication highlights the college's screen scoring program, industry partnerships, and ...

26/11/2025

Animated Series Love Through a Prism' Casts New Light on Romance Between Aristocrat and Exchange Student in London

Back to All News Animated Series Love Through a Prism' Casts New Light on ...

26/11/2025

NALIP Unveils Fifth Cohort of Director Incubator

Back to All News NALIP Unveils Fifth Cohort of Director Incubator Social Impact 26 November 2025 United States Link copied to clipboard The National Assoc...

26/11/2025

YouView Achieves Greenly Gold Certification for Sustainability

YouView Achieves Greenly Gold Certification for SustainabilityNov 26, 2025 YouView is proud to announce a Gold Certification award from Greenly for our perform...