Sony Pixel Power calrec Sony

Into the Omniverse: OpenUSD Workflows Advance Physical AI for Robotics, Autonomous Vehicles

22/01/2025

Editor's note: This post is part of Into the Omniverse, a series focused on how developers, 3D practitioners and enterprises can transform their workflows using the latest advances in Universal Scene Description (OpenUSD) and NVIDIA Omniverse.

The next frontier of AI is physical AI. Physical AI models can understand instructions and perceive, interact and perform complex actions in the real world to power autonomous machines like robots and self-driving cars.

Similar to how large language models can process and generate text, physical AI models can understand the world and generate actions. To do this, these models must be trained in simulation environments to comprehend physical dynamics, like gravity, friction or inertia - and understand geometric and spatial relationships, as well as the principles of cause and effect.

Global leaders in software development and professional services are using NVIDIA Omniverse, powered by OpenUSD, to build new products and services that will accelerate the development of AI and controllable simulations to enable the creation of true-to-reality virtual worlds, known as digital twins, that can be used to train physical AI with unprecedented accuracy and detail.

Generate Exponentially More Synthetic Data With Omniverse and NVIDIA Cosmos At CES, NVIDIA announced generative AI models and blueprints that expand Omniverse integration further into physical AI applications such as robotics, autonomous vehicles and vision AI.

Among these announcements was NVIDIA Cosmos, a platform of state-of-the-art generative world foundation models, advanced tokenizers, guardrails and an accelerated video processing pipeline - all designed to accelerate physical AI development.

Developing physical AI models is a costly, resource- and time-intensive process that requires vast amounts of real-world data and testing. Cosmos' world foundation models (WFM), which predict future world states as videos based on multimodal inputs, provide an easy way for developers to generate massive amounts of photoreal, physics-based synthetic data to train and evaluate AI for robotics, autonomous vehicles and machines. Developers can also fine-tune Cosmos WFMs to build downstream world models or improve quality and efficiency for specific physical AI use cases.

When paired with Omniverse, Cosmos creates a powerful synthetic data multiplication engine. Developers can use Omniverse to create 3D scenarios, then feed the outputs into Cosmos to generate controlled videos and variations. This can drastically accelerate the development of physical AI systems such as autonomous vehicles and robots by rapidly generating exponentially more training data covering a variety of environments and interactions.

OpenUSD ensures the data in these scenarios is seamlessly integrated and consistently represented, enhancing the realism and effectiveness of the simulations.

Leading robotics and automotive companies, including 1X, Agile Robots, Agility Robotics, Figure AI, Foretellix, Fourier, Galbot, Hillbot, IntBot, Neura Robotics, Skild AI, Virtual Incision, Waabi and XPENG, along with ridesharing giant Uber, are among the first to adopt Cosmos.

Learn more about how world foundation models will advance physical AI by listening to the NVIDIA AI Podcast episode with Ming-Yu Liu, vice president of research at NVIDIA.

See Cosmos in Action for Physical AI Use Cases Cosmos WFMs are revolutionizing industries by providing a unified framework for developing, training and deploying large-scale AI models across various applications. Enterprises in the automotive, industrial and robotics sectors can harness the power of generative physical AI and simulation to accelerate innovation and operational efficiency.

Humanoid robots: The NVIDIA Isaac GR00T Blueprint for synthetic motion generation helps developers generate massive synthetic motion datasets to train humanoid robots using imitation learning. With GR00T workflows, users can capture human actions and use Cosmos to exponentially increase the size and variety of the dataset, making it more robust for training physical AI systems.

Autonomous vehicles: Autonomous vehicle (AV) simulation powered by Omniverse Sensor RTX application programming interfaces lets AV developers replay driving data, generate new ground-truth data and perform closed-loop testing to accelerate their pipelines. With Cosmos, developers can generate synthetic driving scenarios to amplify training data by orders of magnitude, accelerating physical AI model development for autonomous vehicles. Global ridesharing giant Uber is partnering with NVIDIA to accelerate autonomous mobility. Rich driving datasets from Uber, combined with Cosmos and NVIDIA DGX Cloud, can help AV partners build stronger AI models more efficiently.

Industrial settings: Mega is an Omniverse Blueprint for developing, testing and optimizing physical AI and robot fleets at scale in a USD-based digital twin before deployment in factories and warehouses. The blueprint uses Omniverse Cloud Sensor RTX APIs to simultaneously render multisensor data from any type of intelligent machine, enabling high-fidelity sensor simulation at scale. Cosmos can enhance Mega by generating synthetic edge case scenarios to amplify training data, significantly improving the robustness and efficiency of training robots in simulation. KION Group, a supply chain solutions company, is among the first to adopt Mega to drive warehouse automation in retail, consumer packaged goods, parcel services and more.

Get Plugged Into the World of OpenUSD For more on Cosmos, watch the replay of NVIDIA CEO Jensen Huang's CES keynote, and get started with Cosmos WFMs available now under an open model license on Hugging Face and the NVIDIA NGC catalog. Join the upcoming livestream on Wednesday, February 5 for a deep dive into Cosmos WFMs and physical AI work
LINK: https://blogs.nvidia.com/blog/openusd-advances-physical-ai/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

29/04/2026

Reduce Workflow Complexity with nxtedition at MPTS 2026

nxtedition is bringing its range of consolidated production tools to MPTS 2026, with new developments spanning transcription, editing, graphics and AI-assisted ...

29/04/2026

Telxius taps Synamedia to unlock seamless global multi-CD...

Quortex Switch to boost the streaming experience for Telxius customers, reaching millions of viewers worldwide Synamedia and Telxius, the leading global connec...

29/04/2026

freispace and Projective Announce Joint Showcase of Integ...

freispace, the leading ERP-as-a-Service platform for media and entertainment production, and Projective, a leading provider of post-production collaboration tec...

29/04/2026

DHD Celebrates 30th Anniversary at 2026 NAB Show

DHD reports strong interest in its broadcast audio product range, exhibited at the April 19th-22nd NAB Show in Las Vegas. The event attracted a claimed 58,000 a...

29/04/2026

Student Spotlight: Alan Catz

Student Spotlight: Alan Catz The Argentine film and game composer talks about working on League of Legends, receiving Berklee's BMI Award, and the lifelon...

29/04/2026

Jay Jennings Builds the Worlds You Hear on Screen

Jay Jennings Builds the Worlds You Hear on Screen The supervising sound designer behind A Minecraft Movie, The Meg, Letters from Iwo Jima, and dozens of other...

29/04/2026

Paramount Will be 49.5% Foreign Owned After WBD Merger

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

NAB Show 2026: AI, Vertical and BPS Dominate Broadcasters' Discussions

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

IAB Tech Lab Announces New OpenRTB Attributes

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

Listen to the birds sing! Experience mother nature in a seven-hour radio marathon

The Dawn Chorus airs Sunday 3 May from midnight to 7am on RT Radio 1 and RT ly...

29/04/2026

April 28, 2026

Jin-Quan Yu elected to the National Academy of Sciences Yu is recognized for his pioneering work in synthetic organic chemistry. April 28, 2026 LA JOLLA, CA S...

28/04/2026

Sound for the NFL Draft Show Balances Often Conflicting Goals

The audio team for the entertainment event must blend speech intelligibility with full-range music reproduction while considering the broadcast Last week's...

28/04/2026

Pac-12 Releases New Primary Mark Ahead of 2026-27 Season

The Pac-12 Conference has released an updated primary mark and logo as the starting point of the new league's brand identity. The mark was soft-launched acr...

28/04/2026

DP World Tour to Become First Professional Sports Organization to Use Amazon Leo Satellite Connectivity

The DP World Tour and Amazon Leo have signed an agreement making Amazon's lo...

28/04/2026

Pixellot and HELIOS Announce Automated Hockey Shift Video Integration

Pixellot and HELIOS have announced an integration that automatically converts full-game hockey video into individualized shift videos for each athlete, without ...

28/04/2026

Daktronics Installs New LED Video Display at Asheville Tourists Ballpark

Daktronics has partnered with the Asheville Tourists to manufacture and install a new LED video display. The installation was completed in late 2025 and is now ...

28/04/2026

Eutelsat and PCTV Renew Long-Term Video Distribution Partnership in Mexico

Eutelsat has announced the renewal of its partnership with PCTV, a content aggregation and distribution company in Mexico and part of Megacable Holdings, for co...

28/04/2026

Gary SouthShore RailCats Install New Daktronics LED Display at U.S. Steel Yard

Daktronics has partnered with the Gary SouthShore RailCats to install a new LED video display at U.S. Steel Yard, replacing the previous Daktronics display inst...

28/04/2026

Telos Alliance and College Radio Foundation Award Omnia.11 Processor to Wright State Universitys WWSU-FM

Telos Alliance and the College Radio Foundation have announced that WWSU-FM of W...

28/04/2026

Inside the Mix: A1s Florian Brown and James Deason on Golfs Biggest Broadcasts

Golf viewership is growing. The 2025 Ryder Cup drew five million viewers in the UK, a 45% increase over the 2023 event. The US Open was the most streamed golf e...

28/04/2026

The CW Network Acquires Exclusive Broadcast Rights to WWE NXT Premium Live Events

The CW Network and WWE, part of TKO Group Holdings (NYSE: TKO), have announced t...

28/04/2026

NAB 2026: AIMS Wins NAB Show 2026 Product of the Year Award for IPMX

The Alliance for IP Media Solutions (AIMS) has announced that the Internet Protocol Media Experience (IPMX) suite of standards and specifications has been named...

28/04/2026

NAB 2026 SportsTechBuzz in Review: A Look Back at the Big News From 150+ Companies in Vegas

The 2026 NAB Show is in the books and the show once again served up a cavalcade ...

28/04/2026

Gray Media, RAJ Sports Launch New Womens Sports RSN in Portland - Rose City SportsNet

Gray Media and RAJ Sports have announced Rose City SportsNet (RCSN), a new netwo...

28/04/2026

Spotify Reports First Quarter 2026 Earnings

Today, we announced our First Quarter 2026 earnings, starting the Year of Raising Ambition with strong momentum across the business and continued innovation acr...

28/04/2026

Spotify rapporterar resultat fr frsta kvartalet 2026

I dag presenterade vi v rt resultat f r det f rsta kvartalet 2026. Vi inleder ret med starkt momentum i hela verksamheten och fortsatt innovation p plattforme...

28/04/2026

Mojave introduce the MA-C vocal mic

New handheld promises studio performance for the stage Mojave have just introduced a new live-focused handheld vocal mic created by award-winning designer D...

28/04/2026

VSTOPIA announce Dynamic Split Module for Ableton

Max for Live device offers AI-powered stem separation Dynamic Split Module (DSM) is a new Max for Live device created by Ostin Solo, a developer and musican...

28/04/2026

Focusrite unveil the ISA C8X

First interface equipped with ISA preamps Focusrite have just announced the launch of a new high-end audio interface that features a pair of their legendary...

28/04/2026

Nielsen and Triton Digital Collaborate to Bring Greater Visibility to Podcast Audiences in Nielsen's Media Impact Tool

Triton Digital's Podcast Metrics Demos+ Data Integration Enables Comprehensi...

28/04/2026

Paramount Skydance Will be 49.5% Foreign Owned After WBD Merger

Share Copy link Facebook X Linkedin Bluesky Email...

28/04/2026

KEET PBS Deploys PMVG TechBundle Services To Modernize Operations

Share Copy link Facebook X Linkedin Bluesky Email...

28/04/2026

TAG Video Systems Lens Wins Industry Awards for Cutting T...

TAG Video Systems, the leading IP-native Realtime Media Platform, today announced that Lens, its visual service health interface for broadcast operations, recei...

28/04/2026

AIMS Wins NAB Show 2026 Product of the Year Award for IPM...

Open AV-over-IP Standard Recognized in IT Networking/Infrastructure and Security Category The Alliance for IP Media Solutions (AIMS) today announced that the ...

28/04/2026

VFX History: Slit Scan

VFX History: Slit Scan Graham Quince April 28, 2026 0 Comments How did 2001: A Space Odyssey, Star Wars, Doctor Who and Star Trek: The Next Generation...

28/04/2026

These DaVinci Resolve Effects Will Make You a More Creative Colorist

These DaVinci Resolve Effects Will Make You a More Creative Colorist Kasia Jarco April 28, 2026 0 Comments Creativity in color grading is not about ha...

28/04/2026

A Simple Introduction to Cavalry: Indexed Circle

A Simple Introduction to Cavalry: Indexed Circle Simon Ubsdell April 28, 2026 0 Comments In this new introductory tutorial for Cavalry we're going...

28/04/2026

Rise Upskill - Applications Now Open for Free Global Trai...

Rise, the award-winning advocacy group for gender diversity in the broadcast and media technology sector, is pleased to announce a new global training programme...

28/04/2026

Clear Com Announces New Roles for Brian Grahn and Ben Tur...

Clear-Com has appointed Brian Grahn as Market Outreach Manager of the Americas and Ben Turnwell as Business Development Manager for EMEA live, expanding their ...

28/04/2026

LiveU Steps into the Future at MPTS 2026 with the Introdu...

LiveU is inviting MPTS visitors to step into the companys new Q Era on Stand D32, at The Grand Hall, Olympia, London (May 13-14). The company will showcase its ...

28/04/2026

IBC launches 2026 Innovation Awards to spotlight real-wor...

IBC today announces the launch of the IBC2026 Innovation Awards, with nominations now open for projects, programmes and initiatives that exemplify breakthrough ...

28/04/2026

WNBA to Stream All Preseason Games for Free

Share Copy link Facebook X Linkedin Bluesky Email...

28/04/2026

Nexstar Media Charitable Foundation Sets 30 Days of Giving'

Share Copy link Facebook X Linkedin Bluesky Email...

28/04/2026

Sinclair's Chief Compliance Officer Jeff Lewis to Retire

Share Copy link Facebook X Linkedin Bluesky Email...