Sony Pixel Power calrec Sony

NVIDIA Advances Physical AI at CVPR With Largest Indoor Synthetic Dataset

17/06/2024

NVIDIA contributed the largest ever indoor synthetic dataset to the Computer Vision and Pattern Recognition (CVPR) conference's annual AI City Challenge - helping researchers and developers advance the development of solutions for smart cities and industrial automation.

The challenge, garnering over 700 teams from nearly 50 countries, tasks participants to develop AI models to enhance operational efficiency in physical settings, such as retail and warehouse environments, and intelligent traffic systems.

Teams tested their models on the datasets that were generated using NVIDIA Omniverse, a platform of application programming interfaces (APIs), software development kits (SDKs) and services that enable developers to build Universal Scene Description (OpenUSD)-based applications and workflows.

Creating and Simulating Digital Twins for Large Spaces In large indoor spaces like factories and warehouses, daily activities involve a steady stream of people, small vehicles and future autonomous robots. Developers need solutions that can observe and measure activities, optimize operational efficiency, and prioritize human safety in complex, large-scale settings.

Researchers are addressing that need with computer vision models that can perceive and understand the physical world. It can be used in applications like multi-camera tracking, in which a model tracks multiple entities within a given environment.

To ensure their accuracy, the models must be trained on large, ground-truth datasets for a variety of real-world scenarios. But collecting that data can be a challenging, time-consuming and costly process.

AI researchers are turning to physically based simulations - such as digital twins of the physical world - to enhance AI simulation and training. These virtual environments can help generate synthetic data used to train AI models. Simulation also provides a way to run a multitude of what-if scenarios in a safe environment while addressing privacy and AI bias issues.

Creating synthetic data is important for AI training because it offers a large, scalable, and expandable amount of data. Teams can generate a diverse set of training data by changing many parameters including lighting, object locations, textures and colors.

Building Synthetic Datasets for the AI City Challenge This year's AI City Challenge consists of five computer vision challenge tracks that span traffic management to worker safety.

NVIDIA contributed datasets for the first track, Multi-Camera Person Tracking, which saw the highest participation, with over 400 teams. The challenge used a benchmark and the largest synthetic dataset of its kind - comprising 212 hours of 1080p videos at 30 frames per second spanning 90 scenes across six virtual environments, including a warehouse, retail store and hospital.

Created in Omniverse, these scenes simulated nearly 1,000 cameras and featured around 2,500 digital human characters. It also provided a way for the researchers to generate data of the right size and fidelity to achieve the desired outcomes.

The benchmarks were created using Omniverse Replicator in NVIDIA Isaac Sim, a reference application that enables developers to design, simulate and train AI for robots, smart spaces or autonomous machines in physically based virtual environments built on NVIDIA Omniverse.

Omniverse Replicator, an SDK for building synthetic data generation pipelines, automated many manual tasks involved in generating quality synthetic data, including domain randomization, camera placement and calibration, character movement, and semantic labeling of data and ground-truth for benchmarking.

Ten institutions and organizations are collaborating with NVIDIA for the AI City Challenge:

Australian National University, Australia

Emirates Center for Mobility Research, UAE

Indian Institute of Technology Kanpur, India

Iowa State University, U.S.

Johns Hopkins University, U.S.

National Yung-Ming Chiao-Tung University, Taiwan

Santa Clara University, U.S.

The United Arab Emirates University, UAE

University at Albany - SUNY, U.S.

Woven by Toyota, Japan

Driving the Future of Generative Physical AI Researchers and companies around the world are developing infrastructure automation and robots powered by physical AI - which are models that can understand instructions and autonomously perform complex tasks in the real world.

Generative physical AI uses reinforcement learning in simulated environments, where it perceives the world using accurately simulated sensors, performs actions grounded by laws of physics, and receives feedback to reason about the next set of actions.

Developers can tap into developer SDKs and APIs, such as the NVIDIA Metropolis developer stack - which includes a multi-camera tracking reference workflow - to add enhanced perception capabilities for factories, warehouses and retail operations. And with the latest release of NVIDIA Isaac Sim, developers can supercharge robotics workflows by simulating and training AI-based robots in physically based virtual spaces before real-world deployment.

Researchers and developers are also combining high-fidelity, physics-based simulation with advanced AI to bridge the gap between simulated training and real-world application. This helps ensure that synthetic training environments closely mimic real-world conditions for more seamless robot deployment.

NVIDIA is taking the accuracy and scale of simulations further with the recently announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines.

This technology will allow autonomous systems, whether a factory, vehicle or robot, to gather essential data to effectively perceive, navigate and interact with the real world. Using these microservices, developers can run large-scale te
LINK: https://blogs.nvidia.com/blog/ai-city-challenge-omniverse-cvpr/...
See more stories from nvidia

North America Stories

22/09/2026

New TVU Pocket Offers Broadcast Features In Phone-Sized Unit

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Operative Launches Unified Platform for Advertising and Broadcast Management

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Element Launches New TiVo-Powered Smart TVs in the U.S.

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

NOAA Taps Tomorrow.io for Improved U.S. Weather Forecasts

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Gray Media Promotes Tim Maloney to GM and Director of Sales at WBKO

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Reports: Paramount Reaches Settlement with State AGs in Antitrust Suit

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

New Research Lays Out Streaming Services Best Defense Against Churn

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Sinclair Expands The National News Desk

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Paramount Settles Antitrust Lawsuit over WBD Merger

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

U.S. Broadband Industry Investment Hits $92.6 Billion

Share Copy link Facebook X Linkedin Bluesky Email...

21/09/2026

Manifold Taps Ryhaan Williams for North America COO Post

Manifold Technologies, developer of software-defined, FPGA-accelerated infrastructure for live production, has named Ryhaan Williams as Chief Commercial Officer...

21/09/2026

SNS Wins IAMT Best Journal Article Award

Studio Network Solutions (SNS) has won Best Journal Article at the 2026 IAMT Impact Awards for What Video Editors Need When Everything Is Changing Around Them,...

21/09/2026

MLW Announces Streaming Deal with Tubi for Weekly FUSION Series

Major League Wrestling (MLW) has announced a streaming distribution agreement with Tubi for its weekly series MLW FUSION. Beginning September 17, new episodes w...

21/09/2026

Adam Smith Named Chairman of Direct-to-Consumer at Disney Entertainment

Disney Entertainment has announced that Adam Smith has been named Chairman, Direct-to-Consumer, with responsibility for Disney and Hulu including product, engi...

21/09/2026

United Airlines and DISH to Offer Live Football on Starlink-Enabled Seatback Screens

United Airlines and DISH have announced an agreement to bring live professional ...

21/09/2026

SES to Provide Satellite Distribution for JioStar Channels Across India

SES and JioStar have announced a multi-year agreement under which SES will provide satellite distribution services for JioStar's television channel portfoli...

21/09/2026

Detroit Red Wings Announce Detroit SportsNet Broadcast Details for 2026-27 Season

The Detroit Red Wings have announced details for their inaugural season on Detro...

21/09/2026

Manifold Technologies Appoints Ryhaan Williams as CCO for North America

Manifold Technologies has appointed Ryhaan Williams as Chief Commercial Officer for North America, effective October 1, 2026. Based in Jacksonville, Florida, Wi...

21/09/2026

DAZN Pick Up L.A. Kings DTC Streaming Rights; ABTV Will Continue as Local TV Home

Angels Broadcast Television (ABTV), the local television home of the Los Angeles...

21/09/2026

Inside ESPNs US Open Effort: Embracing the Cloud and a New 20,000 Sq. Ft. Broadcast Center

The 2026 US Open Tennis Championships marked a year of change with respect to TV...

21/09/2026

Behind The Mic: Andrs Cantor Joins Netflixs FIFA Womens World Cup Commentary Team; Doc Rivers Joins NBC Sports NBA Coverage

Behind The Mic provides a roundup of recent news regarding on-air talent, includ...

21/09/2026

From Graphic Novel to 30-Minute Animated Film: What I Learned Building an AI-Assisted Production Pipeline

From Graphic Novel to 30-Minute Animated Film: What I Learned Building an AI-Ass...

21/09/2026

Liam Hopkins Builds Recording Studio with Blackmagic Design

Liam Hopkins Builds Recording Studio with Blackmagic Design Brie Clayton September 20, 2026 0 Comments Video complements audio production with music v...

21/09/2026

Why Deploying Physical AI at Scale Demands Safety at Every Layer

Physical AI is moving rapidly from research to large-scale deployment. By 2035, ABI Research projects an installed base of 49 million level 3-5 autonomous vehic...

21/09/2026

From Enablement to Execution, Egypt's AI Ecosystem Reaches Production Scale

Today, Egypt's AI builders gathered in the Grand Egyptian Museum for a reception that highlighted the nation's rapidly growing AI ecosystem - spanning A...

21/09/2026

NVIDIA Launches DSX Ready to Qualify Power and Cooling Products for AI Factories

Every AI factory needs power and cooling that fit its computing architecture. As AI infrastructure expands, power, cooling, water, site and grid constraints are...

21/09/2026

AI Security Is an Engineering Problem - How to Solve It at Every Layer of the Agent Stack

AI security is an engineering problem. That means defined security requirements,...

21/09/2026

5 Companies Using NVIDIA AI for Clean Energy

Clean energy isn't hard to come by, but the pace of large-scale adoption has historically been slow due to bottlenecks - including out-of-date infrastructur...

20/09/2026

COW Jobs: Video Editor, Motion Designer - Remote, PT, 6 Months

COW Jobs: Video Editor, Motion Designer - Remote, PT, 6 Months Brie Clayton September 20, 2026 0 Comments Video Editor + Motion Designer (Remote | PT ...

19/09/2026

Two High Profile Professional Sports Franchises Renew cre...

Two High-Profile Professional Sports Franchises Renew creative.space Contracts with DigitalGlue NFL team and Major League Soccer Club cite ease of use, secure ...

19/09/2026

Limited Edition Audio releases VoxSplit Pro for overlapping dialogue separation and cleanup on macOS

Limited Edition Audio releases VoxSplit Pro for overlapping dialogue separation ...

18/09/2026

IBC 2026 Tech Moves Beyond the Gloss and Into the Rack

IBC 2026 Tech Moves Beyond the Gloss and Into the Rack If you spent any time in the halls of the RAI this year, you noticed a clear shift in conversation. The c...

18/09/2026

SVG GameDay, Ep. 31: Seattle Mariners Tim Walsh - Creative Content in the Emerald City

In-venue and creative video staffers at the professional and collegiate level ha...

18/09/2026

Qvest Appoints Jens Miedek as SVP Sales, Media and Entertainment

Qvest has appointed Jens Miedek as Senior Vice President Sales, Media and Entertainment. Before joining Qvest, Miedek spent 16 years at Riedel Communications an...

18/09/2026

Fonn Group Acquires everviz, Relaunches as Teikna Graphics Platform

Fonn Group has acquired full ownership of everviz, a cloud-native platform for interactive charts, maps, and data-driven graphics, and will rebrand it as Teikna...

18/09/2026

OWC Acquires OpenDrives to Expand Enterprise Storage Portfolio

Other World Computing (OWC) has announced the acquisition of OpenDrives, a provider of software-defined video and rich media storage management solutions. The a...

18/09/2026

Kaltura and Verimatrix Partner on Content Security for OTT and Cloud TV

Kaltura and Verimatrix have announced a partnership integrating Verimatrix's Digital Rights Management and security capabilities into Kaltura's Cloud TV...

18/09/2026

Movicom's Robycam Aerial Cable System Deployed at World Nomad Games

The sixth edition of the World Nomad Games combined traditional sports, large-scale cultural performances, and some decidedly modern broadcast technology. Held ...

18/09/2026

DAZN and Super League Basketball Extend Partnership for 2026-27 Season

DAZN and Super League Basketball (SLB) have extended their partnership for the 2026-27 season. Every Championship, Cup, and Play-Off fixture will be available l...

18/09/2026

Nielsen Shifts Streaming Content Ratings to Daily Delivery Cadence

Nielsen has announced that its Streaming Content Ratings (SCR) product will move from weekly to daily data delivery, with clients now receiving viewership data ...

18/09/2026

ABC to Air LALIGAs ElClsico for First Time on October 25

ESPN will present LALIGA's ElCl sico on ABC for the first time, with FC Barcelona vs. Real Madrid airing Sunday, October 25 at 4 p.m. ET. A one-hour pre-mat...

18/09/2026

University of Pennsylvania and JMI Sports Extend Multimedia Rights Partnership Through 2035-36

The University of Pennsylvania's Division of Recreation and Intercollegiate ...

18/09/2026

Charlotte Hornets Name DAZN Exclusive Direct-to-Consumer Streaming Partner

The Charlotte Hornets have announced DAZN as their exclusive direct-to-consumer streaming partner for the 2026-27 NBA season. The service launches immediately, ...

18/09/2026

With New AM Radio Act, Live Sports Remain on the Air - and on the Road

Broadcast sports' original medium will let drivers keep on hearing the plays thanks to a new bill set to pass the Senate....

18/09/2026

Apple TV Expands iPhone Camera Deployment for Friday Night Baseball' Finale at Dodger Stadium

Fourteen iPhone 18 Pro devices - eight connected over a T-Mobile Private 5G Netw...

18/09/2026

Minnesota Lynx to Air Final Regular Season Local Broadcast on FOX 9+

The Minnesota Lynx have announced that their final local broadcast of the 2026 regular season will air free on FOX 9 following the shutdown of Victory . The Ly...

18/09/2026

Monumental+ Now Available on Prime Video

Monumental Sports Network (MNMT) has announced that its Monumental+ streaming service is now available as a subscription through Prime Video, giving Capitals an...

18/09/2026

WTGL Announces Inaugural Season Schedule on ESPN, ABC, and ION

WTGL, the women's team golf league created by TMRW Sports in partnership with the LPGA, has announced its inaugural season schedule. The season premieres Su...

18/09/2026

Ratings Roundup: NFL Week 1 Down 13% From Record 2025; ESPN Earns Its Best US Open Mens Final on Record

Ratings Roundup is a rundown of recent ratings news and is derived from press re...

18/09/2026

SNS Wins IAMT Award For Addressing What Video Editors Need In The AI Era

Studio Network Solutions (SNS) has won Best Journal Article at the 2026 IAMT Impact Awards for What Video Editors Need When Everything Is Changing Around Them, ...