Sony Pixel Power calrec Sony

NVIDIA Advances Physical AI at CVPR With Largest Indoor Synthetic Dataset

17/06/2024

NVIDIA contributed the largest ever indoor synthetic dataset to the Computer Vision and Pattern Recognition (CVPR) conference's annual AI City Challenge - helping researchers and developers advance the development of solutions for smart cities and industrial automation.

The challenge, garnering over 700 teams from nearly 50 countries, tasks participants to develop AI models to enhance operational efficiency in physical settings, such as retail and warehouse environments, and intelligent traffic systems.

Teams tested their models on the datasets that were generated using NVIDIA Omniverse, a platform of application programming interfaces (APIs), software development kits (SDKs) and services that enable developers to build Universal Scene Description (OpenUSD)-based applications and workflows.

Creating and Simulating Digital Twins for Large Spaces In large indoor spaces like factories and warehouses, daily activities involve a steady stream of people, small vehicles and future autonomous robots. Developers need solutions that can observe and measure activities, optimize operational efficiency, and prioritize human safety in complex, large-scale settings.

Researchers are addressing that need with computer vision models that can perceive and understand the physical world. It can be used in applications like multi-camera tracking, in which a model tracks multiple entities within a given environment.

To ensure their accuracy, the models must be trained on large, ground-truth datasets for a variety of real-world scenarios. But collecting that data can be a challenging, time-consuming and costly process.

AI researchers are turning to physically based simulations - such as digital twins of the physical world - to enhance AI simulation and training. These virtual environments can help generate synthetic data used to train AI models. Simulation also provides a way to run a multitude of what-if scenarios in a safe environment while addressing privacy and AI bias issues.

Creating synthetic data is important for AI training because it offers a large, scalable, and expandable amount of data. Teams can generate a diverse set of training data by changing many parameters including lighting, object locations, textures and colors.

Building Synthetic Datasets for the AI City Challenge This year's AI City Challenge consists of five computer vision challenge tracks that span traffic management to worker safety.

NVIDIA contributed datasets for the first track, Multi-Camera Person Tracking, which saw the highest participation, with over 400 teams. The challenge used a benchmark and the largest synthetic dataset of its kind - comprising 212 hours of 1080p videos at 30 frames per second spanning 90 scenes across six virtual environments, including a warehouse, retail store and hospital.

Created in Omniverse, these scenes simulated nearly 1,000 cameras and featured around 2,500 digital human characters. It also provided a way for the researchers to generate data of the right size and fidelity to achieve the desired outcomes.

The benchmarks were created using Omniverse Replicator in NVIDIA Isaac Sim, a reference application that enables developers to design, simulate and train AI for robots, smart spaces or autonomous machines in physically based virtual environments built on NVIDIA Omniverse.

Omniverse Replicator, an SDK for building synthetic data generation pipelines, automated many manual tasks involved in generating quality synthetic data, including domain randomization, camera placement and calibration, character movement, and semantic labeling of data and ground-truth for benchmarking.

Ten institutions and organizations are collaborating with NVIDIA for the AI City Challenge:

Australian National University, Australia

Emirates Center for Mobility Research, UAE

Indian Institute of Technology Kanpur, India

Iowa State University, U.S.

Johns Hopkins University, U.S.

National Yung-Ming Chiao-Tung University, Taiwan

Santa Clara University, U.S.

The United Arab Emirates University, UAE

University at Albany - SUNY, U.S.

Woven by Toyota, Japan

Driving the Future of Generative Physical AI Researchers and companies around the world are developing infrastructure automation and robots powered by physical AI - which are models that can understand instructions and autonomously perform complex tasks in the real world.

Generative physical AI uses reinforcement learning in simulated environments, where it perceives the world using accurately simulated sensors, performs actions grounded by laws of physics, and receives feedback to reason about the next set of actions.

Developers can tap into developer SDKs and APIs, such as the NVIDIA Metropolis developer stack - which includes a multi-camera tracking reference workflow - to add enhanced perception capabilities for factories, warehouses and retail operations. And with the latest release of NVIDIA Isaac Sim, developers can supercharge robotics workflows by simulating and training AI-based robots in physically based virtual spaces before real-world deployment.

Researchers and developers are also combining high-fidelity, physics-based simulation with advanced AI to bridge the gap between simulated training and real-world application. This helps ensure that synthetic training environments closely mimic real-world conditions for more seamless robot deployment.

NVIDIA is taking the accuracy and scale of simulations further with the recently announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines.

This technology will allow autonomous systems, whether a factory, vehicle or robot, to gather essential data to effectively perceive, navigate and interact with the real world. Using these microservices, developers can run large-scale te
LINK: https://blogs.nvidia.com/blog/ai-city-challenge-omniverse-cvpr/...
See more stories from nvidia

Most recent headlines

20/11/2025

Cinegy wins the Swedish Hockey League

15 March 2012 SHARE Facebook Twitter Linkedin Email Cinegy and Vericom are pleased to announce the recent deployment of Cinegy Archive for production and as...

20/11/2025

2012

6 September 2012 SHARE Facebook Twitter Linkedin Email FIC Turkey supplies media content to Pay TV under Fox TV Channel brands in Turkey. At the start of s...

20/11/2025

VIVA Indonesia broadcasting the World Cup Brazil 2014 with Cinegy

25 June 2014 SHARE Facebook Twitter Linkedin Email Last month VIVA finalized its Cinegy installation in order to produce and broadcast the World Cup Brazil ...

20/11/2025

2014

1 October 2014 SHARE Facebook Twitter Linkedin Email With 5 locations spread out around Baden-W rttemberg, Schw bisch Media required a scalable end to end s...

20/11/2025

Strictly Come Dancing Goes with a Cinegy Workflow

27 April 2015 SHARE Facebook Twitter Linkedin Email Cinegy, which develops and produces media asset management products that are used in flagship production...

20/11/2025

Cinegy announces new Latin America partnership

12 August 2015 SHARE Facebook Twitter Linkedin Email Munich, Germany, 26 October 2015 - Cinegy, which develops and produces software technology for digital ...

20/11/2025

2015

5 November 2015 SHARE Facebook Twitter Linkedin Email Munich, Germany 3 November 2015 - Cinegy, which develops and produces software technology for digital ...

20/11/2025

Cinegy launches Cinegy Open' initiative

12 February 2016 SHARE Facebook Twitter Linkedin Email Munich, Germany 9 February 2016 - Cinegy, which develops and produces software technology for digital...

20/11/2025

2016

22 June 2016 SHARE Facebook Twitter Linkedin Email Munich, Germany 21 June 2016 - Cinegy today announced that ABS BROADCAST, one of the largest independentl...

20/11/2025

Cinegy News - nominee of NewBay Media Awards 2016

10 January 2017 SHARE Facebook Twitter Linkedin Email Find us on page 12 of the Program Guide 2016 Tags NewBay Media Awards , Program Guide 2016 , broadc...

20/11/2025

Freeview Thailand deploys Cinegy Air PRO Bundles for multi-channel playout

15 March 2017 SHARE Facebook Twitter Linkedin Email Munich, Germany 15 March 2017 - Cinegy today announced that Thailand's Next Step direct-to-home Free...

20/11/2025

2017

12 December 2017 SHARE Facebook Twitter Linkedin Email Munich, Germany 13 December 2017 - Cinegy today announced that it is making its Cinegy Air Cloud Pack...

20/11/2025

2018

25 June 2018 SHARE Facebook Twitter Linkedin Email Munich 25th June 2018 - Cinegy today announced that Canada-based Accessible Media Inc. (AMI) has implemen...

20/11/2025

2019

27 May 2019 SHARE Facebook Twitter Linkedin Email The best quality live video from the worst networks - Haivision Although DVB and the World Wide Web are...

20/11/2025

2020

24 November 2020 SHARE Facebook Twitter Linkedin Email ATV Expand and Increase their Playout and Editing Capabilities using Cinegy Software Munich, Germany...

20/11/2025

Cinegy further commits to Eastern European market with new appointment

1 September 2021 SHARE Facebook Twitter Linkedin Email Munich, Germany, 1st September 2021: Cinegy GmbH, the premier provider of software technology for di...

20/11/2025

2021

20 September 2021 SHARE Facebook Twitter Linkedin Email ElectronicVerve continues a strong history of supporting Cinegy customers in India Munich, Germany,...

20/11/2025

2022

1 November 2022 SHARE Facebook Twitter Linkedin Email Munich, Germany, 01 November 2022: Cinegy, a premier provider of software technology for digital video...

20/11/2025

Britt Broadcast implements Cinegy Air PRO at TBC (Tanzanian Broadcasting Corporation]

31 October 2023 SHARE Facebook Twitter Linkedin Email Britt Broadcast at TBC...

20/11/2025

Oklahoma Community Television Overcomes ATSC 3.0 Translator Challenge

CINCINNATI Oklahoma Community Television (OCT) has selected GatesAir and Triveni Digital as key technology partners in an ATSC 3.0 deployment that establishes a...

20/11/2025

FCC Launches Wide-Ranging Examination of Network Affiliate Relations

WASHINGTON The Federal Communications Commission has opened a wide-ranging inquiry into the relations between broadcast networks and their affiliates that could...

20/11/2025

IAB: Creator Economy Ad Spend Now Dwarfs Ad Spend for Total Media Industry

NEW YORK Ad spend in the creator economy has more than doubled since 2021 from $13.9B to $29.5B in 2024 and that amount is projected to reach $37 billion in 202...

20/11/2025

Graduate Spotlight: Krysta Mirsik DePuy

Graduate Spotlight: Krysta Mirsik DePuy The educator from Hackettstown, New Jersey, shares how it took her 11 years to find the right graduate program for her...

20/11/2025

Rise Awards 2025 Celebrate the Industry's Best

LONDON The winners of the Rise Awards 2025, which recognize women and companies whose achievements have stood out in the media technology industry, have been an...

20/11/2025

Lionsgate, Debmar-Mercury Launch MovieSphere Gold Diginet

SANTA MONICA & NEW YORK Lionsgate's Worldwide Television Distribution Group and Debmar-Mercury have launched MovieSphere Gold in more than 30 million homes....

20/11/2025

Clear-Com Unveils New 4-Channel HelixNet Beltpack

ALAMEDA, Calif. Clear-Com has introduced its four-channel HelixNet beltpack, a next-generation advancement of its widely used two-channel model, and anticipates...

20/11/2025

Kiswe Core Aims to Simplify Cloud-Based Streaming Distribution

NEW PROVIDENCE, N.J. Streaming technology and services provider Kiswe has launched Kiswe Core, a next-generation cloud-based platform designed to simplify the s...

20/11/2025

Dolby Partners Up With Bay Area Super Bowl Host Committee

MOUNTAIN VIEW, Calif. Dolby Laboratories has signed on as an official signature partner of the Bay Area Host Committee (BAHC) for the 2026 Super Bowl and the we...

20/11/2025

Telus TV+ Expands Distribution to Samsung and LG Smart TVs

STUTTGART, Germany 3 Screen Solutions (3SS) has announced that the Canadian multiservice operator Telus has launched its Telus TV+ entertainment platform on Sam...

20/11/2025

MLB Strikes Rights Deals With ESPN, NBCUniversal, Netflix

Major League Baseball has announced deals with ESPN, NBCUniversal and Netflix covering 2026-2028 that make ESPN the exclusive distributor for MLB.TV, put regula...

20/11/2025

Berklee Presents Holiday Extravaganza: Duke Ellingtons The Nutcracker Suite and GenNext All-Stars' Noel

Berklee Presents Holiday Extravaganza: Duke Ellingtons The Nutcracker Suite and ...

20/11/2025

Richard Osman, Kyla Cobbler and Vittorio Angeloni among the guests on this week's Late Late Show

It's the Late Late Toy Show BIG BIG Ticket Giveaway day! It's the Late ...

20/11/2025

Into the Omniverse: How Smart City AI Agents Transform Urban Operations

Editor's note: This post is part of Into the Omniverse, a series focused on how developers, 3D practitioners and enterprises can transform their workflows u...

20/11/2025

Guests revealed for The 2 Johnnies Late Night Lock In tonight

The How To Gael pocast gals, comedian Chris Kent and quizzer Darragh Ennis drop by for the fifth episode of The 2 Johnnies Late Night Lock In In the penultimat...

20/11/2025

Ultimate Cloud Gaming Is Everywhere With GeForce NOW

The NVIDIA Blackwell RTX upgrade is nearing the finish line, letting GeForce NOW Ultimate members across the globe experience true next-generation cloud gaming ...

20/11/2025

The Largest Digital Zoo: Biology Model Trained on NVIDIA GPUs Identifies Over a Million Species

Tanya Berger-Wolf's first computational biology project started as a bet wit...

19/11/2025

Sphere Sound Kicks Up Its Heels at Radio City Music Hall

Sphere Sound Kicks Up Its Heels at Radio City Music HallUltra-immersive system from the singular Las Vegas venue is also ready for sportsBy Dan Daley, Audio Edi...

19/11/2025

The Future of Stadium and Arena Video Control Rooms: Tech Leaders Talk 4K, IP, and More

The Future of Stadium and Arena Video Control Rooms: Tech Leaders Talk 4K, IP, a...

19/11/2025

One Year at Intuit Dome: Walking Through the Stunning Arena's Advanced Video Production Layout with the LA Clippers

One Year at Intuit Dome: Walking Through the Stunning Arena's Advanced Video...

19/11/2025

SVG Sit-Down: IMG's Francois Westcombe Goes Inside IMG's Annual Digital Trends Report

SVG Sit-Down: IMG's Francois Westcombe Goes Inside IMG's Annual Digital ...

19/11/2025

Inside Brighton & Hove Albion's Fan-First Media Machine

Inside Brighton & Hove Albion's fan-first media machine By George Bevir Tuesday, November 18, 2025 - 10:10 Print This Story Brighton & Hove Albion'...

19/11/2025

Rock Chalk Rebuild, Part 1: Kansas Athletics Brings New Production Power to Rebuilt David Booth Kansas Memorial Stadium

Rock Chalk Rebuild, Part 1: Kansas Athletics Brings New Production Power to Rebu...

19/11/2025

Rock Chalk Rebuild, Part 2: A Pro's Guide to Kansas Athletics' New Video-Production Infrastructure

Rock Chalk Rebuild, Part 2: A Pro's Guide to Kansas Athletics' New Video...

19/11/2025

MLB Media Rights Shakeup: NBC Inks Three-Year Deal for Sunday Night Baseball, Peacock Exclusive Game, and More

MLB Media Rights Shakeup: NBC Inks Three-Year Deal for Sunday Night Baseball, Pe...

19/11/2025

Sabar Bonda (Cactus Pears) Takes a Tender View on Queer Relationships

Rohan Parashuram Kanawade attends the premiere of Sabar Bonda (Cactus Pears) at the 2025 Sundance Film Festival at Egyptian Theatre on January 26, 2025, in Pa...

19/11/2025

Sundance Institute-Supported Seeds, Apocalypse in the Tropics, Selena y Los Dinos, and More Earn IDA Documentary Awards Nominations

Awards season is heating up, and the International Documentary Association just ...

19/11/2025

L3Harris Breaks Ground on Arkansas Advanced Propulsion Facilities

Aerojet Rocketdyne President Ken Bedingfield and Arkansas Gov. Sarah Huckabee Sanders were joined by state and local officials to celebrate the groundbreaking o...