Sony Pixel Power calrec Sony

NVIDIA Advances Physical AI at CVPR With Largest Indoor Synthetic Dataset

17/06/2024

NVIDIA contributed the largest ever indoor synthetic dataset to the Computer Vision and Pattern Recognition (CVPR) conference's annual AI City Challenge - helping researchers and developers advance the development of solutions for smart cities and industrial automation.

The challenge, garnering over 700 teams from nearly 50 countries, tasks participants to develop AI models to enhance operational efficiency in physical settings, such as retail and warehouse environments, and intelligent traffic systems.

Teams tested their models on the datasets that were generated using NVIDIA Omniverse, a platform of application programming interfaces (APIs), software development kits (SDKs) and services that enable developers to build Universal Scene Description (OpenUSD)-based applications and workflows.

Creating and Simulating Digital Twins for Large Spaces In large indoor spaces like factories and warehouses, daily activities involve a steady stream of people, small vehicles and future autonomous robots. Developers need solutions that can observe and measure activities, optimize operational efficiency, and prioritize human safety in complex, large-scale settings.

Researchers are addressing that need with computer vision models that can perceive and understand the physical world. It can be used in applications like multi-camera tracking, in which a model tracks multiple entities within a given environment.

To ensure their accuracy, the models must be trained on large, ground-truth datasets for a variety of real-world scenarios. But collecting that data can be a challenging, time-consuming and costly process.

AI researchers are turning to physically based simulations - such as digital twins of the physical world - to enhance AI simulation and training. These virtual environments can help generate synthetic data used to train AI models. Simulation also provides a way to run a multitude of what-if scenarios in a safe environment while addressing privacy and AI bias issues.

Creating synthetic data is important for AI training because it offers a large, scalable, and expandable amount of data. Teams can generate a diverse set of training data by changing many parameters including lighting, object locations, textures and colors.

Building Synthetic Datasets for the AI City Challenge This year's AI City Challenge consists of five computer vision challenge tracks that span traffic management to worker safety.

NVIDIA contributed datasets for the first track, Multi-Camera Person Tracking, which saw the highest participation, with over 400 teams. The challenge used a benchmark and the largest synthetic dataset of its kind - comprising 212 hours of 1080p videos at 30 frames per second spanning 90 scenes across six virtual environments, including a warehouse, retail store and hospital.

Created in Omniverse, these scenes simulated nearly 1,000 cameras and featured around 2,500 digital human characters. It also provided a way for the researchers to generate data of the right size and fidelity to achieve the desired outcomes.

The benchmarks were created using Omniverse Replicator in NVIDIA Isaac Sim, a reference application that enables developers to design, simulate and train AI for robots, smart spaces or autonomous machines in physically based virtual environments built on NVIDIA Omniverse.

Omniverse Replicator, an SDK for building synthetic data generation pipelines, automated many manual tasks involved in generating quality synthetic data, including domain randomization, camera placement and calibration, character movement, and semantic labeling of data and ground-truth for benchmarking.

Ten institutions and organizations are collaborating with NVIDIA for the AI City Challenge:

Australian National University, Australia

Emirates Center for Mobility Research, UAE

Indian Institute of Technology Kanpur, India

Iowa State University, U.S.

Johns Hopkins University, U.S.

National Yung-Ming Chiao-Tung University, Taiwan

Santa Clara University, U.S.

The United Arab Emirates University, UAE

University at Albany - SUNY, U.S.

Woven by Toyota, Japan

Driving the Future of Generative Physical AI Researchers and companies around the world are developing infrastructure automation and robots powered by physical AI - which are models that can understand instructions and autonomously perform complex tasks in the real world.

Generative physical AI uses reinforcement learning in simulated environments, where it perceives the world using accurately simulated sensors, performs actions grounded by laws of physics, and receives feedback to reason about the next set of actions.

Developers can tap into developer SDKs and APIs, such as the NVIDIA Metropolis developer stack - which includes a multi-camera tracking reference workflow - to add enhanced perception capabilities for factories, warehouses and retail operations. And with the latest release of NVIDIA Isaac Sim, developers can supercharge robotics workflows by simulating and training AI-based robots in physically based virtual spaces before real-world deployment.

Researchers and developers are also combining high-fidelity, physics-based simulation with advanced AI to bridge the gap between simulated training and real-world application. This helps ensure that synthetic training environments closely mimic real-world conditions for more seamless robot deployment.

NVIDIA is taking the accuracy and scale of simulations further with the recently announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines.

This technology will allow autonomous systems, whether a factory, vehicle or robot, to gather essential data to effectively perceive, navigate and interact with the real world. Using these microservices, developers can run large-scale te
LINK: https://blogs.nvidia.com/blog/ai-city-challenge-omniverse-cvpr/...
See more stories from nvidia

North America Stories

24/09/2026

Standalone Streaming Services Continue to Decline in the U.S.

Share Copy link Facebook X Linkedin Bluesky Email...

24/09/2026

Hollyland Unveils LARK M3 Wireless Audio System

Share Copy link Facebook X Linkedin Bluesky Email...

24/09/2026

Adobe Completes Purchase Of Topaz Labs

Share Copy link Facebook X Linkedin Bluesky Email...

24/09/2026

FCC to Vote on Satellite Spectrum, Unlicensed Ultra-Wideband Devices

Share Copy link Facebook X Linkedin Bluesky Email...

24/09/2026

LYNX Technik Launches New Series 5000 Card

Share Copy link Facebook X Linkedin Bluesky Email...

24/09/2026

KRK V SERIES MONITORS HELP INDUSTRY VETERAN SANTIAGO URI...

Music Supervisor, Composer, and Mixing Engineer Santiago Uribe has built a career around producing music for film and television, contributing to more than 40 p...

24/09/2026

NABA Cyber University to Convene Broadcast Industry Leade...

The North American Broadcasters Association (NABA) will hold a full-day cybersecurity summit focused on defending against growing cyber threats facing the broad...

23/09/2026

University of Washington's New Sundodger Studios Leverages ST 2110 to Bring Broadcast, Videoboard, and Creative Under One Roof

Four control rooms support B1G+ streams, videoboard shows, creative content, and...

23/09/2026

Mountain West Conference Moves Replay Operations West, Cutting Late-Night Hours

By Andrew Cohen, SVG Coorespondant The Mountain West debuted its new Replay Operations Center at the start of this football season, bringing real-time collabor...

23/09/2026

SVG Students to Watch: Brian Short, Ball State University

The Taylorville, IL, native is pushing creative boundaries at Ball State, gaining reps across directing, technical directing, tape, and more than 100 ESPN+ broa...

23/09/2026

The Musical and the Power of Spiteful Thinking

Alexander Heller, Nevada Jose, Will Brill, and Giselle Bonilla pose with cast members at The Musical premiere during the 2026 Sundance Film Festival at Eccles...

23/09/2026

McDonald's takes top spot as New Zealand's fast-food tastes evolve, according to Nielsen

Nielsen CMI data reveals a changing fast-food landscape, with value, convenience...

23/09/2026

AFP, Dalet To Collaborate On News Video Provenance, Authenticity

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

Spectrum News NECN Launches Across New England

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

COW Jobs: Blender on Windows - Generalist - Remote

COW Jobs: Blender on Windows - Generalist - Remote Brie Clayton September 23, 2026 0 Comments Application Users - Blender on Windows - Generalist Se...

23/09/2026

Bitfocus wins IBC2026 Best of Show Award for latest Butto...

Version 1.8 recognised for new workflow tools, boosted control panel flexibility and ease of integration into IP networks...

23/09/2026

Hiltron Reports a Successful IBC2026

Hiltron Communications reports a successful 2026 International Broadcasting Convention, held at the RAI Amsterdam. Exhibiting on stand A37 in Hall 1, Hiltron ex...

23/09/2026

LiveU Showcases AI Driven Production Alongside Resilient...

Reinforcing its long-term commitment to the Middle East and North Africa (MENA) broadcast market, LiveU is presenting the Q Era at CABSAT 2026, on Stand H1-D35 ...

23/09/2026

AFP and Dalet Partner to Strengthen the Provenance and Au...

Agence France-Presse (AFP), a global news agency, and Dalet, a leading technology and service provider for media-rich organizations, today announced a partnersh...

23/09/2026

Berklee Connect Unlocks the Power of a Global Network

Berklee Connect Unlocks the Power of a Global Network Katie Vitolins BM '04 knows firsthand what Berklee's alumni community can make possible. Now she...

23/09/2026

Study: Renewals for Streaming Shows Continue to Decline

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

Study: More Than 4 in 10 U.S. Adults Own or Are Interested in Smart Glasses

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

Television Critics Association Revises Bylaws to Address AI

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

eX Celebrates Completion of Holland Mich. FTTP Network

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

Gray Media Renews Ohio Valley Conference Deal

Share Copy link Facebook X Linkedin Bluesky Email...

23/09/2026

QuickLink StudioPro Powers Next Generation Student Media...

Westminster Schools, a K through 12 school in Atlanta, Georgia with approximately 2,000 students, has transformed its student broadcast and digital storytelling...

23/09/2026

UK Pavilion Brings Leading British Technology to CABSAT a...

The 2026 edition of CABSAT and SATExpo may have a new home and dates in the diary, but it still retains a strong presence from UK companies in the GREAT Britain...

23/09/2026

Glensound marks 60 years with new audio developments at C...

Glensound will return to CABSAT with new developments in professional monitoring and Dante interfacing, alongside its established commentary, esports and parlia...

23/09/2026

Sakeena Fiza Helps NVIDIA Hardware Succeed at Scale

When Sakeena Fiza describes her work as a validation engineer at NVIDIA, she does so in terms more befitting a detective story than a world-class engineering la...

22/09/2026

ESPN, Precision Lasers Light Up NHL Player Media Tour in Las Vegas

The annual preseason content-gathering event brought 36 NHL players to Las Vegas, where ESPN used 2.5 kilowatts of laser power and a motion-control camera syste...

22/09/2026

Neumann Introduces KK 104 A and KK 105 A Wireless Capsule Heads

Neumann has introduced the KK 104 A (cardioid) and KK 105 A (supercardioid) wireless capsule heads, succeeding the KK 204 and KK 205. The new capsules are compa...

22/09/2026

NFL and TMRW Sports Announce Pro Flag Showcase During Super Bowl LXI Week

The NFL and TMRW Sports have announced a new annual Pro Flag Showcase to be held during Super Bowl LXI week in Los Angeles. The event will feature top flag foot...

22/09/2026

Washington State University Unveils New Daktronics Video Board and Audio System at Gesa Field

Washington State University has unveiled a new end zone video display and audio ...

22/09/2026

SVG Sit-Down: Disney's Dave Johnson on ESPN's New 4K HDR Distribution Pipeline and What It Means for Super Bowl LXI and Beyond

Disney's SVP of Live Operations and Engineering details the technology behin...

22/09/2026

Cosm Promotes Mazen Alawar to SVP of Marketing, Alexis Scalice to VP of Cosm Studios

Cosm has announced two senior promotions. Mazen Alawar has been promoted to SVP,...

22/09/2026

Mississippi State University Extends Camera Relationship with KOKUSAI DENKI

Mississippi State University's University Television Center has purchased four KOKUSAI DENKI SK-UHD7000-S2 cameras since 2024, with plans to add two more, a...

22/09/2026

Net Insight Appoints Christine Kjellvard as Interim CFO

Net Insight has announced that Christine Kjellvard will assume the role of interim Chief Financial Officer on November 1, 2026. Kjellvard has previously held in...

22/09/2026

Net Insight Launches Zyntai TimeNode Access PTP Grandmaster for Edge Sites

Net Insight has announced Zyntai TimeNode Access, a compact PTP Grandmaster designed for distributed edge sites. The product carries secure, precise time across...

22/09/2026

The Weight Explores the High Cost of Freedom in the Depression Era

Sam Hazeldine, Austin Amelio, Padraic McKinley, Julia Jones, and Ethan Hawke attend the premiere of The Weight at The Ray Theatre on January 26, 2026, in Park...

22/09/2026

Apogee Plugins Are Moving to Plugin Alliance

Apogee Plugins Are Moving to Plugin Alliance We're excited to share an important update about the future of Apogee plugins. Apogee plugins are moving to Pl...

22/09/2026

SipRadius honored with a Technology And Engineering Emmy...

Parkland, Florida, US, September 22, 2026: SipRadius, widely recognized for making content processing and connectivity secure and seamless, has been honored wit...

22/09/2026

VanityFilter 3.1.5 adds project files and adjustable protection around lips, brows and ears

VanityFilter 3.1.5 adds project files and adjustable protection around lips, bro...

22/09/2026

Chile's TV+ Modernizes Broadcast Operations with Blackmagic Design

Chile's TV Modernizes Broadcast Operations with Blackmagic Design Brie Clayton September 22, 2026 0 Comments URSA Broadcast G2 brings versatility...

22/09/2026

OWC Shares Sneak Peek at Amazon Prime Big Deal Days Pricing

OWC Shares Sneak Peek at Amazon Prime Big Deal Days Pricing Brie Clayton September 22, 2026 0 Comments Image courtesy of Deposit Photos Business and ...

22/09/2026

Berklee's First Homecoming Block Party Brings Alumni Home for Global Music Festival

Berklee's First Homecoming Block Party Brings Alumni Home for Global Music F...

22/09/2026

New TVU Pocket Offers Broadcast Features In Phone-Sized Unit

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Operative Launches Unified Platform for Advertising and Broadcast Management

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Element Launches New TiVo-Powered Smart TVs in the U.S.

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

NOAA Taps Tomorrow.io for Improved U.S. Weather Forecasts

Share Copy link Facebook X Linkedin Bluesky Email...

22/09/2026

Gray Media Promotes Tim Maloney to GM and Director of Sales at WBKO

Share Copy link Facebook X Linkedin Bluesky Email...