Sony Pixel Power calrec Sony

NVIDIA Advances Physical AI at CVPR With Largest Indoor Synthetic Dataset

17/06/2024

NVIDIA contributed the largest ever indoor synthetic dataset to the Computer Vision and Pattern Recognition (CVPR) conference's annual AI City Challenge - helping researchers and developers advance the development of solutions for smart cities and industrial automation.

The challenge, garnering over 700 teams from nearly 50 countries, tasks participants to develop AI models to enhance operational efficiency in physical settings, such as retail and warehouse environments, and intelligent traffic systems.

Teams tested their models on the datasets that were generated using NVIDIA Omniverse, a platform of application programming interfaces (APIs), software development kits (SDKs) and services that enable developers to build Universal Scene Description (OpenUSD)-based applications and workflows.

Creating and Simulating Digital Twins for Large Spaces In large indoor spaces like factories and warehouses, daily activities involve a steady stream of people, small vehicles and future autonomous robots. Developers need solutions that can observe and measure activities, optimize operational efficiency, and prioritize human safety in complex, large-scale settings.

Researchers are addressing that need with computer vision models that can perceive and understand the physical world. It can be used in applications like multi-camera tracking, in which a model tracks multiple entities within a given environment.

To ensure their accuracy, the models must be trained on large, ground-truth datasets for a variety of real-world scenarios. But collecting that data can be a challenging, time-consuming and costly process.

AI researchers are turning to physically based simulations - such as digital twins of the physical world - to enhance AI simulation and training. These virtual environments can help generate synthetic data used to train AI models. Simulation also provides a way to run a multitude of what-if scenarios in a safe environment while addressing privacy and AI bias issues.

Creating synthetic data is important for AI training because it offers a large, scalable, and expandable amount of data. Teams can generate a diverse set of training data by changing many parameters including lighting, object locations, textures and colors.

Building Synthetic Datasets for the AI City Challenge This year's AI City Challenge consists of five computer vision challenge tracks that span traffic management to worker safety.

NVIDIA contributed datasets for the first track, Multi-Camera Person Tracking, which saw the highest participation, with over 400 teams. The challenge used a benchmark and the largest synthetic dataset of its kind - comprising 212 hours of 1080p videos at 30 frames per second spanning 90 scenes across six virtual environments, including a warehouse, retail store and hospital.

Created in Omniverse, these scenes simulated nearly 1,000 cameras and featured around 2,500 digital human characters. It also provided a way for the researchers to generate data of the right size and fidelity to achieve the desired outcomes.

The benchmarks were created using Omniverse Replicator in NVIDIA Isaac Sim, a reference application that enables developers to design, simulate and train AI for robots, smart spaces or autonomous machines in physically based virtual environments built on NVIDIA Omniverse.

Omniverse Replicator, an SDK for building synthetic data generation pipelines, automated many manual tasks involved in generating quality synthetic data, including domain randomization, camera placement and calibration, character movement, and semantic labeling of data and ground-truth for benchmarking.

Ten institutions and organizations are collaborating with NVIDIA for the AI City Challenge:

Australian National University, Australia

Emirates Center for Mobility Research, UAE

Indian Institute of Technology Kanpur, India

Iowa State University, U.S.

Johns Hopkins University, U.S.

National Yung-Ming Chiao-Tung University, Taiwan

Santa Clara University, U.S.

The United Arab Emirates University, UAE

University at Albany - SUNY, U.S.

Woven by Toyota, Japan

Driving the Future of Generative Physical AI Researchers and companies around the world are developing infrastructure automation and robots powered by physical AI - which are models that can understand instructions and autonomously perform complex tasks in the real world.

Generative physical AI uses reinforcement learning in simulated environments, where it perceives the world using accurately simulated sensors, performs actions grounded by laws of physics, and receives feedback to reason about the next set of actions.

Developers can tap into developer SDKs and APIs, such as the NVIDIA Metropolis developer stack - which includes a multi-camera tracking reference workflow - to add enhanced perception capabilities for factories, warehouses and retail operations. And with the latest release of NVIDIA Isaac Sim, developers can supercharge robotics workflows by simulating and training AI-based robots in physically based virtual spaces before real-world deployment.

Researchers and developers are also combining high-fidelity, physics-based simulation with advanced AI to bridge the gap between simulated training and real-world application. This helps ensure that synthetic training environments closely mimic real-world conditions for more seamless robot deployment.

NVIDIA is taking the accuracy and scale of simulations further with the recently announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines.

This technology will allow autonomous systems, whether a factory, vehicle or robot, to gather essential data to effectively perceive, navigate and interact with the real world. Using these microservices, developers can run large-scale te
LINK: https://blogs.nvidia.com/blog/ai-city-challenge-omniverse-cvpr/...
See more stories from nvidia

North America Stories

28/05/2026

Cool Concentric Text for Cavalry

Cool Concentric Text for Cavalry Simon Ubsdell May 27, 2026 0 Comments In this tutorial I'll show you how to make this popular text effect that is...

28/05/2026

May 21, 2026

Scripps Research's Skaggs Graduate School awards doctoral degrees to 34th graduating class May 21, 2026 Scripps Research's Skaggs Graduate School of ...

28/05/2026

May 27, 2026

Scripps Research chemist Jin-Quan Yu is named a Fellow of the Royal Society Yu is honored by the U.K.'s national academy of sciences for his work in synthet...

27/05/2026

Telestream Appoints Benjamin Desbois as CEO, Effective July 1

Telestream has announced that its Board of Directors has appointed Benjamin Desbois as Chief Executive Officer, effective July 1, 2026. Desbois, currently Teles...

27/05/2026

FOX MLB Leads Live-Event Categories; ESPN Is Tops Overall at 47th Annual Sports Emmy Awards

ESPN garnered 10 awards; NBC's Sunday Night Football received the Outstandin...

27/05/2026

Matrox Video Marks 50th Anniversary, Announces New Product Launch for June

Matrox Video is celebrating its 50th anniversary, marking five decades of operations from its headquarters in Montreal, Canada. Founded in 1976, the company has...

27/05/2026

MLB Announces Fan Engagement Initiatives for Americas 250th Anniversary

Major League Baseball has announced a series of initiatives tied to America's Semiquincentennial, including a national marketing campaign, Fourth of July br...

27/05/2026

Advanced Systems Group Hires Brian Gross as Account Manager for Audio Team

Advanced Systems Group (ASG) has announced that Brian Gross has joined the company as an Account Manager on its Audio team, based in the Burbank office. He will...

27/05/2026

Nielsen Research: Hispanic Fans, Asian Markets Drive Global Soccer Audience Ahead of World Cup 2026

Nielsen has released new research on soccer fandom ahead of the FIFA World Cup 2...

27/05/2026

ESL FACEIT Group Debuts First Ever Esports Vertical Stream Co-Developed With TikTok

ESL FACEIT Group (EFG) has unveiled a new partnership with TikTok to bring broad...

27/05/2026

Two Weeks Away: FIFA Outlines Production Plans for Highly Anticipated North American-Based World Cup

FIFA's Oscar Sanchez gives a deeper look to how this tournament will be cove...

27/05/2026

SVG Students To Watch: Maggie Lynn, Virginia Tech

The soon-to-be senior from Charlottesville is building her skills in replay, TD, and even creative content for HokieVision and its ACC Network productions In t...

27/05/2026

A Global Festival of Football: FOX Sports Illustrates Strategy to Bring Every FIFA Mens World Cup Match to the U.S. Audience

FOX Sports' Mike Davies breaks down the vision for this summer's showcas...

27/05/2026

Top-Tier Storytelling: Host Broadcast Services Works at Capturing the Atmosphere of the FIFA Mens World Cup

HBS's Paul King, FIFA's Oscar Sanchez preview how the masses at home wil...

27/05/2026

Matt Gangl & Pete Macheska on FOX MLBs Huge Night and an Unforgettable Postseason Run

FOX's MLB coverage dominated the night at the 47th Annual Sports Emmy Awards...

27/05/2026

FOXs Mike Davies and Team on Outstanding Technical Team Win for 2025 World Series

One of the most memorable Postseasons in baseball history would have had no memo...

27/05/2026

NBC Sports Rob Hyland Reflects on an Unforgettable Sunday Night Football Season

NBC's Sunday Night Football is among the most decorated and most watched programs in the history of television. It added to its jam-packed trophy case on Tu...

27/05/2026

Prime Videos John Ward and Mike Francis on Groundbreaking NBA on Prime Video Studio

The 2026 Sports Emmys marked a watershed moment for Prime Video Sports. After bu...

27/05/2026

Countdown to FIFA World Cup 2026: SVG Launches SportsTechLive Blog in Lead-up to Winter Games

With the Opening Match just over two weeks away, the entire sports-production-te...

27/05/2026

L3Harris Introduces the XL Converge 300P Portable Public Safety Radio

The XL Converge 300P radio system emerges with a groundbreaking feature set enhancing the mission-critical communications of public safety, federal and critica...

27/05/2026

Modernizing Public Safety Communications

Pairing Two47 MCX software with existing LTE networks means tailored system upgrades that can save time, money and lives....

27/05/2026

L3Harris Strengthens Global Solid Rocket Motor Supply Chain With New PAC-3 Propulsion Supplier

PAC-3 MSE offers improved range, speed, and maneuverability, making it an effect...

27/05/2026

Brightcove Adds New Features to Its AI Suite for Video Advertising

Share Copy link Facebook X Linkedin Bluesky Email...

27/05/2026

Star Trek VFX: Recreating John Knoll's Iconic Warp Stars without a Slitscan Camera

Star Trek VFX: Recreating John Knoll's Iconic Warp Stars without a Slitscan ...

27/05/2026

Adventure World Uses Blackmagic Replay for Marine Live

Adventure World Uses Blackmagic Replay for Marine Live Brie Clayton May 27, 2026 0 Comments Large screen displays and slow motion replays dynamically ...

27/05/2026

Berklee Alumna and Assistant Professor Olivia Prez-Collellmir to Premiere Original Work at Gaud Centennial in Barcelona

Berklee Alumna and Assistant Professor Olivia P rez-Collellmir to Premiere Origi...

27/05/2026

Gravity Media Expands Into Creative Services With New Agency

Share Copy link Facebook X Linkedin Bluesky Email...

27/05/2026

Tegna Names Patrick Paolini as CEO

Share Copy link Facebook X Linkedin Bluesky Email...

27/05/2026

Telestream Taps Company Vet Benjamin Desbois as CEO

Share Copy link Facebook X Linkedin Bluesky Email...

27/05/2026

HDR10+ Technologies to Launch Eclipsa Video Certification Program

Share Copy link Facebook X Linkedin Bluesky Email...

27/05/2026

ATSC to Gather in Washington Next Week for Annual Meeting

Share Copy link Facebook X Linkedin Bluesky Email...

27/05/2026

Telestream Appoints Benjamin Desbois as Chief Executive O...

Co-founder Dan Castles to transition to Executive Chair; internal promotion reinforces continuity and long-term growth Telestream, a global leader in media wor...

27/05/2026

Big Blue Marble Announces First End-to-End 5G Broadcast S...

Big Blue Marble today announced that its Nakolos platform is the first end-to-end 5G Broadcast solution worldwide to implement the complete feature set introduc...

27/05/2026

Lightware Continues Its ESG Commitment Through Girls Day...

Lightware recently hosted the Girls' Day event in April at its headquarters in Budapest, welcoming students for an interactive introduction to engineering a...

26/05/2026

Matrox Video Marks 50 Year Milestone

Share Copy link Facebook X Linkedin Bluesky Email...

26/05/2026

Roku Expands Premium Subscriptions With Fox One

Share Copy link Facebook X Linkedin Bluesky Email...

26/05/2026

Brian Gross Joins ASG's Audio Team as Account Manager

Share Copy link Facebook X Linkedin Bluesky Email...

26/05/2026

MPA Urges FCC Not to Reclassify vMVPDs

Share Copy link Facebook X Linkedin Bluesky Email...

26/05/2026

Cobalt Digital to Showcase End-to-End IPMX Ecosystem at I...

Cobalt Digital to Showcase End-to-End IPMX Ecosystem at InfoComm 2026, Making ST 2110 Easy for Pro AV blueCORE standalone processors headline solutions designe...

26/05/2026

Matrox Video Celebrates 50 Years of Innovation and Looks...

Matrox Video today celebrates its 50th anniversary, marking five decades of innovation, engineering excellence, and customer-focused evolution from its headquar...

26/05/2026

Cuez Helps ITV Studios Run Three Live Shows as One includ...

ITV Studios, the production arm of the UKs largest commercial broadcaster, has deployed the Cuez live production platform to unify the management of three back-...

26/05/2026

CETA Software releases Morpheus, an AI tool for real-time post-production project oversight

CETA Software releases Morpheus, an AI tool for real-time post-production projec...

26/05/2026

Ikegami Accelerates Motorsport Broadcast Innovation with...

In a move set to redefine motorsports coverage across the Asia Pacific region, Ikegami Electronics announces that Two Wheels Motor Racing Sdn Bhd (TWMR), a lead...

26/05/2026

Netflix Releases Official Trailer and Poster for 'The Root Of The Game,' A Documentary Series Premiering June 20

Back to All News Netflix Releases Official Trailer and Poster for The Root Of T...

26/05/2026

'MED,' Production Begins on Netflix's First Medical Drama From Brazil, Starring Clara Moneke

Back to All News MED, Production Begins on Netflixs First Medical Drama From Br...

26/05/2026

Netflix Announces Five New Brazilian Productions and Expands Its Presence at Rio2C 2026

Back to All News Netflix Announces Five New Brazilian Productions and Expands I...

26/05/2026

Netflix Celebrates the Release of the New Animated Series 'Due Spicci' With a Major Event at Circo Massimo in Rome

Back to All News Netflix Celebrates the Release of the New Animated Series Due ...

26/05/2026

Made In New Mexico: Building The Boroughs' From the Ground Up

Back to All News Made In New Mexico: Building The Boroughs' From the Ground Up A photo from The Boroughs.' (Courtesy of Netflix 2026) Entertainme...

26/05/2026

Broadcast Pix Introduces ONix Pro Control Panel

Smart Production Control. Total Confidence. Tyngsborough, MA, May 27, 2026 - Broadcast Pix today announced the ONix Pro Control Panel, its most advanced hard...