Sony Pixel Power calrec Sony

NVIDIA Advances Physical AI at CVPR With Largest Indoor Synthetic Dataset

17/06/2024

NVIDIA contributed the largest ever indoor synthetic dataset to the Computer Vision and Pattern Recognition (CVPR) conference's annual AI City Challenge - helping researchers and developers advance the development of solutions for smart cities and industrial automation.

The challenge, garnering over 700 teams from nearly 50 countries, tasks participants to develop AI models to enhance operational efficiency in physical settings, such as retail and warehouse environments, and intelligent traffic systems.

Teams tested their models on the datasets that were generated using NVIDIA Omniverse, a platform of application programming interfaces (APIs), software development kits (SDKs) and services that enable developers to build Universal Scene Description (OpenUSD)-based applications and workflows.

Creating and Simulating Digital Twins for Large Spaces In large indoor spaces like factories and warehouses, daily activities involve a steady stream of people, small vehicles and future autonomous robots. Developers need solutions that can observe and measure activities, optimize operational efficiency, and prioritize human safety in complex, large-scale settings.

Researchers are addressing that need with computer vision models that can perceive and understand the physical world. It can be used in applications like multi-camera tracking, in which a model tracks multiple entities within a given environment.

To ensure their accuracy, the models must be trained on large, ground-truth datasets for a variety of real-world scenarios. But collecting that data can be a challenging, time-consuming and costly process.

AI researchers are turning to physically based simulations - such as digital twins of the physical world - to enhance AI simulation and training. These virtual environments can help generate synthetic data used to train AI models. Simulation also provides a way to run a multitude of what-if scenarios in a safe environment while addressing privacy and AI bias issues.

Creating synthetic data is important for AI training because it offers a large, scalable, and expandable amount of data. Teams can generate a diverse set of training data by changing many parameters including lighting, object locations, textures and colors.

Building Synthetic Datasets for the AI City Challenge This year's AI City Challenge consists of five computer vision challenge tracks that span traffic management to worker safety.

NVIDIA contributed datasets for the first track, Multi-Camera Person Tracking, which saw the highest participation, with over 400 teams. The challenge used a benchmark and the largest synthetic dataset of its kind - comprising 212 hours of 1080p videos at 30 frames per second spanning 90 scenes across six virtual environments, including a warehouse, retail store and hospital.

Created in Omniverse, these scenes simulated nearly 1,000 cameras and featured around 2,500 digital human characters. It also provided a way for the researchers to generate data of the right size and fidelity to achieve the desired outcomes.

The benchmarks were created using Omniverse Replicator in NVIDIA Isaac Sim, a reference application that enables developers to design, simulate and train AI for robots, smart spaces or autonomous machines in physically based virtual environments built on NVIDIA Omniverse.

Omniverse Replicator, an SDK for building synthetic data generation pipelines, automated many manual tasks involved in generating quality synthetic data, including domain randomization, camera placement and calibration, character movement, and semantic labeling of data and ground-truth for benchmarking.

Ten institutions and organizations are collaborating with NVIDIA for the AI City Challenge:

Australian National University, Australia

Emirates Center for Mobility Research, UAE

Indian Institute of Technology Kanpur, India

Iowa State University, U.S.

Johns Hopkins University, U.S.

National Yung-Ming Chiao-Tung University, Taiwan

Santa Clara University, U.S.

The United Arab Emirates University, UAE

University at Albany - SUNY, U.S.

Woven by Toyota, Japan

Driving the Future of Generative Physical AI Researchers and companies around the world are developing infrastructure automation and robots powered by physical AI - which are models that can understand instructions and autonomously perform complex tasks in the real world.

Generative physical AI uses reinforcement learning in simulated environments, where it perceives the world using accurately simulated sensors, performs actions grounded by laws of physics, and receives feedback to reason about the next set of actions.

Developers can tap into developer SDKs and APIs, such as the NVIDIA Metropolis developer stack - which includes a multi-camera tracking reference workflow - to add enhanced perception capabilities for factories, warehouses and retail operations. And with the latest release of NVIDIA Isaac Sim, developers can supercharge robotics workflows by simulating and training AI-based robots in physically based virtual spaces before real-world deployment.

Researchers and developers are also combining high-fidelity, physics-based simulation with advanced AI to bridge the gap between simulated training and real-world application. This helps ensure that synthetic training environments closely mimic real-world conditions for more seamless robot deployment.

NVIDIA is taking the accuracy and scale of simulations further with the recently announced NVIDIA Omniverse Cloud Sensor RTX, a set of microservices that enable physically accurate sensor simulation to accelerate the development of fully autonomous machines.

This technology will allow autonomous systems, whether a factory, vehicle or robot, to gather essential data to effectively perceive, navigate and interact with the real world. Using these microservices, developers can run large-scale te
LINK: https://blogs.nvidia.com/blog/ai-city-challenge-omniverse-cvpr/...
See more stories from nvidia

North America Stories

13/07/2024

VideoAmp Integrates Snap Inc. Ad Inventory

NEW YORK VideoAmp has announced a wide-ranging collaboration with Snap Inc. that will see the social technology platform integrate its first-party data, video, ...

13/07/2024

RAI Moves up Launch of DVB-T2 HEVC Broadcasts

The Italian public broadcaster RAI will start broadcasting on Multiplex B in the DVB-T2 HEVC standard on August 28, according to Marco Rossignoli, president of ...

13/07/2024

Mile-High AI: NVIDIA Research to Present Advancements in Simulation and Gen AI at SIGGRAPH

Mile-High AI: NVIDIA Research to Present Advancements in Simulation and Gen AI a...

13/07/2024

Future Today Adds Studio Content to Fawesome

Future Today said it has signed licencing deals with Sony Pictures Entertainment, Samuel Goldwyn Films and Gravitas Ventures that will add fresh content to its ...

13/07/2024

VideoAmp To Integrate Data, Inventory From Snap Into Ad-Planning Tools

VideoAmp said it made a deal enabling the measurement company to integrate Snap's first-party data, video and augmented-reality ad inventory into VideoAmp&#...

13/07/2024

ABC Renews Bachelor in Paradise' for Season 10

ABC has ordered a tenth season of Bachelor in Paradise. The season will air in 2025 on the network and stream the next day on Hulu....

13/07/2024

Bill Belichick Joins Inside the NFL' for Season 2 on The CW

Bill Belichick, the legendary head coach who led the New England Patriots to eight Super Bowl wins, is joining Inside the NFL, which starts its second season on...

13/07/2024

Allen Media Group's HBCU Go Set To Kick Off College Football Season

Allen Media Group's HBCU Go said it will televise 26 college football games played by Historically Black Colleges and Universities....

13/07/2024

Backed By Top Producers, Exploding Kittens' Card Game Adapted to Netflix Series

Exploding Kittens, an animated series based on the card game, debuts on Netflix ...

13/07/2024

ABC Owned Stations, Ava DuVernay Partner on Our America' Special

ABC Owned Television Stations premieres Our America: Hidden Stories with Ava DuVernay starting Sunday, July 14. The one-hour special sheds light on the interco...

13/07/2024

Shania Twain To Host People's Choice Country Awards' on NBC

Shania Twain will host the People's Choice Country Awards, which happens Thursday, September 26 on NBC and Peacock. The two-hour special will be live at the...

12/07/2024

Fox Sports to Create Virtual Reality Experience for Copa Amrica Final

CAMPBELL, Calif. Fox Sports and YBVR have announced that they will be creating an immersive virtual reality experience for the Copa Am rica Final on Meta Quest,...

12/07/2024

Gray TV Stations in Alaska to Air Seattle Kraken Games

TYSONS, Va. Tegna Inc., the Seattle Kraken and Gray Media have announced that Gray's KAUU in Anchorage and KYEX in Juneau will expand the Kraken broadcast n...

12/07/2024

Hallmark To Launch New Subscription Streaming Service

PASADENA, Calif. Hallmark has unveiled Hallmark+, a subscription streaming television service that will combine the company's brand of lifestyle experience ...

12/07/2024

What the Clearhaven Deal Means for Zixi's Future

In recent years, as TV companies and producers pushed to launch streaming services and deploy IP infrastructures, Zixi has been at the center of that tectonic t...

12/07/2024

Ateme To Focus on Maximizing Output, Minimizing Costs At IBC 2024

PARIS Ateme will feature technologies pushing the boundaries of transformation, monetization and experience at IBC 2024, Sept. 13-16, at the RAI Convention Cent...

12/07/2024

DNEG Group Completes Prime Focus Technologies Acquisition

The DNEG Group has completed acquisition of Prime Focus Technologies....

12/07/2024

Wimbledon and Hawk-Eye Innovations Extend Tech Agreement

LONDON The All England Lawn Tennis Club and Sony's Hawk-Eye Innovations have announced a new multi-year agreement that will see Hawk-Eye continue to provide...

12/07/2024

Vision2 Marketing to Rep FOR-A as Part of Acceleration in...

As part of a more expansive push into the U.S. market, FOR-A announces its partnership with Vision2 Marketing, a manufacturers' rep for pro audio, video, li...

12/07/2024

KTRK Optimizes Look of Live Streams and Broadcasts with S...

Houston broadcaster KTRK-TV has been on the scene for 70 years and is known for community outreach as well as investigative journalism. Journalist alumni includ...

12/07/2024

QuickLink Expands its US Presence With Two New Seasoned T...

QuickLink, the leading global provider of multi-camera video productions and remote contributions, announces a significant growth strategy with two high-level p...

12/07/2024

Imagine Communications Names Don Durand SVP Global Sales...

Imagine Communications has appointed international media industry executive Don Durand to the position of senior vice president of global sales for its ad tech ...

12/07/2024

BH Telecom selects Appear to deliver Moja TV service to s...

Appear, the global leader in live production technology, has been selected by BH Telecom to enhance the live channel processing software for its Moja TV service...

12/07/2024

Ikegami Announces New Additions to its Range of Broadcast...

Ikegami announces two new additions to its range of broadcast quality picture monitors: the 24-inch HLM-2460WA and 18.5 inch HLM-1860WR. These will make their E...

12/07/2024

Vizrt focuses on BroadcastAV market with PTZ3 PLUS camera...

Vizrt, the leader in real-time graphics and live production solutions for content creators, brings the BroadcastAV market sharply into focus with today's la...

12/07/2024

Czech TV Live Streams Prague Marathon with Resilient Deje...

Local production company Livecast counts on Dejero Smart Blending Technology for connectivity on the move Dejero recently provided Czech production company Li...

12/07/2024

LTN playout solution powers European Football Championshi...

LTN, the industry leader in transformative media technology and video transport solutions, announces the successful delivery of live and pre-recorded coverage t...

12/07/2024

Ateme to Showcase Cutting-Edge Solutions to Maximise Outp...

Ateme's solutions deliver unprecedented video experiences and content monetisation while optimizsng workflows through hybrid on-premises and cloud systems. ...

12/07/2024

Lightware Announces New Leadership in the Enterprise Team

Lightware Visual Engerineering, a leading manufacturer of connectivity solutions for the professional integrated systems market and a pioneer in signal manageme...

12/07/2024

Anton Bauer Salt-E Dog Wins Innovation Impact Award at Ro...

Anton/Bauer has announced that its pioneering sustainable power product, Salt-E Dog, has been awarded the prestigious Innovation Impact Award at the Royal Telev...

12/07/2024

Teatro TV Browser by iWedia Surpasses 40 million Devices...

iWedia, a leading innovator in software solutions for connected TV devices, is thrilled to announce a significant milestone for its flagship product, the Teatro...

12/07/2024

Tim Belcher joins Light Iron as new Managing Director

Light Iron, the post-production creative-services division of Panavision, has named industry veteran Tim Belcher the company's new Managing Director. Belche...

12/07/2024

COW Job Listing: Need a Drone Shot in Boardman, Oregon

COW Job Listing: Need a Drone Shot in Boardman, Oregon Brie Clayton July 11, 2024 0 Comments Need a Drone shot July 9, 2024COW Job Listing: Freelanc...

12/07/2024

American Underground Celebrates Pride & Juneteenth with Free, Public Learning Events

American Underground, Capitol Broadcasting's co-working community division, ...

12/07/2024

5 Session-Saving Tips for Studio One

By Craig Anderton I admit it: the following tips are based on personally embarrassing experiences. I like to work fast to keep the creative juices flowing, but...

12/07/2024

Fox Sports Heads to Berlin for Euro Final; Wraps Up Massive Month of International Soccer Coverage

Fox Sports Heads to Berlin for Euro Final; Wraps Up Massive Month of Internation...

12/07/2024

Euro 2024: UEFA Gets Set for Spain v England Final

Euro 2024: UEFA gets set for Spain v England final By George Bevir Friday, July 12, 2024 - 11:19 Print This Story When England take on Spain on Sunday in ...

12/07/2024

Euro 2024: Inside UEFA's International Broadcast Centre

Euro 2024: Inside UEFA's International Broadcast Centre By George Bevir Friday, July 12, 2024 - 08:00 Print This Story The IBC in Leipzig, Germany (Ph...

12/07/2024

NBC Sports Bay Area & California's Rusty West on Vendor Partnerships Bringing Broadcast Elements to Life

NBC Sports Bay Area & California's Rusty West on Vendor Partnerships Bringin...

12/07/2024

Long-time PSSI Broadcast/Mechanical Engineer Paul Blomstrom Dead at 44

Long-time PSSI Broadcast/Mechanical Engineer Paul Blomstrom Dead at 44 By Ken Kerschbaumer, Editorial Director Friday, July 12, 2024 - 12:55 pm Print This ...

12/07/2024

SVG Sit-Down: FOX Sports' Michael Bucklin Explores Successful Live Social Video During Euros, Copa Amrica

SVG Sit-Down: FOX Sports' Michael Bucklin Explores Successful Live Social Vi...

12/07/2024

Mile-High AI: NVIDIA Research to Present Advancements in Simulation and Gen AI at SIGGRAPH

NVIDIA is taking an array of advancements in rendering, simulation and generativ...

11/07/2024

Utah Scientific Bounces Back From Pandemic With Price Reduction

Utah Scientific Bounces Back From Pandemic With Price Reduction Buyers Could Save as Much as 30% on UDS Compact Series and UHD 12G Video Routing Systems SAL...

11/07/2024

Little Miss Sunshine, Boyhood, and More Celebrate Birthdays in July

The Hoover family embarks on an eventful road trip to California for a junior beauty pageant in Little Miss Sunshine....

11/07/2024

Grass Valley and Diversified sign multimillion-dollar strategic partnership agreement

MONTREAL, CANADA - July 11, 2024 - Grass Valley, a leader in Live Production Sol...

11/07/2024

Media Excel To Showcase New DIVA AI-Powered Encoding Technology at 2024IBC

AUSTIN, Texas Media Excel will showcase its new artificial intelligence-based DIVA (Dynamic Intelligent Video Adaptive) encoding technology at the 2024IBC, Sept...

11/07/2024

German Broadcaster ZDF Adopts Avid MediaCentral | Cloud UX

BURLINGTON, Mass. German public broadcaster ZDF is migrating its existing asset management systems onto Avid's MediaCentral | Cloud UX software platform, th...

11/07/2024

ESPN Taps DP Bryan TWZ Brousseau, hsc.tv to Light Wimbledon Studio

WIMBLEDON, U.K. Video production company hsc.tv worked with Director of Photography Bryan TWZ Brousseau to create a unique lighting and set design for ESPN'...

11/07/2024

Fourier Audio Embeds iLok USB Hardware in transform.engine for Advanced Security & Protection

Fourier Audio Embeds iLok USB Hardware in transform.engine for Advanced Security...

11/07/2024

Create with Maxon: Product Visualization from Brief to Render - Week 3

Create with Maxon: Product Visualization from Brief to Render - Week 3 Brie Clayton July 11, 2024 0 Comments Spicing it up with Materials (3/5) - Crea...