Sony Pixel Power calrec Sony

Now See This: NVIDIA Launches Blueprint for AI Agents That Can Analyze Video

06/01/2025

The next big moment in AI is in sight - literally.

Today, more than 1.5 billion enterprise level cameras deployed worldwide are generating roughly 7 trillion hours of video per year. Yet, only a fraction of it gets analyzed.

It's estimated that less than 1% of video from industrial cameras is watched live by humans, meaning critical operational incidents can go largely unnoticed.

This comes at a high cost. For example, manufacturers are losing trillions of dollars annually to poor product quality or defects that they could've spotted earlier, or even predicted, by using AI agents that can perceive, analyze and help humans take action.

Interactive AI agents with built-in visual perception capabilities can serve as always-on video analysts, helping factories run more efficiently, bolster worker safety, keep traffic running smoothly and even up an athlete's game.

To accelerate the creation of such agents, NVIDIA today announced early access to a new version of the NVIDIA AI Blueprint for video search and summarization. Built on top of the NVIDIA Metropolis platform - and now supercharged by NVIDIA Cosmos Nemotron vision language models (VLMs), NVIDIA Llama Nemotron large language models (LLMs) and NVIDIA NeMo Retriever - the blueprint provides developers with the tools to build and deploy AI agents that can analyze large quantities of video and image content.

The blueprint integrates the NVIDIA AI Enterprise software platform - which includes NVIDIA NIM microservices for VLMs, LLMs and advanced AI frameworks for retrieval-augmented generation - to enable batch video processing that's 30x faster than watching it in real time.

The blueprint contains several agentic AI features - such as chain-of-thought reasoning, task planning and tool calling - that can help developers streamline the creation of powerful and diverse visual agents to solve a range of problems.

AI agents with video analysis abilities can be combined with other agents with different skill sets to enable even more sophisticated agentic AI services. Enterprises have the flexibility to build and deploy their AI agents from the edge to the cloud.

How Video Analyst AI Agents Can Help Industrial Businesses AI agents with visual perception and analysis skills can be fine-tuned to help businesses with industrial operations by:

Increasing productivity and reducing waste: Agents can help ensure standard operating procedures are followed during complex industrial processes like product assembly. They can also be fine-tuned to carefully watch and understand nuanced actions, and the sequence in which they're implemented.

Boosting asset management efficiency through better space utilization: Agents can help optimize inventory storage in warehouses by performing 3D volume estimation and centralizing understanding across various camera streams.

Improving safety through auto-generation of incident reports and summaries: Agents can process huge volumes of video and summarize it into contextually informative reports of accidents. They can also help ensure personal protective equipment compliance in factories, improving worker safety in industrial settings.

Preventing accidents and production problems: AI agents can identify atypical activity to quickly mitigate operational and safety risks, whether in a warehouse, factory or airport, or at a traffic intersection or other municipal setting.

Learning from the past: Agents can search through operations video archives, find relevant information from the past and use it to solve problems or create new processes.

Video Analysts for Sports, Entertainment and More Another industry where video analysis AI agents stand to make a mark is sports - a $500 billion market worldwide, with hundreds of billions in projected growth over the next several years.

Coaches, teams and leagues - whether professional or amateur - rely on video analytics to evaluate and enhance player performance, prioritize safety and boost fan engagement through player analytics platforms and data visualization. With visually perceptive AI agents, athletes now have unprecedented access to deeper insights and opportunities for improvement.

During his CES opening keynote, NVIDIA founder and CEO Jensen Huang demonstrated an AI video analytics agent that assessed the fastball pitching skills of an amateur baseball player compared with a professional's. Using video captured from the ceremonial first pitch that Huang threw for the San Francisco Giants baseball team, the video analytics AI agent was able to suggest areas for improvement.

https://blogs.nvidia.com/wp-content/uploads/2025/01/JHH-pitch-metropolis-trim-final.mp4

The $3 trillion media and entertainment industry is also poised to benefit from video analyst AI agents. Through the NVIDIA Media2 initiative, these agents will help drive the creation of smarter, more tailored and more impactful content that can adapt to individual viewer preferences.

Worldwide Adoption and Availability Partners from around the world are integrating the blueprint for building AI agents for video analysis into their own developer workflows, including Accenture, Centific, Deloitte, EY, Infosys, Linker Vision, Pegatron, TATA Consultancy Services (TCS), Telit Cinterion and VAST.

Apply for early access to the NVIDIA Blueprint for video search and summarization.

See notice regarding software product information.

Editor's note: Omdia is the source for 1.5 billion enterprise-level cameras deployed.
LINK: https://blogs.nvidia.com/blog/metropolis-ai-blueprint-video/...
See more stories from nvidia

More from Nvidia

07/02/2025

AI-Designed Proteins Take on Deadly Snake Venom

Every year, venomous snakes kill over 100,000 people and leave 300,000 more with devastating injuries - amputations, paralysis and permanent disabilities. The v...

06/02/2025

When the Earth Talks, AI Listens

AI built for speech is now decoding the language of earthquakes. A team of researchers from the Earth and environmental sciences division at Los Alamos Nationa...

06/02/2025

Medieval Mayhem Arrives With Kingdom Come: Deliverance II' on GeForce NOW

GeForce NOW celebrates its fifth anniversary this February with a lineup of five major releases. The month kicks off with Kingdom Come: Deliverance II. Prepare ...

05/02/2025

Building More Builders: Gooey.AI Makes AI More Accessible Across Communities

When non-technical users can create and deploy reliable AI workflows, organizations can do more to serve their clientele Platforms for developing no- and low-c...

05/02/2025

AI Pays Off: Survey Reveals Financial Industry's Latest Technological Trends

The financial services industry is reaching an important milestone with AI, as organizations move beyond testing and experimentation to successful AI implementa...

05/02/2025

How GeForce RTX 50 Series GPUs Are Built to Supercharge Generative AI on PCs

NVIDIA's GeForce RTX 5090 and 5080 GPUs - which are based on the groundbreaking NVIDIA Blackwell architecture -offer up to 8x faster frame rates with NVIDIA...

04/02/2025

NVIDIA Blackwell Now Generally Available in the Cloud

AI reasoning models and agents are set to transform industries, but delivering their full potential at scale requires massive compute and optimized software. Th...

31/01/2025

Accelerate DeepSeek Reasoning Models With NVIDIA GeForce RTX 50 Series AI PCs

The recently released DeepSeek-R1 model family has brought a new wave of excitement to the AI community, allowing enthusiasts and developers to run state-of-the...

30/01/2025

DeepSeek-R1 Now Live With NVIDIA NIM

DeepSeek-R1 is an open model with state-of-the-art reasoning capabilities. Instead of offering direct responses, AI models like DeepSeek-R1 perform reasoning th...

30/01/2025

GeForce NOW Celebrates Five Years of Cloud Gaming With AAA Blockbusters

GeForce NOW turns five this February. Five incredible years of high-performance gaming have been made possible thanks to the members who've joined the cloud...

30/01/2025

Lights, Camera, Action: New NVIDIA Broadcast AI Features Now Streaming With GeForce RTX 50 Series GPUs

New GeForce RTX 5090 and RTX 5080 GPUs - built on the NVIDIA Blackwell architect...

29/01/2025

Leveling Up User Experiences With Agentic AI, From Bots to Autonomous Agents

AI agents with advanced perception and cognition capabilities are making digital experiences more dynamic and personalized across retail, finance, entertainment...

27/01/2025

Amphitrite Rides AI Wave to Boost Maritime Shipping, Ocean Cleanup With Real-Time Weather Prediction and Simulation

Named after Greek mythology's goddess of the sea, France-based startup Amphi...

23/01/2025

Fast, Low-Cost Inference Offers Key to Profitable AI

Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using t...

23/01/2025

Baldur's Gate 3' Mod Support Launches in the Cloud

GeForce NOW is expanding mod support for hit game Baldur's Gate 3 in collaboration with Larian Studios and mod.io for Ultimate and Performance members. Thi...

22/01/2025

How AI Helps Fight Fraud in Financial Services, Healthcare, Government and More

Companies and organizations are increasingly using AI to protect their customers and thwart the efforts of fraudsters around the world. Voice security company ...

22/01/2025

Into the Omniverse: OpenUSD Workflows Advance Physical AI for Robotics, Autonomous Vehicles

Editor's note: This post is part of Into the Omniverse, a series focused on ...

22/01/2025

The Future of Marketing: How AI Agents Can Enhance Customer Journeys in Retail

AI agents - which can understand, adapt to and support each user's unique journey - are making online shopping and digital marketing more efficient and pers...

21/01/2025

NoTraffic Reduces Road Delays, Carbon Emissions With NVIDIA AI and Accelerated Computing

More than 90 million new vehicles are introduced to roads across the globe every...

16/01/2025

Fantastic Four-ce Awakens: Season One of Marvel Rivals' Joins GeForce NOW

Time to suit up, members. The multiverse is about to get a whole lot cloudier as GeForce NOW opens a portal to the first season of hit game Marvel Rivals from N...

16/01/2025

NVIDIA Releases NIM Microservices to Safeguard Applications for Agentic AI

AI agents are poised to transform productivity for the world's billion knowledge workers with knowledge robots that can accomplish a variety of tasks. To ...

15/01/2025

How AI Is Enhancing Surgical Safety and Education

Troves of unwatched surgical video footage are finding new life, fueling AI tools that help make surgery safer and enhance surgical education. The Surgical Data...

14/01/2025

Healthcare Leaders, NVIDIA CEO Share AI Innovation Across the Industry

AI is making inroads across the entire healthcare industry - from genomic research to drug discovery, clinical trial workflows and patient care. In a fireside ...

14/01/2025

NVIDIA GTC 2025: Quantum Day to Illuminate the Future of Quantum Computing

Quantum computing is one of the most exciting areas in computer science, promising progress in accelerated computing beyond what's considered possible today...

13/01/2025

NVIDIA Statement on the Biden Administration's Misguided AI Diffusion' Rule

For decades, leadership in computing and software ecosystems has been a cornerst...

13/01/2025

NVIDIA Statement on the Biden Administration's Misguided ‘AI Diffusion’ Rule

For decades, leadership in computing and software ecosystems has been a cornerst...

13/01/2025

NVIDIA and IQVIA Build Domain-Expert Agentic AI for Healthcare and Life Sciences

IQVIA, the world's leading provider of clinical research services, commercial insights and healthcare intelligence, is working with NVIDIA to build custom f...

10/01/2025

AI Gets Real for Retailers: 9 Out of 10 Retailers Now Adopting or Piloting AI, Latest NVIDIA Survey Finds

Artificial intelligence is rapidly becoming the cornerstone of innovation in the...

09/01/2025

Hyundai Motor Group Embraces NVIDIA AI and Omniverse for Next-Gen Mobility

Driving the future of smart mobility, Hyundai Motor Group (the Group) is partnering with NVIDIA to develop the next generation of safe, secure mobility with AI ...

09/01/2025

GeForce NOW at CES: Bring PC RTX Gaming Everywhere With the Power of GeForce NOW

This GFN Thursday recaps the latest cloud announcements from the CES trade show, including GeForce RTX gaming expansion across popular devices such as Steam Dec...

08/01/2025

Unveiling a New Era of Local AI With NVIDIA NIM Microservices and AI Blueprints

Over the past year, generative AI has transformed the way people live, work and play, enhancing everything from writing and content creation to gaming, learning...

07/01/2025

Why Enterprises Need AI Query Engines to Fuel Agentic AI

Data is the fuel of AI applications, but the magnitude and scale of enterprise data often make it too expensive and time-consuming to use effectively. Accordin...

07/01/2025

Why World Foundation Models Will Be Key to Advancing Physical AI

In the fast-evolving landscape of AI, it's becoming increasingly important to develop models that can accurately simulate and predict outcomes in physical, ...

06/01/2025

Now See This: NVIDIA Launches Blueprint for AI Agents That Can Analyze Video

The next big moment in AI is in sight - literally. Today, more than 1.5 billion enterprise level cameras deployed worldwide are generating roughly 7 trillion h...

06/01/2025

Building Smarter Autonomous Machines: NVIDIA Announces Early Access for Omniverse Sensor RTX

Generative AI and foundation models let autonomous machines generalize beyond th...

06/01/2025

NVIDIA Unveils Mega' Omniverse Blueprint for Building Industrial Robot Fleet Digital Twins

According to Gartner, the worldwide end-user spending on all IT products for 202...

02/01/2025

How AI Is Helping Us Do Better-for the Planet and for Each Other

Artificial intelligence and accelerated computing are being used to help solve the world's greatest challenges. NVIDIA has reinvented the computing stack -...

02/01/2025

GeForce NOW Rings in the New Year With 14 New Games

GeForce NOW is kicking off 2025 by delivering 14 games to the cloud this month, with two available to stream this week so members can get started on their New Y...

30/12/2024

Research Galore From 2024: Recapping AI Advancements in 3D Simulation, Climate Science and Audio Engineering

The pace of technology innovation has accelerated in the past year, most dramati...

27/12/2024

Have You Heard? 5 AI Podcast Episodes Listeners Loved in 2024

NVIDIA's AI Podcast gives listeners the inside scoop on the ways AI is transforming nearly every industry. Since the show's debut in 2016, it's gar...

26/12/2024

Cheers to 2024: GeForce NOW Recaps Year of Ultimate Cloud Gaming

This GFN Thursday wraps up another incredible year for cloud gaming. Take a look back at the top games and new features that made 2024 a standout for GeForce NO...

24/12/2024

From Generative to Agentic AI, Wrapping the Year's AI Advancements

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...

19/12/2024

AI's in Style: Ulta Beauty Helps Shoppers Virtually Try New Hairstyles

Shoppers pondering a new hairstyle can now try styles before committing to curls or a new color. An AI app by Ulta Beauty, the largest specialty beauty retailer...

19/12/2024

NieR Perfect: GeForce NOW Loops Square Enix's NieR:Automata' and NieR Replicant ver.1.22474487139' Into the Cloud

Stuck in a gaming rut? Get out of the loop this GFN Thursday with four new games...

18/12/2024

AI at Your Service: Digital Avatars With Speech Capabilities Offer Interactive Customer Experiences

Editor's note: This post is part of the AI On blog series, which explores th...

18/12/2024

Imbue's Kanjun Qiu Shares Insights on How to Build Smarter AI Agents

Imagine a future in which everyone is empowered to build and use their own AI agents. That future may not be far off, as new software is infused with intelligen...

18/12/2024

NVIDIA Awards up to $60,000 Research Fellowships to PhD Students

For more than two decades, the NVIDIA Graduate Fellowship Program has supported graduate students doing outstanding work relevant to NVIDIA technologies. Today,...

17/12/2024

AI in Your Own Words: NVIDIA Debuts NeMo Retriever Microservices for Multilingual Generative AI Fueled by Data

In enterprise AI, understanding and working across multiple languages is no long...

17/12/2024

NVIDIA Unveils Its Most Affordable Generative AI Supercomputer

NVIDIA is taking the wraps off a new compact generative AI supercomputer, offering increased performance at a lower price with a software upgrade. The new NVID...

16/12/2024

Tech Leader, AI Visionary, Endlessly Curious Jensen Huang to Keynote CES 2025

On Jan. 6 at 6:30 p.m. PT, NVIDIA founder and CEO Jensen Huang - with his trademark leather jacket and an unwavering vision - will step onto the CES 2025 stage....