GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.
Three technical reasons, and many stories, explain why that's so. Each reason has multiple facets well worth exploring, but at a high level:
GPUs employ parallel processing.
GPU systems scale up to supercomputing heights.
The GPU software stack for AI is broad and deep.
The net result is GPUs perform technical calculations faster and with greater energy efficiency than CPUs. That means they deliver leading performance for AI training and inference as well as gains across a wide array of applications that use accelerated computing.
In its recent report on AI, Stanford's Human-Centered AI group provided some context. GPU performance has increased roughly 7,000 times since 2003 and price per performance is 5,600 times greater, it reported.
A 2023 report captured the steep rise in GPU performance and price/performance. The report also cited analysis from Epoch, an independent research group that measures and forecasts AI advances.
GPUs are the dominant computing platform for accelerating machine learning workloads, and most (if not all) of the biggest models over the last five years have been trained on GPUs [they have] thereby centrally contributed to the recent progress in AI, Epoch said on its site.
A 2020 study assessing AI technology for the U.S. government drew similar conclusions.
We expect [leading-edge] AI chips are one to three orders of magnitude more cost-effective than leading-node CPUs when counting production and operating costs, it said.
NVIDIA GPUs have increased performance on AI inference 1,000x in the last ten years, said Bill Dally, the company's chief scientist in a keynote at Hot Chips, an annual gathering of semiconductor and systems engineers.
ChatGPT Spread the News ChatGPT provided a powerful example of how GPUs are great for AI. The large language model (LLM), trained and run on thousands of NVIDIA GPUs, runs generative AI services used by more than 100 million people.
Since its 2018 launch, MLPerf, the industry-standard benchmark for AI, has provided numbers that detail the leading performance of NVIDIA GPUs on both AI training and inference.
For example, NVIDIA Grace Hopper Superchips swept the latest round of inference tests. NVIDIA TensorRT-LLM, inference software released since that test, delivers up to an 8x boost in performance and more than a 5x reduction in energy use and total cost of ownership. Indeed, NVIDIA GPUs have won every round of MLPerf training and inference tests since the benchmark was released in 2019.
In February, NVIDIA GPUs delivered leading results for inference, serving up thousands of inferences per second on the most demanding models in the STAC-ML Markets benchmark, a key technology performance gauge for the financial services industry.
A RedHat software engineering team put it succinctly in a blog: GPUs have become the foundation of artificial intelligence.
AI Under the Hood A brief look under the hood shows why GPUs and AI make a powerful pairing.
An AI model, also called a neural network, is essentially a mathematical lasagna, made from layer upon layer of linear algebra equations. Each equation represents the likelihood that one piece of data is related to another.
For their part, GPUs pack thousands of cores, tiny calculators working in parallel to slice through the math that makes up an AI model. This, at a high level, is how AI computing works.
Highly Tuned Tensor Cores Over time, NVIDIA's engineers have tuned GPU cores to the evolving needs of AI models. The latest GPUs include Tensor Cores that are 60x more powerful than the first-generation designs for processing the matrix math neural networks use.
In addition, NVIDIA Hopper Tensor Core GPUs include a Transformer Engine that can automatically adjust to the optimal precision needed to process transformer models, the class of neural networks that spawned generative AI.
Along the way, each GPU generation has packed more memory and optimized techniques to store an entire AI model in a single GPU or set of GPUs.
Models Grow, Systems Expand The complexity of AI models is expanding a whopping 10x a year.
The current state-of-the-art LLM, GPT4, packs more than a trillion parameters, a metric of its mathematical density. That's up from less than 100 million parameters for a popular LLM in 2018.
In a recent talk at Hot Chips, NVIDIA Chief Scientist Bill Dally described how single-GPU performance on AI inference expanded 1,000x in the last decade. GPU systems have kept pace by ganging up on the challenge. They scale up to supercomputers, thanks to their fast NVLink interconnects and NVIDIA Quantum InfiniBand networks.
For example, the DGX GH200, a large-memory AI supercomputer, combines up to 256 NVIDIA GH200 Grace Hopper Superchips into a single data-center-sized GPU with 144 terabytes of shared memory.
Each GH200 superchip is a single server with 72 Arm Neoverse CPU cores and four petaflops of AI performance. A new four-way Grace Hopper systems configuration puts in a single compute node a whopping 288 Arm cores and 16 petaflops of AI performance with up to 2.3 terabytes of high-speed memory.
And NVIDIA H200 Tensor Core GPUs announced in November pack up to 288 gigabytes of the latest HBM3e memory technology.
Software Covers the Waterfront An expanding ocean of GPU software has evolved since 2007 to enable every facet of AI, from deep-tech features to high-level applications.
The NVIDIA AI platform includes hundreds of software libraries and apps. The CUDA programming language and the cuDNN-X library for deep learning provide a base on top of which developers have created software like NVIDIA NeMo, a framework to let users build, customize and run inference on their o
North America Stories
06/12/2025
FORT LAUDERDALE, Fla. A new survey from Pixitmedia by Datacore revealed a major shift in the Media & Entertainment industry in media archiving, with 85% of resp...
06/12/2025
HACKENSACK, N.J. LiveU has announced that the national public broadcaster Czech Television has completed one of the largest LiveU live production deployments fo...
06/12/2025
NEW YORK The National Academy of Television Arts and Sciences (NATAS) presented the Excellence in Production Technology Emmy Award to NASA+ and Dr. Tom Leight...
05/12/2025
2025 Sports Broadcasting Hall of Fame: Curt Gowdy Jr. - Master Storyteller, Nati...
05/12/2025
SVG Sit-Down: Veritone's Sean King on the Power of Mining Video, Audio DataThe company's Data Refinery offers users total control and governance over da...
05/12/2025
Platinum White Paper: Inside the Nashville Predators' Unified, Flexible, Sca...
05/12/2025
Netflix Reaches Agreement To Acquire Warner Bros. Following Planned WBD SplitThe deal does not include WBDs sports assets like TNT Sports (US, UK, LatAm), Euros...
05/12/2025
FOX Sports Returns to Indianapolis for Primetime Broadcast of Big Ten Championsh...
05/12/2025
SVG Summit 2025 Preview: Digital Engagement & Monetization Workshop Tackles the ...
05/12/2025
Atlanta United Lights Up New Emory Healthcare Studio With First Live Broadcast f...
05/12/2025
As Messi Takes the Pitch, MLS, Apple, NEP Roll Out Largest MLS Cup Production Ev...
05/12/2025
ESPN Enters College Football's Most Intense Month With Elevated Workflows fo...
05/12/2025
It's about that time! Awards season is in full swing, and the Film Independe...
05/12/2025
LOS ANGELES Netflix announced it has entered into an agreement to acquire the assets of Warner Bros. for $82.7 billion....
05/12/2025
NEW YORK Nielsens Gracenote has launched Gracenote Content Connect, a new ad platform that provides agencies, brands, supply-side platforms (SSPs) and demand-si...
05/12/2025
NEW YORK In an most important update to the workings of deal-based programmatic advertising, IAB Tech Lab has released version 1.0 of its Deals API for public c...
05/12/2025
NEW YORK Pass the turkey. Pass the stuffing. Pass the cranberry sauce. All are common requests of Americans celebrating Thanksgiving Day with family and f...
05/12/2025
NEW YORK Iris, the new cloud-connected camera control platform, has officially launched with features that turn virtually any PTZ camera into a software-connect...
05/12/2025
HOLLYWOOD, Calif. Netflix announced today that it has entered into an agreement to acquire the assets of Warner Bros. for $82.7 billion....
05/12/2025
NEW YORK Iris, the new cloud-connected camera control platform, has officially launched with features that turn virtually any PTZ camera into a software-connect...
05/12/2025
WASHINGTON The Federal Communications Commission has approved AT&T's $1.02 billion acquisition of spectrum from UScellular in a decision that was issued sho...
05/12/2025
The Best Coldplay Songs: 21 Tracks That Shoot for the Stars From Yellow to Viva La Vida, Fix You to Paradise, this playlist goes back to the start.
December ...
05/12/2025
Zafris Lecture Series Brings Nabil Ayers to Berklee The 32nd annual James G. Zafris Distinguished Lecture series was held on Thursday, November 13, with guest...
05/12/2025
Back to All News
Don Lee, Lee Jin-uk, and Lalisa Manobal to Star in Netflix Act...
04/12/2025
SVG Sit-Down: ProximaVision's Claudio Lisman on Why Tethered Drones Could Be...
04/12/2025
SVG Campus Shot Callers: Imry Halevi, Senior Associate Director of Athletics, Co...
04/12/2025
Platinum White Paper: LiveU Lightweight Sports Production: A Step Change in Spor...
04/12/2025
London to Riyadh: DAZN brings the boxing glamour to new production levels for Be...
04/12/2025
Analysis: Paramount bets on the battering ram' with Champions League play By Callum McCarthy, Editor-at-Large
Tuesday, December 2, 2025 - 10:12
Print ...
04/12/2025
Space City Home Network Launches SCHN DTC App for Astros and RocketsThe Rockets and Astros were previously the lone NBA and MLB teams without a DTC appBy Jason...
04/12/2025
SVG Summit 2025 Preview: Content Workflows Workshop Spotlights Evolution of Spor...
04/12/2025
New Sponsor Spotlight: Geotech's Patrick Wambold On the Unreal Engine Revolu...
04/12/2025
Curt Gowdy Jr. - Master Storyteller, Nationally and RegionallyBy Jason Dachman, Editorial Director, U.S.
Thursday, December 4, 2025 - 1:52 pm
Print This Sto...
04/12/2025
(L-R) Rebecca Lichtenfeld, Mohammadreza Eyni, Sara Khaki, and Judith Helfand att...
04/12/2025
Coronal mass ejections caused by eruptions on the surface of the sun can have fa...
04/12/2025
Gracenote Content Connect enables media ecosystem to precisely align ad campaigns and programming based on rich content signals
NEW YORK - December 4, 2025 - N...
04/12/2025
Lightware, a global specialist in AV connectivity, is looking back on a year defined by new advancements, strong collaboration and continued growth. Across the ...
04/12/2025
Riedel Communications today announced a new partnership with Haivision, a leading global provider of mission-critical, real-time video networking and visual col...
04/12/2025
Harmonic (NASDAQ: HLIT) and Normann Engineering today announced a major milestone in their strategic collaboration, celebrating 20 successful broadband deployme...
04/12/2025
Creative software developer Foundry today announced Mari 7.5, the latest iteration of its artist-friendly paint toolset that can handle large, detailed assets w...
04/12/2025
Professional Wireless Systems (PWS), a leading provider of wireless audio solutions and RF management, was on site at Dreamforce 2025 in San Francisco providing...
04/12/2025
LTN's purpose-built IP video network brings all-movie diginet to over 100 stations and streaming platforms in just three months while eliminating satellite ...
04/12/2025
Bitmovin, the leading provider of video streaming solutions, today announced a strategic partnership with ThinkAnalytics, the global leader in AI-powered data a...
04/12/2025
The HELM, a global expert in cinematic live broadcast and high-end production workflows, has signed a partnership agreement with Keslow Camera, one of North Ame...
04/12/2025
At ISE 2026, LiveU will showcase its expanded IP-video EcoSystem, enabling broadcasters, sports, production companies and pro-AV professionals to share their st...
04/12/2025
Since the beginning of commercial television, advertising has been a key part of broadcasting. Over the years, the technology for inserting ads into programs ha...
04/12/2025
MUNICH and MILAN Warner Bros. Discovery said HBO Max is expanding into Germany, Italy, Austria, Switzerland, Luxembourg and Liechtenstein on Jan. 13, 2026, and ...
04/12/2025
SAN FRANCISCO AudioShake has launched its first streaming-capable software development kits (SDKs) designed specifically for real-time music detection and copyr...
04/12/2025
NASHVILLE The mobile and REMI production company TNDV has announced that it headed south into Mexico to live-produce the three-day 2025 Zane Grey Championship P...
GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.
