GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.
Three technical reasons, and many stories, explain why that's so. Each reason has multiple facets well worth exploring, but at a high level:
GPUs employ parallel processing.
GPU systems scale up to supercomputing heights.
The GPU software stack for AI is broad and deep.
The net result is GPUs perform technical calculations faster and with greater energy efficiency than CPUs. That means they deliver leading performance for AI training and inference as well as gains across a wide array of applications that use accelerated computing.
In its recent report on AI, Stanford's Human-Centered AI group provided some context. GPU performance has increased roughly 7,000 times since 2003 and price per performance is 5,600 times greater, it reported.
A 2023 report captured the steep rise in GPU performance and price/performance. The report also cited analysis from Epoch, an independent research group that measures and forecasts AI advances.
GPUs are the dominant computing platform for accelerating machine learning workloads, and most (if not all) of the biggest models over the last five years have been trained on GPUs [they have] thereby centrally contributed to the recent progress in AI, Epoch said on its site.
A 2020 study assessing AI technology for the U.S. government drew similar conclusions.
We expect [leading-edge] AI chips are one to three orders of magnitude more cost-effective than leading-node CPUs when counting production and operating costs, it said.
NVIDIA GPUs have increased performance on AI inference 1,000x in the last ten years, said Bill Dally, the company's chief scientist in a keynote at Hot Chips, an annual gathering of semiconductor and systems engineers.
ChatGPT Spread the News ChatGPT provided a powerful example of how GPUs are great for AI. The large language model (LLM), trained and run on thousands of NVIDIA GPUs, runs generative AI services used by more than 100 million people.
Since its 2018 launch, MLPerf, the industry-standard benchmark for AI, has provided numbers that detail the leading performance of NVIDIA GPUs on both AI training and inference.
For example, NVIDIA Grace Hopper Superchips swept the latest round of inference tests. NVIDIA TensorRT-LLM, inference software released since that test, delivers up to an 8x boost in performance and more than a 5x reduction in energy use and total cost of ownership. Indeed, NVIDIA GPUs have won every round of MLPerf training and inference tests since the benchmark was released in 2019.
In February, NVIDIA GPUs delivered leading results for inference, serving up thousands of inferences per second on the most demanding models in the STAC-ML Markets benchmark, a key technology performance gauge for the financial services industry.
A RedHat software engineering team put it succinctly in a blog: GPUs have become the foundation of artificial intelligence.
AI Under the Hood A brief look under the hood shows why GPUs and AI make a powerful pairing.
An AI model, also called a neural network, is essentially a mathematical lasagna, made from layer upon layer of linear algebra equations. Each equation represents the likelihood that one piece of data is related to another.
For their part, GPUs pack thousands of cores, tiny calculators working in parallel to slice through the math that makes up an AI model. This, at a high level, is how AI computing works.
Highly Tuned Tensor Cores Over time, NVIDIA's engineers have tuned GPU cores to the evolving needs of AI models. The latest GPUs include Tensor Cores that are 60x more powerful than the first-generation designs for processing the matrix math neural networks use.
In addition, NVIDIA Hopper Tensor Core GPUs include a Transformer Engine that can automatically adjust to the optimal precision needed to process transformer models, the class of neural networks that spawned generative AI.
Along the way, each GPU generation has packed more memory and optimized techniques to store an entire AI model in a single GPU or set of GPUs.
Models Grow, Systems Expand The complexity of AI models is expanding a whopping 10x a year.
The current state-of-the-art LLM, GPT4, packs more than a trillion parameters, a metric of its mathematical density. That's up from less than 100 million parameters for a popular LLM in 2018.
In a recent talk at Hot Chips, NVIDIA Chief Scientist Bill Dally described how single-GPU performance on AI inference expanded 1,000x in the last decade. GPU systems have kept pace by ganging up on the challenge. They scale up to supercomputers, thanks to their fast NVLink interconnects and NVIDIA Quantum InfiniBand networks.
For example, the DGX GH200, a large-memory AI supercomputer, combines up to 256 NVIDIA GH200 Grace Hopper Superchips into a single data-center-sized GPU with 144 terabytes of shared memory.
Each GH200 superchip is a single server with 72 Arm Neoverse CPU cores and four petaflops of AI performance. A new four-way Grace Hopper systems configuration puts in a single compute node a whopping 288 Arm cores and 16 petaflops of AI performance with up to 2.3 terabytes of high-speed memory.
And NVIDIA H200 Tensor Core GPUs announced in November pack up to 288 gigabytes of the latest HBM3e memory technology.
Software Covers the Waterfront An expanding ocean of GPU software has evolved since 2007 to enable every facet of AI, from deep-tech features to high-level applications.
The NVIDIA AI platform includes hundreds of software libraries and apps. The CUDA programming language and the cuDNN-X library for deep learning provide a base on top of which developers have created software like NVIDIA NeMo, a framework to let users build, customize and run inference on their o
North America Stories
03/12/2025
WASHINGTON The Federal Communications Commission has opened a docket for comments on the proposed $6.2 billion Nexstar acquisition for Tegna and set deadlines f...
03/12/2025
MILAN, Italy Brightcove has released seven new features designed to expand global reach, improve audience engagement, enhance live-streaming quality and streaml...
03/12/2025
NEW YORK Great American Media said it plans to launch Pure Flix Familia, a dedicated Spanish-language platform, in 2026....
03/12/2025
SOUTHPORT, Conn. Main Street Sports Group has announced that the FanDuel Sports Network app is now available directly on Vizio and on smart TVs with Vizio OS. T...
03/12/2025
SAN JOSE, Calif. Harmonic has announced that Telia, the second-largest telecom operator in Norway, is modernizing its broadband network with the company's c...
03/12/2025
GREEN BAY, Wis. Sinclair said Jay Zollar, vice president and general manager of WLUK-WCWF here, will retire Dec. 31 after 26 years running the stations. Station...
02/12/2025
Case Study: How Mid-Atlantic Sports Network Moved to All-IP Distribution in 60 D...
02/12/2025
2025 Sports Broadcasting Hall of Fame: Lee Corso, Coach, Commentator, FirebrandBy Ken Kerschbaumer
Tuesday, December 2, 2025 - 7:00 am
Print This Story | S...
02/12/2025
SVG All-Stars: Dan Nabors, Senior Director, Remote Engineering, TNT SportsThe veteran tech leader is helping guide Warner Bros. Discovery's at-home' re...
02/12/2025
National Lacrosse League Opens Season With New Cloud-Based Official Replay-Revie...
02/12/2025
Platinum White Paper: The Cinematic Look in Live Production - Bridging Aesthetic...
02/12/2025
SVG India Announces Advisory Board; JioStar's Prashant Khanna Named ChairmanBy Ken Kerschbaumer
Tuesday, December 2, 2025 - 11:15 am
Print This Story |...
02/12/2025
SVG Summit 2025 Preview: New AI Production Tools Workshop' Features ESPN, F...
02/12/2025
Written and directed by Hailey Gates, Atropia won the U.S. Grand Jury Prize: Dramatic at the 2025 Sundance Film Festival....
02/12/2025
The lightweight and highly portable CORVUS-RAVEN, recently demonstrated at the V...
02/12/2025
The lightweight, rugged Panther 2 tri-band VSAT delivers high-speed data communications for Internet, VPN connectivity and video transmission over commercial an...
02/12/2025
NGID volunteers in the United States
NGID volunteers in Mexico
NGID volunteers in India
NGID volunteers in Spain
NGID volunteers in Australia
NGID voluntee...
02/12/2025
WASHINGTON The Federal Communications Commission's Media Bureau has issued a reminder that stations in DMAs 111 through 120 must implement its audio descrip...
02/12/2025
SAN DIEGO Sony Electronics today introduced the Alpha 7 V (ILCE-7M5), the fifth generation of its Alpha 7 full-frame mirrorless lineup, powered by the newly dev...
02/12/2025
WASHINGTON The Federal Communications Commission has opened a docket for comments on Nexstar Media Group's proposed $6.2 billion acquisition of Tegna and se...
02/12/2025
WASHINGTON The National Association of Broadcasters has added a trio of media executives, Veronica Rodriguez, Matt Schnaars and Jeff Warshaw, to its board of di...
02/12/2025
MILAN, Italy Brightcove has released seven new features designed to expand global reach, improve audience engagement, enhance live-streaming quality and streaml...
02/12/2025
BCNEXXT, the developers of the advanced playout platform Vipe, today announced a new Service Provider Partner agreement with NPC Media, the Australian managed p...
02/12/2025
DPA Microphones today announces the immediate availability of its new N-Series Digital Wireless Microphone System for customers throughout the U.S. In addition ...
02/12/2025
Hiltron Communications reports a well attended and successful Space Tech Expo exhibition, held at the Messe Bremen from Tuesday November 18 through Thursday Nov...
02/12/2025
The beginning of September saw Riverfire by Australian Retirement Trust, in association with Channel 9 and Triple M, kick off the 2025 Brisbane Festival in spec...
02/12/2025
SGM Lighting is pleased to announce an exciting new partnership with ACT Entertainment, under which ACT will serve as the exclusive distributor for SGM's ne...
02/12/2025
Harmonic (NASDAQ: HLIT) today announced that Telia, the second largest telecom operator in Norway, is modernizing its broadband network with the company's i...
02/12/2025
European cloud data protection specialist Keepit is expanding its sales network in the DACH region and has gained TIM AG, one of the leading value-added distrib...
02/12/2025
Featured products at ISE 2026
This year s ISE 2026 theme "Push Beyond" reflects Calrec s direction of travel. The company is dedicated to supporting the AV an...
02/12/2025
Czech Television, the national public broadcaster, has deployed one of the largest LiveU live production projects in the region for live sports and news coverag...
02/12/2025
Test & measurement innovator, Leader Electronics, has announced the release of v8.2 software for its ZEN and ZEN-W Series, bringing advanced cinematic camera se...
02/12/2025
Global provider of video streaming software and services, Accedo, and OTT platform provider, Magine Pro, have signed a binding agreement to carve out and merge ...
02/12/2025
With proven success in broadcast, OTT, and content supply chains, K.S Avinash set to scale Skandha's service-first approach across APAC
Skandha Media Servi...
02/12/2025
AJA Video Systems has joined the RAVENNA community, a network of partners devoted to advancing the RAVENNA standard for distributing real-time audio over intern...
02/12/2025
HUNT VALLEY, Md. Sinclair and its wholly-owned subsidiary One Media Technologies have released a new report focused on how ATSC 3.0, the IP-based NextGen TV sta...
02/12/2025
WASHINGTON The Federal Communications Commission's Media Bureau has issued a reminder that stations in DMAs 111 through 120 must implement its audio descrip...
02/12/2025
WASHINGTON Members of the Advanced Television Systems Committee have elected four industry veterans to three-year terms on the organization's board of direc...
02/12/2025
A new survey from S&P Global Market Intelligence Kagan highlights how important social media is for younger consumers, with Gen Z spending 5.1 hours a day on so...
02/12/2025
Faculty Notes: Fall/Winter 2025 Recent accomplishments, releases, and events by Berklee faculty.
December 1, 2025
By
Editorial Staff
The cast of Kind of M...
02/12/2025
Today, Mistral AI announced the Mistral 3 family of open-source multilingual, multimodal models, optimized across NVIDIA supercomputing and edge platforms.
M...
02/12/2025
Ring In The Holidays With Our Biggest Sale Ever On Ivory II Pianos!From now until December 31st, you can save 50% or more on every Ivory II piano and collection...
02/12/2025
Back to All News
Made in America: How Stranger Things' Electrified the US Economy
Entertainment
02 December 2025
GlobalUnited States
Link copied to cl...
02/12/2025
Back to All News
Culinary Class Wars' Season 2 Trailer Teases New Rules, R...
02/12/2025
At AWS re:Invent, NVIDIA and Amazon Web Services expanded their strategic collab...
01/12/2025
L3Harris and PentenAmio formalise their teaming agreement at MilCIS 2025, streng...
01/12/2025
Artemis II is NASA's first crewed flight test of the Space Launch System rocket and Orion spacecraft. The crew, from left: Commander Reid Wiseman, Pilot Vic...
01/12/2025
IRVINE, Calif. Wooden Camera has introduced its new Accessory Collection for the Canon EOS C50. The new lineup includes a low-profile, gimbal-ready cage, expand...
01/12/2025
WASHINGTON The Federal Communications Commission has released a tentative agenda for its Dec. 18 Open Commission Meeting that will include a vote on a report an...
GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.
