
GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.
Three technical reasons, and many stories, explain why that's so. Each reason has multiple facets well worth exploring, but at a high level:
GPUs employ parallel processing.
GPU systems scale up to supercomputing heights.
The GPU software stack for AI is broad and deep.
The net result is GPUs perform technical calculations faster and with greater energy efficiency than CPUs. That means they deliver leading performance for AI training and inference as well as gains across a wide array of applications that use accelerated computing.
In its recent report on AI, Stanford's Human-Centered AI group provided some context. GPU performance has increased roughly 7,000 times since 2003 and price per performance is 5,600 times greater, it reported.
A 2023 report captured the steep rise in GPU performance and price/performance. The report also cited analysis from Epoch, an independent research group that measures and forecasts AI advances.
GPUs are the dominant computing platform for accelerating machine learning workloads, and most (if not all) of the biggest models over the last five years have been trained on GPUs [they have] thereby centrally contributed to the recent progress in AI, Epoch said on its site.
A 2020 study assessing AI technology for the U.S. government drew similar conclusions.
We expect [leading-edge] AI chips are one to three orders of magnitude more cost-effective than leading-node CPUs when counting production and operating costs, it said.
NVIDIA GPUs have increased performance on AI inference 1,000x in the last ten years, said Bill Dally, the company's chief scientist in a keynote at Hot Chips, an annual gathering of semiconductor and systems engineers.
ChatGPT Spread the News ChatGPT provided a powerful example of how GPUs are great for AI. The large language model (LLM), trained and run on thousands of NVIDIA GPUs, runs generative AI services used by more than 100 million people.
Since its 2018 launch, MLPerf, the industry-standard benchmark for AI, has provided numbers that detail the leading performance of NVIDIA GPUs on both AI training and inference.
For example, NVIDIA Grace Hopper Superchips swept the latest round of inference tests. NVIDIA TensorRT-LLM, inference software released since that test, delivers up to an 8x boost in performance and more than a 5x reduction in energy use and total cost of ownership. Indeed, NVIDIA GPUs have won every round of MLPerf training and inference tests since the benchmark was released in 2019.
In February, NVIDIA GPUs delivered leading results for inference, serving up thousands of inferences per second on the most demanding models in the STAC-ML Markets benchmark, a key technology performance gauge for the financial services industry.
A RedHat software engineering team put it succinctly in a blog: GPUs have become the foundation of artificial intelligence.
AI Under the Hood A brief look under the hood shows why GPUs and AI make a powerful pairing.
An AI model, also called a neural network, is essentially a mathematical lasagna, made from layer upon layer of linear algebra equations. Each equation represents the likelihood that one piece of data is related to another.
For their part, GPUs pack thousands of cores, tiny calculators working in parallel to slice through the math that makes up an AI model. This, at a high level, is how AI computing works.
Highly Tuned Tensor Cores Over time, NVIDIA's engineers have tuned GPU cores to the evolving needs of AI models. The latest GPUs include Tensor Cores that are 60x more powerful than the first-generation designs for processing the matrix math neural networks use.
In addition, NVIDIA Hopper Tensor Core GPUs include a Transformer Engine that can automatically adjust to the optimal precision needed to process transformer models, the class of neural networks that spawned generative AI.
Along the way, each GPU generation has packed more memory and optimized techniques to store an entire AI model in a single GPU or set of GPUs.
Models Grow, Systems Expand The complexity of AI models is expanding a whopping 10x a year.
The current state-of-the-art LLM, GPT4, packs more than a trillion parameters, a metric of its mathematical density. That's up from less than 100 million parameters for a popular LLM in 2018.
In a recent talk at Hot Chips, NVIDIA Chief Scientist Bill Dally described how single-GPU performance on AI inference expanded 1,000x in the last decade. GPU systems have kept pace by ganging up on the challenge. They scale up to supercomputers, thanks to their fast NVLink interconnects and NVIDIA Quantum InfiniBand networks.
For example, the DGX GH200, a large-memory AI supercomputer, combines up to 256 NVIDIA GH200 Grace Hopper Superchips into a single data-center-sized GPU with 144 terabytes of shared memory.
Each GH200 superchip is a single server with 72 Arm Neoverse CPU cores and four petaflops of AI performance. A new four-way Grace Hopper systems configuration puts in a single compute node a whopping 288 Arm cores and 16 petaflops of AI performance with up to 2.3 terabytes of high-speed memory.
And NVIDIA H200 Tensor Core GPUs announced in November pack up to 288 gigabytes of the latest HBM3e memory technology.
Software Covers the Waterfront An expanding ocean of GPU software has evolved since 2007 to enable every facet of AI, from deep-tech features to high-level applications.
The NVIDIA AI platform includes hundreds of software libraries and apps. The CUDA programming language and the cuDNN-X library for deep learning provide a base on top of which developers have created software like NVIDIA NeMo, a framework to let users build, customize and run inference on their o
North America Stories
17/11/2026
Fabric Insights and Fonn Group have announced a partnership to connect editorial...
17/11/2026
The National Association of Black Journalists is offering up a Thanksgiving Eve treat on Wednesday, Nov. 25, in the form of a live streaming event featuring for...
17/11/2026
Bridge Technologies has announced that its VB440 production probe now supports both monitoring and signal generation for Apple ProRes video streams. The additio...
17/11/2026
Canon U.S.A., Inc. recently welcomed 21 Jericho High School students to its Melv...
17/11/2026
Spectrum News has deployed the Amagi platform to support its live news operations. Amagi says the platform lets Spectrum News streamline newsroom workflows, red...
17/11/2026
Microsoft has expanded its use of Haivision's live video contribution ecosys...
17/11/2026
Setanta Sports has deployed Ateme's TITAN Live platform to support its OTT video delivery operations. Ateme's transcoding technology is integrated into ...
17/11/2026
Media Links has joined the KAIROS Alliance Partner program for Panasonic's KAIROS IT/IP live video production platform. The company completed interoperabili...
17/11/2026
Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...
17/11/2026
In live sports and in life, things go by in the blink of an eye. Savoring the mo...
17/11/2026
Satellite operators are adding a new generation of low-Earth-orbit (LEO) network...
17/11/2026
Clear-Com has delivered five customized test systems for electromagnetic securit...
17/11/2026
Stephen Arnold Music (SAM) has created the sonic identity for WTGL, the women's team golf league from TMRW Sports, created in partnership with the LPGA and ...
17/11/2026
The Professional Audio Manufacturers Alliance (PAMA) has introduced Give Sound a Voice, an initiative aimed at raising awareness of the value of sound among the...
17/11/2026
Fanatics Studios, the sports entertainment joint venture between Fanatics and OBB, has named Jamal Henderson its first President of Film & TV, announced Michael...
17/11/2026
The NFL's Thanksgiving Week games will feature halftime performances by U2, ...
17/11/2026
FanSided, a Minute brand with more than 250 fan communities, has launched a redesigned mobile app that combines team news, analysis, live game coverage, stats a...
17/11/2026
Major League Volleyball (MLV), the longest-running indoor professional volleybal...
17/11/2026
Prime Video will present a 15-hour slate of live sports on Black Friday, Nov. 27...
17/11/2026
Catherine Dior was more than a muse. A Resistance fighter, a survivor of deporta...
17/11/2026
Signiant has acquired the media intelligence technology, Ceivo, developed by Burnlab with founder Aaron Sloman joining the company as Vice President, Product Ma...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Berklees 2027 Alumni Grammy Nominees: See the Full List From pop and R&B to jazz, musical theater, and video game scores, Berklee talent is represented across...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
LA JOLLA, CA-Two Scripps Research faculty, Assistant Professor Christian Diercks...
17/11/2026
Our Biggest Sale of the Year on Ivory 3!
From November 17th through November 30th, enjoy our best savings of the year on Ivory 3 Products!
Whether it's ou...
17/11/2026
Exact 3D Video, a web service based in Japan, has launched its English version, ...
17/11/2026
NVIDIA's Ian Buck kicked off the company's week at SC26 in Chicago on Monday with new tools for AI agents, software that can plan experiments, run code ...
16/11/2026
The ESPN Edge Innovation Conference once again gave key ESPN technology partners as a chance to take to the stage and discuss the future of sports fan experienc...
16/11/2026
As rights holders rethink who tells their stories and where, IMG is preparing th...
16/11/2026
From the sidelines, sonic savants advocate for greater emphasis on sound as sports and entertainment increasingly merge
SVG Live!, a one-day event on Nov. 10 i...
16/11/2026
Deal combines Yamaha's audio technology with Disguise's visual-productio...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Hiltron Communications has received a repeat order from one of Europes longest established communications satellite operators. Central to this project is a Hilt...
16/11/2026
Fabric Insights and Fonn Group today announce a partnership to connect editorial planning and media operations across their complementary product portfolios.
...
16/11/2026
Density 1.0 brings a complete color grading workflow to Final Cut Pro
Brie Clayton November 16, 2026
0 Comments
Filmmaker Pascal Basel's plugin pr...
16/11/2026
Berklee Students to Perform at Boston's CITGO Sign Relighting Kuper Walker, Ollie Mann, and dreamcorp. will perform at the free public celebration in Kenm...
16/11/2026
The Hollywood Reporter Names Berklee Among World's Best Music Schools The annual list spotlights Berklee's screen scoring programs and new master'...
16/11/2026
Five years ago, Lightware launched the original Taurus UCX with a clear vision: USB-C would become the central connection in meeting rooms. The goal was simple:...