
GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.
Three technical reasons, and many stories, explain why that's so. Each reason has multiple facets well worth exploring, but at a high level:
GPUs employ parallel processing.
GPU systems scale up to supercomputing heights.
The GPU software stack for AI is broad and deep.
The net result is GPUs perform technical calculations faster and with greater energy efficiency than CPUs. That means they deliver leading performance for AI training and inference as well as gains across a wide array of applications that use accelerated computing.
In its recent report on AI, Stanford's Human-Centered AI group provided some context. GPU performance has increased roughly 7,000 times since 2003 and price per performance is 5,600 times greater, it reported.
A 2023 report captured the steep rise in GPU performance and price/performance. The report also cited analysis from Epoch, an independent research group that measures and forecasts AI advances.
GPUs are the dominant computing platform for accelerating machine learning workloads, and most (if not all) of the biggest models over the last five years have been trained on GPUs [they have] thereby centrally contributed to the recent progress in AI, Epoch said on its site.
A 2020 study assessing AI technology for the U.S. government drew similar conclusions.
We expect [leading-edge] AI chips are one to three orders of magnitude more cost-effective than leading-node CPUs when counting production and operating costs, it said.
NVIDIA GPUs have increased performance on AI inference 1,000x in the last ten years, said Bill Dally, the company's chief scientist in a keynote at Hot Chips, an annual gathering of semiconductor and systems engineers.
ChatGPT Spread the News ChatGPT provided a powerful example of how GPUs are great for AI. The large language model (LLM), trained and run on thousands of NVIDIA GPUs, runs generative AI services used by more than 100 million people.
Since its 2018 launch, MLPerf, the industry-standard benchmark for AI, has provided numbers that detail the leading performance of NVIDIA GPUs on both AI training and inference.
For example, NVIDIA Grace Hopper Superchips swept the latest round of inference tests. NVIDIA TensorRT-LLM, inference software released since that test, delivers up to an 8x boost in performance and more than a 5x reduction in energy use and total cost of ownership. Indeed, NVIDIA GPUs have won every round of MLPerf training and inference tests since the benchmark was released in 2019.
In February, NVIDIA GPUs delivered leading results for inference, serving up thousands of inferences per second on the most demanding models in the STAC-ML Markets benchmark, a key technology performance gauge for the financial services industry.
A RedHat software engineering team put it succinctly in a blog: GPUs have become the foundation of artificial intelligence.
AI Under the Hood A brief look under the hood shows why GPUs and AI make a powerful pairing.
An AI model, also called a neural network, is essentially a mathematical lasagna, made from layer upon layer of linear algebra equations. Each equation represents the likelihood that one piece of data is related to another.
For their part, GPUs pack thousands of cores, tiny calculators working in parallel to slice through the math that makes up an AI model. This, at a high level, is how AI computing works.
Highly Tuned Tensor Cores Over time, NVIDIA's engineers have tuned GPU cores to the evolving needs of AI models. The latest GPUs include Tensor Cores that are 60x more powerful than the first-generation designs for processing the matrix math neural networks use.
In addition, NVIDIA Hopper Tensor Core GPUs include a Transformer Engine that can automatically adjust to the optimal precision needed to process transformer models, the class of neural networks that spawned generative AI.
Along the way, each GPU generation has packed more memory and optimized techniques to store an entire AI model in a single GPU or set of GPUs.
Models Grow, Systems Expand The complexity of AI models is expanding a whopping 10x a year.
The current state-of-the-art LLM, GPT4, packs more than a trillion parameters, a metric of its mathematical density. That's up from less than 100 million parameters for a popular LLM in 2018.
In a recent talk at Hot Chips, NVIDIA Chief Scientist Bill Dally described how single-GPU performance on AI inference expanded 1,000x in the last decade. GPU systems have kept pace by ganging up on the challenge. They scale up to supercomputers, thanks to their fast NVLink interconnects and NVIDIA Quantum InfiniBand networks.
For example, the DGX GH200, a large-memory AI supercomputer, combines up to 256 NVIDIA GH200 Grace Hopper Superchips into a single data-center-sized GPU with 144 terabytes of shared memory.
Each GH200 superchip is a single server with 72 Arm Neoverse CPU cores and four petaflops of AI performance. A new four-way Grace Hopper systems configuration puts in a single compute node a whopping 288 Arm cores and 16 petaflops of AI performance with up to 2.3 terabytes of high-speed memory.
And NVIDIA H200 Tensor Core GPUs announced in November pack up to 288 gigabytes of the latest HBM3e memory technology.
Software Covers the Waterfront An expanding ocean of GPU software has evolved since 2007 to enable every facet of AI, from deep-tech features to high-level applications.
The NVIDIA AI platform includes hundreds of software libraries and apps. The CUDA programming language and the cuDNN-X library for deep learning provide a base on top of which developers have created software like NVIDIA NeMo, a framework to let users build, customize and run inference on their o
North America Stories
28/04/2026
Rise, the award-winning advocacy group for gender diversity in the broadcast and media technology sector, is pleased to announce a new global training programme...
28/04/2026
Clear-Com has appointed Brian Grahn as Market Outreach Manager of the Americas and Ben Turnwell as Business Development Manager for EMEA live, expanding their ...
28/04/2026
LiveU is inviting MPTS visitors to step into the companys new Q Era on Stand D32, at The Grand Hall, Olympia, London (May 13-14). The company will showcase its ...
28/04/2026
IBC today announces the launch of the IBC2026 Innovation Awards, with nominations now open for projects, programmes and initiatives that exemplify breakthrough ...
28/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/04/2026
Introducing Nx 3-Strip v2 - A Physics-Based Technicolor Reconstruction for DaVin...
27/04/2026
CES Power, a provider of infrastructure for live events, has announced the acquisition of three Ireland-based businesses: GH Energy Rental Ltd, Event Power, and...
27/04/2026
FuboTV Inc. has announced it is developing its Multiview feature for the Fubo streaming service on select LG TVs, including 2024, 2025, and newer 4K and 8K mode...
27/04/2026
Shade, a file management platform for creative teams, has announced a $14 million funding round led by Khosla Ventures, Construct Capital, and Bling Capital, br...
27/04/2026
The Audio Engineering Society (AES) will present the Immersive Audio Academy 12th Edition - Immersive Audio in All Flavors - on April 30, 2026, at 12:00 p.m. ...
27/04/2026
DAZN has announced DAZN48, a creator program for the FIFA World Cup 2026 that will recruit 48 creators - one representing each of the 48 qualified nations - to ...
27/04/2026
FloSports has announced exclusive streaming rights to four CrossFit competitions: Legends Del Mar: CrossFit Semi-Finals, Magic City Games, NorCal Classic, and t...
27/04/2026
Telestream has announced that Pulse, its software-defined test and measurement p...
27/04/2026
Shade has announced it is a Cloud Computing and Storage winner in the 2026 NAB Show Product of the Year Awards. Winners were selected by a panel of industry exp...
27/04/2026
Leading the NBA's video-ads platform, this Penn State grad is at the forefro...
27/04/2026
DAZN has expanded its international broadcast rights for the T100 Triathlon World Tour to include Africa. All races from the T100 calendar will be available for...
27/04/2026
NAB Show has announced the recipients of its 2026 Project of the Year and Product of the Year Awards at a ceremony at the Las Vegas Convention Center.
Each wi...
27/04/2026
DIRECTV has launched on Meta Quest headsets, becoming the first MVPD to offer live TV through the platform. The timing coincides with the stretch run of the MLB...
27/04/2026
TBL Team Boxing League has announced a broadcast agreement with MSG Networks to air all remaining Season 4 fights live across MSG's television and digital p...
27/04/2026
IP integration, interoperability, growth of intercommunications were key concerns for vendors and visitors alike
Attendees at the recently concluded 2026 NAB S...
27/04/2026
Behind The Mic provides a roundup of recent news regarding on-air talent, includ...
27/04/2026
A pro-audio emphasis, spectrum changes, and on-field audio mark the new products and enhancements to existing offerings
Microphones remain the primary point of...
27/04/2026
The Sports Video Group team was all over the NAB Show floor out in Las Vegas las...
27/04/2026
Available for current Nielsen ONE Ads customers, this new capability predicts sa...
27/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
27/04/2026
Nippon TV's In-House Proprietary AI Solution AiDi Wins Product of the Year A...
27/04/2026
Outpost Introduces Unlimited Collaboration Model for Review and Approval Workflo...
27/04/2026
Ikegami Announces VFE-P07D Monocular OLED Viewfinder with Tiltable 3.5-inch LCD ...
27/04/2026
Custom Consoles announces the completion of a large Module-R desk and MediaWall monitor display mount for an expanded master control room at Gravity Medias West...
27/04/2026
Other World Computing Launches OWC Express 4M2 Ultra - Thunderbolt 5 Four-Slot N...
27/04/2026
Back to All News
Netflix announces El sobrino, the new film by Dami n Szifron s...
26/04/2026
Back to All News
Director Yoon Jong-bin Returns with The Generals' (WT), A...
26/04/2026
Back to All News
Nine Queens, Starring Alvaro Morte and Patrick Criado, Starts ...
25/04/2026
Mediagenix Sweeps 2026 NAB Awards With Wins for Product of the Year and Best of ...
25/04/2026
SCHOEPS Microphones Announces Desert Island Boom Set for NAB 2026
Brie Clayton April 24, 2026
0 Comments
Compact modular set ideal for location sound ...
25/04/2026
Berklee Africana Studies Hosts Gospel Extravaganza 2026 The Signature Series event will honor three new inductees to the Berklee Gospel Hall of Fame and celeb...
25/04/2026
Student Spotlight: Siva Maja Gierszewska, who performs under the artist name Siva, shares how she found her songwriting confidence at Berklee.
April 24, 2026...
25/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
24/04/2026
Churchill Downs Inc. (CDI) has announced a definitive agreement to acquire the intellectual property of the Preakness Stakes and Black-Eyed Susan Stakes from 1/...
24/04/2026
Daktronics has partnered with DCL (Design Communications, Ltd.) to design, manuf...
24/04/2026
Chyron has announced PRIME Translate, a workflow solution that produces live content simultaneously in multiple languages within the PRIME platform. The system ...
24/04/2026
Eutelsat has announced a new partnership with Co-op Cable, introducing an expand...
24/04/2026
Pitch Dublin, an indoor golf simulation and hospitality venue on Dawson Street i...
24/04/2026
G&D and VuWall have announced two senior leadership appointments, effective Apri...
24/04/2026
Victory , the free sports streaming service from A Parent Media Co. Inc. (APMC), has announced a multi-year agreement to become the exclusive local streaming ho...