
GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.
Three technical reasons, and many stories, explain why that's so. Each reason has multiple facets well worth exploring, but at a high level:
GPUs employ parallel processing.
GPU systems scale up to supercomputing heights.
The GPU software stack for AI is broad and deep.
The net result is GPUs perform technical calculations faster and with greater energy efficiency than CPUs. That means they deliver leading performance for AI training and inference as well as gains across a wide array of applications that use accelerated computing.
In its recent report on AI, Stanford's Human-Centered AI group provided some context. GPU performance has increased roughly 7,000 times since 2003 and price per performance is 5,600 times greater, it reported.
A 2023 report captured the steep rise in GPU performance and price/performance. The report also cited analysis from Epoch, an independent research group that measures and forecasts AI advances.
GPUs are the dominant computing platform for accelerating machine learning workloads, and most (if not all) of the biggest models over the last five years have been trained on GPUs [they have] thereby centrally contributed to the recent progress in AI, Epoch said on its site.
A 2020 study assessing AI technology for the U.S. government drew similar conclusions.
We expect [leading-edge] AI chips are one to three orders of magnitude more cost-effective than leading-node CPUs when counting production and operating costs, it said.
NVIDIA GPUs have increased performance on AI inference 1,000x in the last ten years, said Bill Dally, the company's chief scientist in a keynote at Hot Chips, an annual gathering of semiconductor and systems engineers.
ChatGPT Spread the News ChatGPT provided a powerful example of how GPUs are great for AI. The large language model (LLM), trained and run on thousands of NVIDIA GPUs, runs generative AI services used by more than 100 million people.
Since its 2018 launch, MLPerf, the industry-standard benchmark for AI, has provided numbers that detail the leading performance of NVIDIA GPUs on both AI training and inference.
For example, NVIDIA Grace Hopper Superchips swept the latest round of inference tests. NVIDIA TensorRT-LLM, inference software released since that test, delivers up to an 8x boost in performance and more than a 5x reduction in energy use and total cost of ownership. Indeed, NVIDIA GPUs have won every round of MLPerf training and inference tests since the benchmark was released in 2019.
In February, NVIDIA GPUs delivered leading results for inference, serving up thousands of inferences per second on the most demanding models in the STAC-ML Markets benchmark, a key technology performance gauge for the financial services industry.
A RedHat software engineering team put it succinctly in a blog: GPUs have become the foundation of artificial intelligence.
AI Under the Hood A brief look under the hood shows why GPUs and AI make a powerful pairing.
An AI model, also called a neural network, is essentially a mathematical lasagna, made from layer upon layer of linear algebra equations. Each equation represents the likelihood that one piece of data is related to another.
For their part, GPUs pack thousands of cores, tiny calculators working in parallel to slice through the math that makes up an AI model. This, at a high level, is how AI computing works.
Highly Tuned Tensor Cores Over time, NVIDIA's engineers have tuned GPU cores to the evolving needs of AI models. The latest GPUs include Tensor Cores that are 60x more powerful than the first-generation designs for processing the matrix math neural networks use.
In addition, NVIDIA Hopper Tensor Core GPUs include a Transformer Engine that can automatically adjust to the optimal precision needed to process transformer models, the class of neural networks that spawned generative AI.
Along the way, each GPU generation has packed more memory and optimized techniques to store an entire AI model in a single GPU or set of GPUs.
Models Grow, Systems Expand The complexity of AI models is expanding a whopping 10x a year.
The current state-of-the-art LLM, GPT4, packs more than a trillion parameters, a metric of its mathematical density. That's up from less than 100 million parameters for a popular LLM in 2018.
In a recent talk at Hot Chips, NVIDIA Chief Scientist Bill Dally described how single-GPU performance on AI inference expanded 1,000x in the last decade. GPU systems have kept pace by ganging up on the challenge. They scale up to supercomputers, thanks to their fast NVLink interconnects and NVIDIA Quantum InfiniBand networks.
For example, the DGX GH200, a large-memory AI supercomputer, combines up to 256 NVIDIA GH200 Grace Hopper Superchips into a single data-center-sized GPU with 144 terabytes of shared memory.
Each GH200 superchip is a single server with 72 Arm Neoverse CPU cores and four petaflops of AI performance. A new four-way Grace Hopper systems configuration puts in a single compute node a whopping 288 Arm cores and 16 petaflops of AI performance with up to 2.3 terabytes of high-speed memory.
And NVIDIA H200 Tensor Core GPUs announced in November pack up to 288 gigabytes of the latest HBM3e memory technology.
Software Covers the Waterfront An expanding ocean of GPU software has evolved since 2007 to enable every facet of AI, from deep-tech features to high-level applications.
The NVIDIA AI platform includes hundreds of software libraries and apps. The CUDA programming language and the cuDNN-X library for deep learning provide a base on top of which developers have created software like NVIDIA NeMo, a framework to let users build, customize and run inference on their o
North America Stories
20/02/2026
Gravity Media and Los Angeles-based Green Couch Entertainment announce a strateg...
20/02/2026
IMAX announces it is working with Apple TV to bring the 2026 FIA Formula One Wor...
20/02/2026
Daktronics has partnered with the Philadelphia Phillies to design, manufacture, ...
20/02/2026
ESPN announces the upcoming launch of Women's Sports Sundays - a first-of-it...
20/02/2026
As the Seattle Seahawks and New England Patriots faced off in the NFL's biggest sporting event of the season on Sun., Feb. 8, Sennheiser wireless solutions ...
20/02/2026
ESPN announces its 2026 Major League Baseball spring training schedule, which includes four national games on ESPN, six games on ESPN Unlimited, and more than 2...
20/02/2026
Open Broadcast Systems, which specializes in software-based professional video transport, has added support for 200 Gigabit Ethernet to its range of encoders an...
20/02/2026
Chyron announces the release of PAINT 10.3, which is designed to help analysts and operators turn live action into clearer, faster on-air storytelling.
PAINT 1...
20/02/2026
With full squad workouts underway, MLB Network's live Spring Training game s...
20/02/2026
Tech enhancements, marquee productions are expected to take advantage of a summe...
20/02/2026
In-venue and creative video staffers at the professional and collegiate level ha...
20/02/2026
Ratings Roundup is a rundown of recent rating news and is derived from press rel...
20/02/2026
Speaking with SVG Europe after one of Team GB's greatest days at a Winter Olympics, BBC Sport's head of major events, Ron Chakraborty, explains the broa...
20/02/2026
Making Winter Games Olympic magic is the goal for every broadcaster in Italy cov...
20/02/2026
Curling, one of the least-dangerous Winter Olympic sports, is dominating the Mil...
20/02/2026
BBC Sport's presence at the 2026 Winter Games is centred around a significan...
20/02/2026
BBC Sport is bringing together its linear TV and streaming digital arms in a str...
20/02/2026
To broaden the appeal of winter sports at Milano Cortina, the BBC has integrated...
20/02/2026
Just in time for the start of Apple TV's inaugural season as the exclusive U...
20/02/2026
One big challenge was to depict the character of each of very different and wide...
20/02/2026
(L-R) Writer-director Amanda Kramer photographs the photographers at the premiere of her film By Design at the Library Center Theatre in Park City. (Photo by ...
20/02/2026
NEW YORK - February 10, 2026 - An estimated 125.6* million viewers watched Super Bowl LX on Sunday, February 8, according to Nielsen's Big Data Panel meas...
20/02/2026
NEW YORK - February 19, 2026 - Nielsen today shared updated and final Super Bowl...
20/02/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/02/2026
A leading global investment bank, with offices at Two International Finance Centre in Hong Kong, partnered with systems integrators Global Vision Engineering (G...
20/02/2026
Rise AV and Rise Broadcast, the global not-for-profit organisations dedicated to improving gender diversity across technical industries, have today announced a ...
20/02/2026
Open Broadcast Systems, the leader in software-based professional video transport, has added support for 200 Gigabit Ethernet to its range of encoders and decod...
20/02/2026
Signiant today announced the formation of its Customer Advisory Board (CAB), bringing together a select group of customers to collaborate on product strategy, r...
20/02/2026
PTZOptics today announced the launch of its Visual Reasoning initiative that makes video more actionable by combining robotic PTZ camera systems, AI, and open i...
20/02/2026
Amino, a global media technology provider delivering devices, software and cloud services that simplify and elevate video delivery, today announced the successf...
20/02/2026
SMPTE , the home of media professionals, technologists, and engineers, today announced its call for technical papers for the SMPTE 2026 Media Technology Summit....
20/02/2026
Wowza Media Systems today announced that Granicus, a leading provider of digital engagement solutions for governments, continues to rely on Wowza to power its h...
20/02/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/02/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/02/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/02/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/02/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/02/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/02/2026
Back to All News
A Friend, A Murderer' Launches on Netflix on March 5, 2026
Main image
Entertainment
20 February 2026
GlobalDenmark
Link copied to cl...
19/02/2026
The Canadian rightsholder deploys its most complex' Olympics setup an ever,...
19/02/2026
Suite Studios, a cloud-native platform that connects creative teams to their med...
19/02/2026
Guitar Center and the Tennessee Titans announce a first-of-its-kind partnership ...
19/02/2026
DAZN is reinforcing its leadership in global boxing through a new five-year deal with Matchroom Boxing in the United States and the United Kingdom. The deal ext...
19/02/2026
PTZOptics announces the launch of its visual reasoning initiative that makes vid...
19/02/2026
Influencer Positions, AI-driven vertical video, and platform-native creators res...
19/02/2026
Host broadcaster evolves from world-feed producer to global content orchestrator, unlocking hidden moments for every platform...
19/02/2026
Custom drones raise expectations for bringing viewers closer to the action in new ways
2026 is not the first time OBS has used drones or event First Person Vie...
19/02/2026
Comcast's Xfinity announces record-setting performance from its new RealTime4K technology during Super Bowl LX, delivering the game to customers' homes ...
19/02/2026
The Women's Monobob turned out to be a historic event: Team USA's Elana Meyers Taylor captured the gold at age 41, making her the oldest Bobsleigh gold ...
19/02/2026
The senior from Georgia has found his calling as a technical director and video engineer
In the live-sports-video industry, the future is bright. Our series SV...