Sony Pixel Power calrec Sony

Why GPUs Are Great for AI

04/12/2023

GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.

Three technical reasons, and many stories, explain why that's so. Each reason has multiple facets well worth exploring, but at a high level:

GPUs employ parallel processing.

GPU systems scale up to supercomputing heights.

The GPU software stack for AI is broad and deep.

The net result is GPUs perform technical calculations faster and with greater energy efficiency than CPUs. That means they deliver leading performance for AI training and inference as well as gains across a wide array of applications that use accelerated computing.

In its recent report on AI, Stanford's Human-Centered AI group provided some context. GPU performance has increased roughly 7,000 times since 2003 and price per performance is 5,600 times greater, it reported.

A 2023 report captured the steep rise in GPU performance and price/performance. The report also cited analysis from Epoch, an independent research group that measures and forecasts AI advances.

GPUs are the dominant computing platform for accelerating machine learning workloads, and most (if not all) of the biggest models over the last five years have been trained on GPUs [they have] thereby centrally contributed to the recent progress in AI, Epoch said on its site.

A 2020 study assessing AI technology for the U.S. government drew similar conclusions.

We expect [leading-edge] AI chips are one to three orders of magnitude more cost-effective than leading-node CPUs when counting production and operating costs, it said.

NVIDIA GPUs have increased performance on AI inference 1,000x in the last ten years, said Bill Dally, the company's chief scientist in a keynote at Hot Chips, an annual gathering of semiconductor and systems engineers.

ChatGPT Spread the News ChatGPT provided a powerful example of how GPUs are great for AI. The large language model (LLM), trained and run on thousands of NVIDIA GPUs, runs generative AI services used by more than 100 million people.

Since its 2018 launch, MLPerf, the industry-standard benchmark for AI, has provided numbers that detail the leading performance of NVIDIA GPUs on both AI training and inference.

For example, NVIDIA Grace Hopper Superchips swept the latest round of inference tests. NVIDIA TensorRT-LLM, inference software released since that test, delivers up to an 8x boost in performance and more than a 5x reduction in energy use and total cost of ownership. Indeed, NVIDIA GPUs have won every round of MLPerf training and inference tests since the benchmark was released in 2019.

In February, NVIDIA GPUs delivered leading results for inference, serving up thousands of inferences per second on the most demanding models in the STAC-ML Markets benchmark, a key technology performance gauge for the financial services industry.

A RedHat software engineering team put it succinctly in a blog: GPUs have become the foundation of artificial intelligence.

AI Under the Hood A brief look under the hood shows why GPUs and AI make a powerful pairing.

An AI model, also called a neural network, is essentially a mathematical lasagna, made from layer upon layer of linear algebra equations. Each equation represents the likelihood that one piece of data is related to another.

For their part, GPUs pack thousands of cores, tiny calculators working in parallel to slice through the math that makes up an AI model. This, at a high level, is how AI computing works.

Highly Tuned Tensor Cores Over time, NVIDIA's engineers have tuned GPU cores to the evolving needs of AI models. The latest GPUs include Tensor Cores that are 60x more powerful than the first-generation designs for processing the matrix math neural networks use.

In addition, NVIDIA Hopper Tensor Core GPUs include a Transformer Engine that can automatically adjust to the optimal precision needed to process transformer models, the class of neural networks that spawned generative AI.

Along the way, each GPU generation has packed more memory and optimized techniques to store an entire AI model in a single GPU or set of GPUs.

Models Grow, Systems Expand The complexity of AI models is expanding a whopping 10x a year.

The current state-of-the-art LLM, GPT4, packs more than a trillion parameters, a metric of its mathematical density. That's up from less than 100 million parameters for a popular LLM in 2018.

In a recent talk at Hot Chips, NVIDIA Chief Scientist Bill Dally described how single-GPU performance on AI inference expanded 1,000x in the last decade. GPU systems have kept pace by ganging up on the challenge. They scale up to supercomputers, thanks to their fast NVLink interconnects and NVIDIA Quantum InfiniBand networks.

For example, the DGX GH200, a large-memory AI supercomputer, combines up to 256 NVIDIA GH200 Grace Hopper Superchips into a single data-center-sized GPU with 144 terabytes of shared memory.

Each GH200 superchip is a single server with 72 Arm Neoverse CPU cores and four petaflops of AI performance. A new four-way Grace Hopper systems configuration puts in a single compute node a whopping 288 Arm cores and 16 petaflops of AI performance with up to 2.3 terabytes of high-speed memory.

And NVIDIA H200 Tensor Core GPUs announced in November pack up to 288 gigabytes of the latest HBM3e memory technology.

Software Covers the Waterfront An expanding ocean of GPU software has evolved since 2007 to enable every facet of AI, from deep-tech features to high-level applications.

The NVIDIA AI platform includes hundreds of software libraries and apps. The CUDA programming language and the cuDNN-X library for deep learning provide a base on top of which developers have created software like NVIDIA NeMo, a framework to let users build, customize and run inference on their o
LINK: https://blogs.nvidia.com/blog/why-gpus-are-great-for-ai/...
See more stories from nvidia

North America Stories

20/02/2026

CBS Baltimore Launches New AR/VR Studio

Share Copy link Facebook X Linkedin Bluesky Email...

20/02/2026

IAB Tech Lab Opens Public Comment on Live Event Ad Playbook

Share Copy link Facebook X Linkedin Bluesky Email...

20/02/2026

Study: Sports Programming on Major Streamers Up 52% YoY

Share Copy link Facebook X Linkedin Bluesky Email...

20/02/2026

ESPN to launch Women's Sports Sundays

Share Copy link Facebook X Linkedin Bluesky Email...

20/02/2026

Tegna, Seattle Kraken Extend Broadcast Deal

Share Copy link Facebook X Linkedin Bluesky Email...

20/02/2026

Signiant Announces Customer Advisory Board

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

CBC Navigates Multi-Zone Winter Olympics With Bilingual Production, Remote Studios, Custom Content Hubs

The Canadian rightsholder deploys its most complex' Olympics setup an ever,...

19/02/2026

Suite Studios Announces S3 Native File Streaming for Real-Time Media Workflows at Petabyte Scale

Suite Studios, a cloud-native platform that connects creative teams to their med...

19/02/2026

Guitar Center Named Official Music Gear Retailer and AV Integrator for Tennessee Titans' New Nissan Stadium

Guitar Center and the Tennessee Titans announce a first-of-its-kind partnership ...

19/02/2026

DAZN, Matchroom Boxing Sign Five-Year Deal to Extend Long-Standing Partnership

DAZN is reinforcing its leadership in global boxing through a new five-year deal with Matchroom Boxing in the United States and the United Kingdom. The deal ext...

19/02/2026

PTZOptics Launches Visual Reasoning Initiative, Partners with Moondream to Automate Video Decision-Making

PTZOptics announces the launch of its visual reasoning initiative that makes vid...

19/02/2026

The Influencer Games? OBS Bakes Digital-Native Content Creators - and Athletes - Into Milano Cortina

Influencer Positions, AI-driven vertical video, and platform-native creators res...

19/02/2026

Scaling the Infinite Feed: OBS Redefines the Multi-Platform Olympic Experience

Host broadcaster evolves from world-feed producer to global content orchestrator, unlocking hidden moments for every platform...

19/02/2026

Got Drones? How OBS's FPV Strategy Changes the Game for Everyone

Custom drones raise expectations for bringing viewers closer to the action in new ways 2026 is not the first time OBS has used drones or event First Person Vie...

19/02/2026

Xfinity Delivered Super Bowl LX Faster Than Any Other Provider

Comcast's Xfinity announces record-setting performance from its new RealTime4K technology during Super Bowl LX, delivering the game to customers' homes ...

19/02/2026

Cortina Sliding Center Photo Gallery: Women's Monobob

The Women's Monobob turned out to be a historic event: Team USA's Elana Meyers Taylor captured the gold at age 41, making her the oldest Bobsleigh gold ...

19/02/2026

SVG Students To Watch: Philip Doherty, Elon University

The senior from Georgia has found his calling as a technical director and video engineer In the live-sports-video industry, the future is bright. Our series SV...

19/02/2026

Cosm Expands its Leadership Team to Scale Its Sports and Entertainment Division

Cosm announces the appointment of two new leaders - Jeff Hughes (President, Sports and Entertainment) and Rob Laycock (Vice President, Head of Venues Marketing)...

19/02/2026

NBC Olympics Brings Its Signature Studio Look Home' to the IBC in Milan

Two main sets feature Unreal Engine environment, LED architecture, camera-tracked virtual windows; a third, in the TV Tower,' overlooks the Duomo...

19/02/2026

Warner Bros. Discovery returns to position one in Poland for first time since May 2025

WBD edges above Polsat in only change in the top 10 content distributor rankings...

19/02/2026

Nielsen data shows Kiwis' love of travel drives 10% lift in ad investment

Airlines and cruise operators ramp up spend as New Zealanders plan, compare and book online Auckland, February 19, 2026 - New Nielsen Ad Intel data shows adver...

19/02/2026

The Gauge: Poland | January 2026

The beginning of the new year brought a clear revival in front of television screens. The winter aura meant that Poles spent an average of 5% more time watching...

19/02/2026

Sports programming on major streaming platforms surges 52% year-over-year, according to Gracenote

New data also shows news content on FAST channels up 58% as streaming catalogs c...

19/02/2026

Sinclair Launches New True Crime Daily Video Podcast

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

Berklee Awards Fenway Neighborhood Improvement Grants to Six Organizations

Berklee Awards Fenway Neighborhood Improvement Grants to Six Organizations A total of $25,000 will be distributed among the Boston-based nonprofits. February...

19/02/2026

Submissions open for Best of Show Awards at NAB 2026

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

SMPTE Issues Call for Summit Papers

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

Broadband Usage Jumps by 9.9% in Q4

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

NBC Sports Details NEPs Extensive Work for Winter Olympic Coverage

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

Fuse Media and Complex to Launch Complex TV

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

Spincast Gets U.S. Patent for AI-Powered Shoppable TV

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

IBCAP Announces $21 Million Lawsuit Against DMTN IPTV

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

NRB Backs ATSC 3.0 Tuner and Must-Carry Requirements

Share Copy link Facebook X Linkedin Bluesky Email...

19/02/2026

Foundry Acquires Griptape to Accelerate AI Integration Ac...

Foundry, the leading developer of creative software for the Media and Entertainment industry, today announced the completion of its acquisition of Griptape, a p...

19/02/2026

Scott D Smith Captures the Chaos and Intimacy of The Bear...

Capturing the raw energy and emotional intensity of FX's hit series The Bear is no small feat, especially when the set itself is as hectic and unpredictab...

19/02/2026

Cobalt Digital Products Receive IPMX Product Certificatio...

Cobalt Digital Inc., a leading designer and manufacturer of award-winning video and audio conversion, processing, and distribution solutions, and a founder of o...

19/02/2026

DHD to Demonstrate Latest-Generation Audio Production Sys...

DHD s complete range of digital audio mixers plus the latest-generation RM1 Pro broadcast-from-anywhere unit will be promoted at the upcoming Lokalradioforbunds...

19/02/2026

Media Companies Gain Operations Observability With Operat...

Operative, the preferred advertising management solution provider for the world's leading media brands, today announced a partnership with GraySwan to bring...

19/02/2026

Witbe Launches New Agentic SDK to Power Agentic Automatio...

Witbe today announced the launch of Agentic SDK, a new test automation framework designed to help video service providers build, operate, and scale agentic auto...

19/02/2026

Network18 Media and Investments Selects Grass Valleys Pla...

Grass Valley today announced that Network18 Media & Investments Ltd., one of India's largest and most influential media conglomerates, is deploying Grass Va...

19/02/2026

Clear-Com to Sponsor and Present USITT Stage Management A...

Clear-Com is sponsoring and presenting the USITT Stage Management Award during the United States Institute for Theatre Technology (USITT) Annual Conference & S...

19/02/2026

Paramores Hayley Williams Praises Berklee Ensemble: Make Noise

Paramores Hayley Williams Praises Berklee Ensemble: Make Noise Williams acknowledged the Paramore Ensemble after their show at the Berklee Performance Center ...

19/02/2026

TRIBECA ANNOUNCES NEW SLATE OF STAR-STUDDED MEMBERSHIP EVENTS FEATURING KATE MCKINNON AND TRACY MORGAN

February 19th, 2026 TRIBECA ANNOUNCES NEW SLATE OF STAR-STUDDED MEMBERSHIP EVEN...

19/02/2026

News Corp Announces Match for U.S. Government's $1,000 Contribution to Children's Investment Accounts

News Corp Announces Match for U.S. Government's $1,000 Contribution to Child...

19/02/2026

Main Trailer for Boyfriend on Demand' Serves Up Virtual Dating Fantasy Packed With Swoon-Worthy Moments

Back to All News Main Trailer for Boyfriend on Demand' Serves Up Virtual D...

19/02/2026

Meet the Singles of 'Love is Blind: Sweden' Season 3

Back to All News Meet the Singles of Love is Blind: Sweden Season 3 Entertainment 19 February 2026 GlobalSweden Link copied to clipboard Meet the singles ...

19/02/2026

All About the Games: Play Over 4,500 Titles With GeForce NOW

The GeForce NOW anniversary celebration keeps on rolling, and this week is all about the games that make it possible. With more than 4,500 titles supported in t...

19/02/2026

Survey Reveals AI Advances in Telecom: Networks and Automation in Driver's Seat as Return on Investment Climbs

AI is accelerating the telecommunications industry's transformation, becomin...

19/02/2026

February 18, 2026

Mapping protein production in brain cells yields new insights for brain disease Scripps Research and UC San Diego scientists used a novel method to show that so...

18/02/2026

Milano Cortina 2026: How OBS is Utilizing Audio QC at the First Winter Games to Use Immersive Sound

Audio quality control (QC) is becoming ever more crucial for Olympic Broadcastin...