Sony Pixel Power calrec Sony

Why GPUs Are Great for AI

04/12/2023

GPUs have been called the rare Earth metals - even the gold - of artificial intelligence, because they're foundational for today's generative AI era.

Three technical reasons, and many stories, explain why that's so. Each reason has multiple facets well worth exploring, but at a high level:

GPUs employ parallel processing.

GPU systems scale up to supercomputing heights.

The GPU software stack for AI is broad and deep.

The net result is GPUs perform technical calculations faster and with greater energy efficiency than CPUs. That means they deliver leading performance for AI training and inference as well as gains across a wide array of applications that use accelerated computing.

In its recent report on AI, Stanford's Human-Centered AI group provided some context. GPU performance has increased roughly 7,000 times since 2003 and price per performance is 5,600 times greater, it reported.

A 2023 report captured the steep rise in GPU performance and price/performance. The report also cited analysis from Epoch, an independent research group that measures and forecasts AI advances.

GPUs are the dominant computing platform for accelerating machine learning workloads, and most (if not all) of the biggest models over the last five years have been trained on GPUs [they have] thereby centrally contributed to the recent progress in AI, Epoch said on its site.

A 2020 study assessing AI technology for the U.S. government drew similar conclusions.

We expect [leading-edge] AI chips are one to three orders of magnitude more cost-effective than leading-node CPUs when counting production and operating costs, it said.

NVIDIA GPUs have increased performance on AI inference 1,000x in the last ten years, said Bill Dally, the company's chief scientist in a keynote at Hot Chips, an annual gathering of semiconductor and systems engineers.

ChatGPT Spread the News ChatGPT provided a powerful example of how GPUs are great for AI. The large language model (LLM), trained and run on thousands of NVIDIA GPUs, runs generative AI services used by more than 100 million people.

Since its 2018 launch, MLPerf, the industry-standard benchmark for AI, has provided numbers that detail the leading performance of NVIDIA GPUs on both AI training and inference.

For example, NVIDIA Grace Hopper Superchips swept the latest round of inference tests. NVIDIA TensorRT-LLM, inference software released since that test, delivers up to an 8x boost in performance and more than a 5x reduction in energy use and total cost of ownership. Indeed, NVIDIA GPUs have won every round of MLPerf training and inference tests since the benchmark was released in 2019.

In February, NVIDIA GPUs delivered leading results for inference, serving up thousands of inferences per second on the most demanding models in the STAC-ML Markets benchmark, a key technology performance gauge for the financial services industry.

A RedHat software engineering team put it succinctly in a blog: GPUs have become the foundation of artificial intelligence.

AI Under the Hood A brief look under the hood shows why GPUs and AI make a powerful pairing.

An AI model, also called a neural network, is essentially a mathematical lasagna, made from layer upon layer of linear algebra equations. Each equation represents the likelihood that one piece of data is related to another.

For their part, GPUs pack thousands of cores, tiny calculators working in parallel to slice through the math that makes up an AI model. This, at a high level, is how AI computing works.

Highly Tuned Tensor Cores Over time, NVIDIA's engineers have tuned GPU cores to the evolving needs of AI models. The latest GPUs include Tensor Cores that are 60x more powerful than the first-generation designs for processing the matrix math neural networks use.

In addition, NVIDIA Hopper Tensor Core GPUs include a Transformer Engine that can automatically adjust to the optimal precision needed to process transformer models, the class of neural networks that spawned generative AI.

Along the way, each GPU generation has packed more memory and optimized techniques to store an entire AI model in a single GPU or set of GPUs.

Models Grow, Systems Expand The complexity of AI models is expanding a whopping 10x a year.

The current state-of-the-art LLM, GPT4, packs more than a trillion parameters, a metric of its mathematical density. That's up from less than 100 million parameters for a popular LLM in 2018.

In a recent talk at Hot Chips, NVIDIA Chief Scientist Bill Dally described how single-GPU performance on AI inference expanded 1,000x in the last decade. GPU systems have kept pace by ganging up on the challenge. They scale up to supercomputers, thanks to their fast NVLink interconnects and NVIDIA Quantum InfiniBand networks.

For example, the DGX GH200, a large-memory AI supercomputer, combines up to 256 NVIDIA GH200 Grace Hopper Superchips into a single data-center-sized GPU with 144 terabytes of shared memory.

Each GH200 superchip is a single server with 72 Arm Neoverse CPU cores and four petaflops of AI performance. A new four-way Grace Hopper systems configuration puts in a single compute node a whopping 288 Arm cores and 16 petaflops of AI performance with up to 2.3 terabytes of high-speed memory.

And NVIDIA H200 Tensor Core GPUs announced in November pack up to 288 gigabytes of the latest HBM3e memory technology.

Software Covers the Waterfront An expanding ocean of GPU software has evolved since 2007 to enable every facet of AI, from deep-tech features to high-level applications.

The NVIDIA AI platform includes hundreds of software libraries and apps. The CUDA programming language and the cuDNN-X library for deep learning provide a base on top of which developers have created software like NVIDIA NeMo, a framework to let users build, customize and run inference on their o
LINK: https://blogs.nvidia.com/blog/why-gpus-are-great-for-ai/...
See more stories from nvidia

North America Stories

03/03/2024

El Conde is First Feature film to use Ed Lachman ASCs EL...

CINELEASE, known for providing the industry s most extensive array of grip, electrical and lighting equipment and full-service studios, opens its doors for a da...

03/03/2024

Elsbeth,' A Police Procedural With a Twinkle In Its Eye,' Premieres on CBS

Elsbeth, featuring a quirky character from The Good Wife and The Good Fight, deb...

03/03/2024

Garth Brooks's New Bar Is the Star of Prime Video Docuseries

Friends in Low Places, a docuseries about Garth Brooks and Trisha Yearwood building a honky-tonk in Nashville, debuts on Prime Video Thursday, March 7. The bar,...

03/03/2024

Brother-and-Sister Anchor Team Reunites as WNDU's Terry McFadden Retires

Terry McFadden, who anchors early evening and late news at WNDU South Bend, Indiana, retires March 1. He joined the station, known as 16 News Now, in 1982....

03/03/2024

Affiliates Seek To Change How They Pay Networks for Programming

Broadcast stations are looking to keep more of their distribution revenue by changing the way they pay networks for programming especially sports to a varia...

03/03/2024

Tegna Extends Local Ratings Deal With Comscore To All Markets

Comscore said that it reached a multi-year agreement to provide local cross-platform measurement for Tegna in all of the 51 markets where Tegna has television s...

03/03/2024

Fubo Adds 173,000 Subs, Cuts Loss in Q4

Fubo, the sports streaming service, said it cut its losses in the fourth quarter as it added subscribers and increased revenue....

03/03/2024

Paramount Names Lee Sears President of International Ad Sales

Paramount Global said it named Lee Sears president of international market advertising sales, a new post at the company....

03/03/2024

Peacock Streams International SVNS' Rugby Tournament March 1-3

Peacock streams the HSBC World Rugby SVNS series March 1-3, as the top nations in the seven-per-side rugby world, including the U.S., compete. The event happens...

03/03/2024

Rick and Michonne Generate Scary Good Numbers for AMC's Latest Walking Dead' Series

The premiere of the latest installment in The Walking Dead Universe The Walkin...

02/03/2024

Nic Dugger Becomes Executive Director of Midsouth Emmy Chapter

Nic Dugger Becomes Executive Director of Midsouth Emmy Chapter Brie Clayton March 1, 2024 0 Comments TNDV Founder Nic Dugger has accepted the role of ...

02/03/2024

Clean Up Your Photo Library at Once: Introducing Centralized Deletion in Peakto

Clean Up Your Photo Library at Once: Introducing Centralized Deletion in Peakto Brie Clayton March 1, 2024 0 Comments CYME unveils a new update for Pe...

02/03/2024

Chaos Unveils its 2024 Roadmap, Including its First AI Tools and an All-New Visual Storytelling Product

Chaos Unveils its 2024 Roadmap, Including its First AI Tools and an All-New Visu...

02/03/2024

Applications Open for Berklee in Puerto Rico 2024

Applications Open for Berklee in Puerto Rico 2024 Registration is open now through April 11. By Colette Greenstein March 1, 2024 Students perform at Berklee...

01/03/2024

Inside the Archives: One Degree of Sundance Institute at the 2024 Oscars

Director Mira Nair and cinematographer Edward Lachman on the set of Mississippi Masala. Photo by Mitch Epstein...

01/03/2024

Release Rundown: What to Watch This March, From Love Lies Bleeding to Shayda

(L-R) Katy O'Brian and Kristen Stewart star in Rose Glass' thriller Love Lies Bleeding, which premiered in the Midnight section at the 2024 Sundance F...

01/03/2024

arkona Continues to Grow Presence in Asia Pacific with Pr...

arkona technologies GmbH, will have a substantial presence during the upcoming METexpo 2024 in Sydney, Australia from 5 - 7 March. The provider of IP core infra...

01/03/2024

Chaos Unveils its 2024 Roadmap Including its First AI Too...

During the first in a series of planned Unboxed events, visualization technology leader Chaos offered the first look at its upcoming 2024 roadmap, including n...

01/03/2024

Audio Quality And High Channel Count Give Prism Sound Dre...

Number Nine Studios, a versatile commercial recording facility in Belgium, has invested in a Prism Sound Dream ADA-128 modular conversion system because its hig...

01/03/2024

Grass Valley Announces Strategic Partnership with First G...

Grass Valley just announced a multi-year, multi-million dollar strategic partnership agreement with First Gulf Company, a pre-eminent systems integration firm b...

01/03/2024

Ateliere Offers New Complimentary Media Supply Chain Anal...

Ateliere Creative Technologies, a leading developer of cloud-native media supply chain solutions, today launched a new complimentary Professional Services offer...

01/03/2024

Presteigne Broadcast Hire Chooses Custom Consoles Module-...

Presteigne Broadcast Hire, one of Britain's leading providers of media-related equipment and services, has chosen Custom Consoles Module-R control room desk...

01/03/2024

Telestream Releases DIVA 9

Telestream , a leading provider of workflow automation, media processing, and distribution solutions, proudly announces the release of DIVA 9, a cutting-edge mi...

01/03/2024

EditShare unveils groundbreaking storage and workflow upg...

EditShare, the technology leader that enables storytellers to create and manage collaborative workflows, is celebrating 20 years of outstanding innovation at th...

01/03/2024

Merapar Opens USA Office and Hires Media and Entertainmen...

Merapar, the media technology and cloud software pioneer, today announced that it has initiated its next growth phase by opening a North America office and hiri...

01/03/2024

Rohde and Schwarz focuses on efficiency sustainability an...

Rohde & Schwarz will showcase the latest developments across its transmitter and media solutions at NAB 2024 (booth SU4039, Las Vegas Convention Center, 14 17...

01/03/2024

Newsbridge Unveils Next-Generation MXT-1-5 AI at the 202...

At the 2024 NAB Show, AI and cloud media company Newsbridge will showcase groundbreaking new features powered by the latest version of its award-winning AI inde...

01/03/2024

VisualOn Transforms Video Streaming Efficiency and Qualit...

2024 NAB Show VisualOn Exhibitor Preview April 14-17 Las Vegas Booth W3225 As video streaming consumption continues to grow, bandwidth usage and delivery ...

01/03/2024

Riedel Announces Strategic Partnership With Hibino Inters...

Riedel Communications today announced a new partnership with Hibino Intersound Corporation, one of Japan's leading distributors in the Pro AV industry, to e...

01/03/2024

Student Spotlight: Shivaraj Natraj

Student Spotlight: Shivaraj Natraj The vocalist, konnakol beatboxer, and percussionist reflects on his musical journey from Oman to India and beyond, and how ...

01/03/2024

Berklee to Present American Master Award to Arturo Sandoval

Berklee to Present American Master Award to Arturo Sandoval Berklee will recognize the Cuban-American legend's legacy and impact on jazz and Latin music. ...

01/03/2024

A Newbie's Look at EditFest LA

A Newbie's Look at EditFest LA Hillary Lewis February 29, 2024 0 Comments Originally published on August 17, 2017 Presented by Blue Collar Post C...

01/03/2024

From Scratch to Scars: Digital Wound FX Tutorial with After Effects & Mocha Pro

From Scratch to Scars: Digital Wound FX Tutorial with After Effects & Mocha Pro Levi Borland February 29, 2024 0 Comments Unlock the secrets of Hollyw...

01/03/2024

Big Night In Wins Emmy

As WRAL-TV gets set to air the 2024 Big Night In for the Arts, the station has received accolades for last year's broadcast! The 2023 Big Night In is the re...

01/03/2024

Paramount Global Cuts Streaming Losses, Posts Higher 4Q Profit

Paramount Global reported higher fourth-quarter earnings as it cut costs and reduced losses at its direct-to-consumer business....

01/03/2024

WCBS New York Changes 6 P.M. Team

Dana Tyler and Dick Brennan, who anchor the 6 p.m. news at WCBS New York, will depart the anchor desk at the end of March. Maurice DuBois and Kristine Johnson, ...

01/03/2024

CBS to Premiere Paramount Plus's Tulsa King' This Summer

Season one of Tulsa King, the Paramount Plus series with Sylvester Stallone, will air on CBS this summer. Season two launches on Paramount Plus in the fall....

01/03/2024

Sinclair Reports 4Q Loss On Diamond Sports Settlement

Sinclair reported a fourth quarter loss as it made a $495 million payment to settle lawsuits surrounding its bankrupt Diamond Sports Group subsidiary....

01/03/2024

Scripps Names Kate OBrian Head of National And Local News

The E.W. Scripps Co. named Kate O'Brian president of news, a new position overseeing the company's local and national news operations....

01/03/2024

Addressable Ads Increase Reach Of Campaigns Of All Sizes, Study Finds

Addressable ad campaigns deliver incremental reach to large and small ad campaigns, a new study found....

01/03/2024

VideoAmp Launches VALID To Push Advanced Currency Adoption

VideoAmp, one of the media measurement companies looking to challenge Nielsen, introduce VALID, its system that uses advanced currencies to help advertisers mor...

01/03/2024

Codeblack Films Signs Up For Pixability's Inclusive Media Initiative

After helping to start the Inclusive Media Initiative, to help brands back diverse YouTube creators with their ad dollars, Pixabiilty sees Hollywood looking for...

01/03/2024

Tegna Earnings Down 65% Amid Absence of Political Ads

The drop in political ad spending in 2023 sharply reduced Tegna's fourth-quarter profits and revenue....

01/03/2024

Paramount Plus's NCIS' Spinoff Reunites Ziva and Tony

NCIS gets another spinoff when Paramount Plus debuts one starring Cote de Pablo and Michael Weatherly as Ziva and Tony. The series does not yet have a name, and...

01/03/2024

No Noobs Here: Top Pro Gamers Bolster Software Quality Assurance Testing

For some NVIDIANs, it's always game day. Our Santa Clara-based software quality assurance team boasts some of the world's top gamers, whose search for ...

01/03/2024

Automakers Electrify Geneva International Motor Show

The Geneva International Motor Show, one of the most important and long-standing global auto exhibitions, opened this week, with the spotlight on several China ...

01/03/2024

NEP Group Back for a Second Helping of MLS Season Pass on Apple TV

NEP Group Back for a Second Helping of MLS Season Pass on Apple TV NEP provides facilities, crew, and a variety of services coast to coast in Year 2 By Jason...

01/03/2024

Full Cast Revealed for TOO MUCH, from Lena Dunham

Back to All News Full Cast Revealed for TOO MUCH, from Lena Dunham Entertainment 01 March 2024 United Kingdom (Great Britain)United States Link copied to c...

01/03/2024

Live at GTC: Hear From Industry Leaders Using AI to Drive Innovation and Agility

Interest in new AI applications reached a fever pitch last year as business leaders began exploring AI pilot programs. This year, they're focused on strateg...