Sony Pixel Power calrec Sony

Scaling to New Heights: NVIDIA MLPerf Training Results Showcase Unprecedented Performance and Elasticity

12/06/2024

The full-stack NVIDIA accelerated computing platform has once again demonstrated exceptional performance in the latest MLPerf Training v4.0 benchmarks.

NVIDIA more than tripled the performance on the large language model (LLM) benchmark, based on GPT-3 175B, compared to the record-setting NVIDIA submission made last year. Using an AI supercomputer featuring 11,616 NVIDIA H100 Tensor Core GPUs connected with NVIDIA Quantum-2 InfiniBand networking, NVIDIA achieved this remarkable feat through larger scale - more than triple that of the 3,584 H100 GPU submission a year ago - and extensive full-stack engineering.

Thanks to the scalability of the NVIDIA AI platform, Eos can now train massive AI models like GPT-3 175B even faster, and this great AI performance translates into significant business opportunities. For example, in NVIDIA's recent earnings call, we described how LLM service providers can turn a single dollar invested into seven dollars in just four years running the Llama 3 70B model on NVIDIA HGX H200 servers. This return assumes an LLM service provider serving Llama 3 70B at $0.60/M tokens, with an HGX H200 server throughput of 24,000 tokens/second.

NVIDIA H200 GPU Supercharges Generative AI and HPC The NVIDIA H200 Tensor GPU builds upon the strength of the Hopper architecture, with 141GB of HBM3 memory and over 40% more memory bandwidth compared to the H100 GPU. Pushing the boundaries of what's possible in AI training, the NVIDIA H200 Tensor Core GPU extended the H100's performance by up to 47% in its MLPerf Training debut.

NVIDIA Software Drives Unmatched Performance Gains Additionally, our submissions using a 512 H100 GPU configuration are now up to 27% faster compared to just one year ago due to numerous optimizations to the NVIDIA software stack. This improvement highlights how continuous software enhancements can significantly boost performance, even with the same hardware.

This work also delivered nearly perfect scaling. As the number of GPUs increased by 3.2x - going from 3,584 H100 GPUs last year to 11,616 H100 GPUs with this submission - so did the delivered performance.

Learn more about these optimizations on the NVIDIA Technical Blog.

Excelling at LLM Fine-Tuning As enterprises seek to customize pretrained large language models, LLM fine-tuning is becoming a key industry workload. MLPerf introduced a new LLM fine-tuning benchmark this round, based on the popular low-rank adaptation (LoRA) technique applied to Meta Llama 2 70B.

The NVIDIA platform excelled at this task, scaling from eight to 1,024 GPUs, with the largest-scale NVIDIA submission completing the benchmark in a record 1.5 minutes.

Accelerating Stable Diffusion and GNN Training NVIDIA also accelerated Stable Diffusion v2 training performance by up to 80% at the same system scales submitted last round. These advances reflect numerous enhancements to the NVIDIA software stack, showcasing how software and hardware improvements go hand-in-hand to deliver top-tier performance.

On the new graph neural network (GNN) test based on R-GAT, the NVIDIA platform with H100 GPUs excelled at both small and large scales. The H200 delivered a 47% boost on single-node GNN training compared to the H100. This showcases the powerful performance and high efficiency of NVIDIA GPUs, which make them ideal for a wide range of AI applications.

Broad Ecosystem Support Reflecting the breadth of the NVIDIA AI ecosystem, 10 NVIDIA partners submitted results, including ASUS, Dell Technologies, Fujitsu, GIGABYTE, Hewlett Packard Enterprise, Lenovo, Oracle, Quanta Cloud Technology, Supermicro and Sustainable Metal Cloud. This broad participation, and their own impressive benchmark results, underscores the widespread adoption and trust in NVIDIA's AI platform across the industry.

MLCommons' ongoing work to bring benchmarking best practices to AI computing is vital. By enabling peer-reviewed comparisons of AI and HPC platforms, and keeping pace with the rapid changes that characterize AI computing, MLCommons provides companies everywhere with crucial data that can help guide important purchasing decisions.

And with the NVIDIA Blackwell platform, next-level AI performance on trillion-parameter generative AI models for both training and inference is coming soon.
LINK: https://blogs.nvidia.com/blog/mlperf-training-benchmarks/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

10/06/2026

Microphone Maker Audix Adds Eric Reese as VP

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Kiswe Expands Partnership With ONE Championship To LaunchGlobal Subscription Platform

Kiswe announces an expanded long-term partnership with ONE Championship (ONE), t...

09/06/2026

SiriusXM to Carry All 104 FIFA World Cup 2026 Matches via FOX Sports Commentary

SiriusXM will broadcast FOX Sports' English-language commentary for all 104 FIFA World Cup 2026 matches from June 11 through July 19, available to subscribe...

09/06/2026

EVS Broadcast Equipment Rebrands as EVS

EVS has announced it is changing its corporate name from EVS Broadcast Equipment to EVS, reflecting the company's expanded portfolio beyond broadcast equipm...

09/06/2026

FOX and NFL Announce Multi-Year Agreement for NFL Coverage in Mexico Starting in 2026

Fox Corporation and the NFL have announced a multi-year agreement to bring NFL c...

09/06/2026

FOX Sports and ReachTV to Carry All 104 FIFA World Cup 2026 Matches Across U.S. Airports

FOX Sports and ReachTV, an airport media network, have announced an agreement to...

09/06/2026

Cosm Atlanta Opens with Private Preview, Public Opening Set for June 10

Cosm Atlanta, a 70,000-square-foot, three-level immersive entertainment venue located within Centennial Yards adjacent to State Farm Arena and Mercedes-Benz Sta...

09/06/2026

NESN Wins Five New England Emmy Awards

NESN captured five awards at the 2026 Boston/New England Emmy Awards, including four program honors and one individual award. These awards reflect the passion...

09/06/2026

Ateme Referenced by Apple at WWDC26 for Apple Immersive Video Workflow Support

Ateme has announced that its support for Apple Immersive Video workflows was referenced by Apple during its 2026 Worldwide Developers Conference (WWDC26). Atem...

09/06/2026

Axel Springer Deploys Bitmovin Player Web X for WELT Audio News Experience

Bitmovin has announced that Axel Springer SE has deployed Player Web X, Bitmovin's web video player, to power audio readouts of news articles and an audio-o...

09/06/2026

Grass Valley and Lawo Announce Technology Collaboration on AMPP and HOME Integration

Grass Valley and Lawo have announced a technology collaboration to validate orch...

09/06/2026

LiveU Deploys Remote VAR for Oceania Football Confederation, First in Pacific Islands

LiveU has announced a deployment with the Oceania Football Confederation (OFC) t...

09/06/2026

Globecast Launches Content Exchange Platform Powered by Oracle Cloud Infrastructure

Globecast has announced the launch of its Content Exchange platform, powered by ...

09/06/2026

OT7 Championship Weekend to Air on NBC and Peacock for First Time

NBC Sports will present live coverage of Overtime's OT7 football league Championship Weekend from Sullivan Field at Loyola Marymount University in Los Angel...

09/06/2026

UFC Freedom 250: Inside the One-in-a-Lifetime Production Behind the White House Fight Card

We're in three different locations, three different production teams. Coveri...

09/06/2026

Murray States Brandon Banks on Firing Up Racers Basketball Fans Through Creative Video

The intro video for Men's Basketball won Outstanding In-Venue Video in the C...

09/06/2026

SVG Sit-Down, Part 2: FIFA's Oscar Sanchez, HBS's Paul King Dive Deep Into IBC Operations, Commentary, and Ref Cam

Today is match day minus two for FIFA and HBS. On Thursday, there will be two ma...

09/06/2026

Designing the Modern Scorebug: How Broadcast Graphics Teams Are Rethinking the Most Important Element On Screen

Since their debut, the clock-and-score graphic has drawn the affinity and ire of...

09/06/2026

Watch LE SSERAFIM's PURE FLOWERS LIVE Performance Videos and Fan Q&A, Only on Spotify

Last month, Spotify hosted PURE FLOWERS LIVE, a special event celebrating the re...

09/06/2026

Watch RADAR UK Artist Skye Newman Perform Live in London for Her SE9' Album Launch

Last month, RADAR U.K. artist Skye Newman took the stage in East London for a sp...

09/06/2026

Announcement from Wayne Jones Audio

Company to cease operating on 30 June 2026 Australian loudspeaker and amplifier manufacturer Wayne Jones Audio have announced that after much consideration,...

09/06/2026

bigBASS from fedDSP

Increases low-end weight and character The latest plug-in release from Sheffield-based fedDSP aims to offer an all-in-one solution for users in search of mo...

09/06/2026

Fender Studio Pro 8.1 arrives

Introduces AI Studio Assistant, Moises Studio integration & more Fender Studio have just announced the launch a significant update that brings an array of n...

09/06/2026

Call for Sediba Scriptwriting Training Programme Bloemfontein, Free State

The National Film and Video Foundation (NFVF) is a statutory body mandated to spearhead the equitable growth and development of the South African film and video...

09/06/2026

BBC Casts Edward Bluemel As Poirot In Hercule,' As BritBox Boards Agatha Christie Remake In U.S.

The BBC has found its Hercule Poirot. After Deadline revealed last month that t...

09/06/2026

Nielsen launches Four-Screen Ad Deduplication measurement on YouTube campaigns in Italy

Media buyers and sellers can now compare YouTube reach from computer, mobile, an...

09/06/2026

Andrew Beaudet Joins Brompton as Head of Sales, Americas

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Mediaset Italia Deploys Mediagenix Platform To Modernize Scheduling

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Mediaset Italia Modernizes Content Supply Chain and Chann...

A project aimed at modernizing scheduling operations across Mediaset's channel portfolio Mediagenix, a global leader in smart content solutions to profitab...

09/06/2026

ACT Entertainment Debuts tvONE 4RU CALICO PRO at InfoComm...

tvONE, an ACT Entertainment brand, debuts its most powerful video processor ever built: the 4RU CALICO PRO (C7-PRO-4200), at InfoComm 2026 (Booth N6813). Engine...

09/06/2026

Composer and Conductor Eric Whitacre to Receive Honorary Doctorate at Berklee Valencia's 2026 Commencement

Composer and Conductor Eric Whitacre to Receive Honorary Doctorate at Berklee Va...

09/06/2026

Comcast Advertising Adds Purchasing Data from Affinity Solutions

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Midwich to Distribute Miri Technologies in North America

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Nielsen: North American Soccer Fans Jump to More Than 136 Million

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Grass Valley and Lawo Collaborate to Advance Open Dynamic...

Technology collaboration to validate AMPP and Lawo HOME orchestration integration, showcasing Dynamic Media Facility principles in practice. Grass Valley and ...

09/06/2026

Bitmovin Player Web X Powers Axel Springer New Audio News...

Bitmovin, a leading provider of video streaming solutions, today announced that Axel Springer SE, international media and technology company, has deployed Playe...

09/06/2026

LiveU Delivers First Remote VAR Deployment Across the Pac...

LiveU, the global leader in live IP-video solutions, today announced a landmark deployment with the Oceania Football Confederation (OFC) that has brought Video ...

09/06/2026

Sam Craig Joins Grass Valley as VP, Global Pre-Sales

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Arkansas TV Will Continue to Air PBS Programming

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Glensound to Show DARK22M Network Audio Interface at InfoComm 2026

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

EVS Rebrands, Trims Its Name

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

FCC Announces Tentative Agenda for June 25 Open Meeting

Share Copy link Facebook X Linkedin Bluesky Email...

09/06/2026

Two Alumni Win Tony Awards

Two Alumni Win Tony Awards Mike Morris and Cedric Leiba Jr. won awards for Best Orchestrations and Best Play, respectively. June 8, 2026 By Tori Donahue ...

09/06/2026

NVIDIA Confidential Computing to Help Expand Apple's Private Cloud Compute

NVIDIA GPUs with Confidential Computing are now used for confidential inference in Apple's Private Cloud Compute (PCC), as it expands beyond Apple's dat...

09/06/2026

X-Rite Pantone Launches Offset360 to Modernize Color Control Across Existing Presses

X-Rite Pantone Launches Offset360 to Modernize Color Control Across Existing Pre...

09/06/2026

VEON Appoints Serkan Ozturk as Chief of Staff & Strategy Officer

09 Jun 2026 VEON Appoints Serkan Ozturk as Chief of Staff & Strategy Officer Dubai and New York, June 9, 2026 - VEON Ltd. (NASDAQ: VEON), a global digital oper...

09/06/2026

Katie Price: Nothing to Hide, a candid and unfiltered account of three decades in the spotlight, coming to Sky and NOW on 8 July

Tuesday 9 June 2026 Katie Price: Nothing to Hide, a candid and unfiltered accou...