Sony Pixel Power calrec Sony

Scaling to New Heights: NVIDIA MLPerf Training Results Showcase Unprecedented Performance and Elasticity

12/06/2024

The full-stack NVIDIA accelerated computing platform has once again demonstrated exceptional performance in the latest MLPerf Training v4.0 benchmarks.

NVIDIA more than tripled the performance on the large language model (LLM) benchmark, based on GPT-3 175B, compared to the record-setting NVIDIA submission made last year. Using an AI supercomputer featuring 11,616 NVIDIA H100 Tensor Core GPUs connected with NVIDIA Quantum-2 InfiniBand networking, NVIDIA achieved this remarkable feat through larger scale - more than triple that of the 3,584 H100 GPU submission a year ago - and extensive full-stack engineering.

Thanks to the scalability of the NVIDIA AI platform, Eos can now train massive AI models like GPT-3 175B even faster, and this great AI performance translates into significant business opportunities. For example, in NVIDIA's recent earnings call, we described how LLM service providers can turn a single dollar invested into seven dollars in just four years running the Llama 3 70B model on NVIDIA HGX H200 servers. This return assumes an LLM service provider serving Llama 3 70B at $0.60/M tokens, with an HGX H200 server throughput of 24,000 tokens/second.

NVIDIA H200 GPU Supercharges Generative AI and HPC The NVIDIA H200 Tensor GPU builds upon the strength of the Hopper architecture, with 141GB of HBM3 memory and over 40% more memory bandwidth compared to the H100 GPU. Pushing the boundaries of what's possible in AI training, the NVIDIA H200 Tensor Core GPU extended the H100's performance by up to 47% in its MLPerf Training debut.

NVIDIA Software Drives Unmatched Performance Gains Additionally, our submissions using a 512 H100 GPU configuration are now up to 27% faster compared to just one year ago due to numerous optimizations to the NVIDIA software stack. This improvement highlights how continuous software enhancements can significantly boost performance, even with the same hardware.

This work also delivered nearly perfect scaling. As the number of GPUs increased by 3.2x - going from 3,584 H100 GPUs last year to 11,616 H100 GPUs with this submission - so did the delivered performance.

Learn more about these optimizations on the NVIDIA Technical Blog.

Excelling at LLM Fine-Tuning As enterprises seek to customize pretrained large language models, LLM fine-tuning is becoming a key industry workload. MLPerf introduced a new LLM fine-tuning benchmark this round, based on the popular low-rank adaptation (LoRA) technique applied to Meta Llama 2 70B.

The NVIDIA platform excelled at this task, scaling from eight to 1,024 GPUs, with the largest-scale NVIDIA submission completing the benchmark in a record 1.5 minutes.

Accelerating Stable Diffusion and GNN Training NVIDIA also accelerated Stable Diffusion v2 training performance by up to 80% at the same system scales submitted last round. These advances reflect numerous enhancements to the NVIDIA software stack, showcasing how software and hardware improvements go hand-in-hand to deliver top-tier performance.

On the new graph neural network (GNN) test based on R-GAT, the NVIDIA platform with H100 GPUs excelled at both small and large scales. The H200 delivered a 47% boost on single-node GNN training compared to the H100. This showcases the powerful performance and high efficiency of NVIDIA GPUs, which make them ideal for a wide range of AI applications.

Broad Ecosystem Support Reflecting the breadth of the NVIDIA AI ecosystem, 10 NVIDIA partners submitted results, including ASUS, Dell Technologies, Fujitsu, GIGABYTE, Hewlett Packard Enterprise, Lenovo, Oracle, Quanta Cloud Technology, Supermicro and Sustainable Metal Cloud. This broad participation, and their own impressive benchmark results, underscores the widespread adoption and trust in NVIDIA's AI platform across the industry.

MLCommons' ongoing work to bring benchmarking best practices to AI computing is vital. By enabling peer-reviewed comparisons of AI and HPC platforms, and keeping pace with the rapid changes that characterize AI computing, MLCommons provides companies everywhere with crucial data that can help guide important purchasing decisions.

And with the NVIDIA Blackwell platform, next-level AI performance on trillion-parameter generative AI models for both training and inference is coming soon.
LINK: https://blogs.nvidia.com/blog/mlperf-training-benchmarks/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

27/07/2026

AES Educational Foundation Announces 2026/2027 Scholarship and Grant Recipients

The Audio Engineering Society Educational Foundation has named its scholarship and grant recipients for the 2026/2027 academic year. The awards support students...

27/07/2026

ESPN and MEAC Reach Multi-Year Media Rights Agreement Through 2030-31

ESPN and the Mid-Eastern Athletic Conference (MEAC) have signed a multi-year media rights agreement covering basketball championships and select football and ba...

27/07/2026

Yahoo Sports and Cameo Integrate Celebrity Video Bookings into Yahoo Fantasy

Yahoo Sports and Cameo have partnered to allow fantasy football users to book personalized celebrity videos directly within the Yahoo Fantasy app and website. T...

27/07/2026

Eutelsat to Receive $504 Million in FCC Upper C-Band Transition Payments

Eutelsat has welcomed the FCC's order establishing the regulatory framework for the transition of Upper C-band spectrum in the United States. Under the orde...

27/07/2026

ESPN Acquires U.S. and International Rights to International Dance League

ESPN and the International Dance League (IDL) have signed a global rights agreement for the 2026 season. Events will air bi-weekly on ESPN2 at 7 p.m. ET beginni...

27/07/2026

NBCUniversal and YouTube Announce Multi-Year Global Partnership

NBCUniversal and YouTube have signed a multi-year global partnership that includes bundling Peacock with YouTube Premium, extending NBCUniversal's distribut...

27/07/2026

Two Weeks Away: 2026 SVG Venues & Teams Summit at Miamis Kaseya Center To Reunite In-Stadium Production Community

Agenda includes presentations from the Miami Heat, Inter Miami, Florida Panthers...

27/07/2026

Behind The Mic: CBS Sports Bolsters College Football Analyst Team

Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...

27/07/2026

PBR Goes Animated With BS: Bull Stories, Giving Its Bucking Bulls a Voice

Ten one-minute shorts will roll out across CBS, The CW, Paramount+ during the 2026 PBR Teams season PBR is officially moving into scripted animation. The Profe...

27/07/2026

Vochlea announce DubBox

New app turns vocal percussion into polished loops Vochlea, the company behind the voice-to-MIDI instrument Dubler 2, have just announced the launch of a ne...

27/07/2026

Sonora Cinematic launch Pure 12-String for Soundbox

Latest MPE-capable instrument completes Pure Trilogy line-up Sonora Cinematic have just released the third and final instalment in their series of acoustic ...

27/07/2026

fedDSP introduce Opto compressor plug-in

Emulates four varieties of optical compression The latest addition to the fedDSP line-up has just been announced, and packs four distinct optical compressor...

27/07/2026

Drawmer reveal the OC1 & FC1

Optical & FET compressors join 500-series line-up At our recent GearExpo UK show, we heard from Drawmer that they would soon be launching a pair of new 500-...

27/07/2026

T-Mobile Tops Ookla's Mobile Network Rankings

Share Copy link Facebook X Linkedin Bluesky Email...

27/07/2026

Pebble to Feature Virtualization, Cloud-Based Playout Tech at IBC

Share Copy link Facebook X Linkedin Bluesky Email...

27/07/2026

NBCU Telemundo Turns To Chyron PRIME 5 For World Cup Graphics

Share Copy link Facebook X Linkedin Bluesky Email...

27/07/2026

Lightware Adds SNMPv3 Monitoring to Taurus Devices

Lightware announces SNMPv3-based monitoring support for selected Taurus devices, available from firmware v1.20.0. The update enables IT administrators and AV in...

27/07/2026

Mediagenix Showcases the Real-Time Media Enterprise at IB...

Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, will showcase its vision for the Real-Time...

27/07/2026

Official trailer: Gugu Mbatha-Raw stars in Sky Original supernatural thriller Possession

All episodes available Thursday 6 August on Sky and streaming service NOWMonday ...

27/07/2026

Seven paradoxes shaping the next era of media production - Episode 4

Why Confidence MattersIn this series, we explore the technologies, architectures and operational realities shaping modern media operations. Along the way, we ex...

27/07/2026

Apple brings AppleCare One to the UK, streamlining coverage - UPDATE - Posted on 27 July 2026

Apple today announced AppleCare One - a new way for customers to protect multipl...

27/07/2026

944,000 VIEWERS WATCH RT'S COVERAGE OF MAYO'S HISTORIC WIN, AFTER A 75 YEAR WAIT TO LIFT THE SAM MAGUIRE

Yesterday witnessed truly memorable scenes in Croke Park, bringing the curtain d...

27/07/2026

SES Statement On FCC Upper C-band Auction Decision

Luxembourg, July 27, 2026 - SES, a leading space solutions company, issued the following statement following the U.S. Federal Communications Commission approval...

27/07/2026

Industry Leaders Unite in Open Secure AI Alliance for AI Safety and Security

Open source software is a critical pillar of the global economy. It underpins cloud computing, financial services, manufacturing, telecommunications, government...

26/07/2026

Audio Vivid: A New Destination for Immersive Audio

What is Audio Vivid? A New Destination for Immersive Audio Over the past several years, immersive audio has evolved from a niche production consideration into a...

26/07/2026

NVIDIA Harnesses Vera CPU to Speed Up Design of Next-Generation CPUs and GPUs

The complexity of modern chip design continues to grow as engineering teams work to develop increasingly sophisticated CPUs, GPUs and AI systems. To help meet t...

25/07/2026

Vochlea introduce DubBox

New app turns vocal percussion into polished loops Vochlea, the company behind the voice-to-MIDI instrument Dubler 2, have just announced the launch of a ne...

25/07/2026

Study: T-Mobile Remains Best Mobile and Fastest 5G Network

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Mark Bridges to Lead CableLabs

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Total YouTube Views for All FIFA World Cup Content Top 200 Billion

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Pebble To Address Future Of Media Operations At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Paramount Agrees to Pause Warner Bros. Discovery Deal

Share Copy link Facebook X Linkedin Bluesky Email...

25/07/2026

Export FCP7 to XML Without Needing FCP7 Installed

Export FCP7 to XML Without Needing FCP7 Installed Dave Rice July 24, 2026 0 Comments I've created an open source tool that allows you to export FC...

25/07/2026

Documentary Sell Your House Edited with DaVinci Resolve Studio

Documentary Sell Your House Edited with DaVinci Resolve Studio Brie Clayton July 24, 2026 0 Comments Filmmakers post feature documentary from opposite...

25/07/2026

Time to Act: Bringing a New Opera to Life

Time to Act: Bringing a New Opera to Life Boston Conservatory at Berklee vocal arts students played a vital role in shaping the opera's creative developme...

25/07/2026

Beyond the Score

Beyond the Score For Dean of Music Kevin Haden, the boundless possibilities of classical music begin with process. July 24, 2026 By Kevin Haden Michelle ...

25/07/2026

Reading the Room

Reading the Room As music director of two major orchestras, conductor Jonathon Heyward aims to make the concert hall a space for everyone. July 24, 2026 By ...

25/07/2026

First look revealed for YAGA, a seductive mystery thriller coming to Sky and NOW this October

Saturday 25 July 2026 First look revealed for YAGA, a seductive mystery thrille...

24/07/2026

ESPN Adds New Production Features for NFL FLAG Championships

Referee hat cameras, player/coach mics, and virtual graphics will be deployed at the three-day event for the first time The NFL FLAG Championships return to ES...

24/07/2026

Qvest, Ateme and Scaleway Deliver Sovereign OTT Platforms for European Broadcasters

Qvest, Ateme, and Scaleway have partnered to offer broadcasters, media organizat...

24/07/2026

LTN Reaches 3,000 Satellite-to-IP Migrations as FCC Votes on C-Band Auction

LTN has completed satellite-to-IP migrations for more than 3,000 broadcaster, MVPD (Multi-Channel Video Programming Distributor), head-end, and content owner si...

24/07/2026

Chyron Releases Virtual Placement 8.1

Chyron has released Virtual Placement 8.1, an update to its live sports virtual graphics platform. The release adds workflow and graphics changes across chroma ...

24/07/2026

Riedel Deploys RefCam and Easy5G at International Soccer Match

Riedel's RefCam Live was used for the first time at an international soccer match during a friendly between the German men's national team and Ghana in ...

24/07/2026

USSI Global Offers Services for Next Phase of C-Band Spectrum Transition

USSI Global said it is prepared to help broadcasters, satellite network operators, and media organizations manage the next phase of C-band spectrum clearing, fo...

24/07/2026

Fubo Launches The Athletic Video Hub on CTV

FuboTV and The Athletic have launched The Athletic Video Hub on Fubo, bringing The Athletic's sports video content to connected TV (CTV) for the first time....