Sony Pixel Power calrec Sony

Scaling to New Heights: NVIDIA MLPerf Training Results Showcase Unprecedented Performance and Elasticity

12/06/2024

The full-stack NVIDIA accelerated computing platform has once again demonstrated exceptional performance in the latest MLPerf Training v4.0 benchmarks.

NVIDIA more than tripled the performance on the large language model (LLM) benchmark, based on GPT-3 175B, compared to the record-setting NVIDIA submission made last year. Using an AI supercomputer featuring 11,616 NVIDIA H100 Tensor Core GPUs connected with NVIDIA Quantum-2 InfiniBand networking, NVIDIA achieved this remarkable feat through larger scale - more than triple that of the 3,584 H100 GPU submission a year ago - and extensive full-stack engineering.

Thanks to the scalability of the NVIDIA AI platform, Eos can now train massive AI models like GPT-3 175B even faster, and this great AI performance translates into significant business opportunities. For example, in NVIDIA's recent earnings call, we described how LLM service providers can turn a single dollar invested into seven dollars in just four years running the Llama 3 70B model on NVIDIA HGX H200 servers. This return assumes an LLM service provider serving Llama 3 70B at $0.60/M tokens, with an HGX H200 server throughput of 24,000 tokens/second.

NVIDIA H200 GPU Supercharges Generative AI and HPC The NVIDIA H200 Tensor GPU builds upon the strength of the Hopper architecture, with 141GB of HBM3 memory and over 40% more memory bandwidth compared to the H100 GPU. Pushing the boundaries of what's possible in AI training, the NVIDIA H200 Tensor Core GPU extended the H100's performance by up to 47% in its MLPerf Training debut.

NVIDIA Software Drives Unmatched Performance Gains Additionally, our submissions using a 512 H100 GPU configuration are now up to 27% faster compared to just one year ago due to numerous optimizations to the NVIDIA software stack. This improvement highlights how continuous software enhancements can significantly boost performance, even with the same hardware.

This work also delivered nearly perfect scaling. As the number of GPUs increased by 3.2x - going from 3,584 H100 GPUs last year to 11,616 H100 GPUs with this submission - so did the delivered performance.

Learn more about these optimizations on the NVIDIA Technical Blog.

Excelling at LLM Fine-Tuning As enterprises seek to customize pretrained large language models, LLM fine-tuning is becoming a key industry workload. MLPerf introduced a new LLM fine-tuning benchmark this round, based on the popular low-rank adaptation (LoRA) technique applied to Meta Llama 2 70B.

The NVIDIA platform excelled at this task, scaling from eight to 1,024 GPUs, with the largest-scale NVIDIA submission completing the benchmark in a record 1.5 minutes.

Accelerating Stable Diffusion and GNN Training NVIDIA also accelerated Stable Diffusion v2 training performance by up to 80% at the same system scales submitted last round. These advances reflect numerous enhancements to the NVIDIA software stack, showcasing how software and hardware improvements go hand-in-hand to deliver top-tier performance.

On the new graph neural network (GNN) test based on R-GAT, the NVIDIA platform with H100 GPUs excelled at both small and large scales. The H200 delivered a 47% boost on single-node GNN training compared to the H100. This showcases the powerful performance and high efficiency of NVIDIA GPUs, which make them ideal for a wide range of AI applications.

Broad Ecosystem Support Reflecting the breadth of the NVIDIA AI ecosystem, 10 NVIDIA partners submitted results, including ASUS, Dell Technologies, Fujitsu, GIGABYTE, Hewlett Packard Enterprise, Lenovo, Oracle, Quanta Cloud Technology, Supermicro and Sustainable Metal Cloud. This broad participation, and their own impressive benchmark results, underscores the widespread adoption and trust in NVIDIA's AI platform across the industry.

MLCommons' ongoing work to bring benchmarking best practices to AI computing is vital. By enabling peer-reviewed comparisons of AI and HPC platforms, and keeping pace with the rapid changes that characterize AI computing, MLCommons provides companies everywhere with crucial data that can help guide important purchasing decisions.

And with the NVIDIA Blackwell platform, next-level AI performance on trillion-parameter generative AI models for both training and inference is coming soon.
LINK: https://blogs.nvidia.com/blog/mlperf-training-benchmarks/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

10/04/2026

The Invisible OPEX Killer: Is Your Server Room Dragging You Down?

The Invisible OPEX Killer: Is Your Server Room Dragging You Down? In the broadcast world, we talk a lot about uptime. We talk about talent retention, latency...

10/04/2026

NAB 2026: Imagine Communications to Showcase Expanded Multiviewer Portfolio

Imagine Communications will showcase its multiviewer portfolio at NAB Show 2026 (April 19-22, Booth N1328, Las Vegas Convention Center), including Prismon and t...

10/04/2026

NAB 2026: Chyron Releases PRIME VSAR 2.3 with Updated Unreal Engine Integration

Chyron has released PRIME VSAR 2.3, an update to its virtual set and augmented reality solution for broadcast. The release adds compatibility with Unreal Engine...

10/04/2026

NAB 2026: Techex to Showcase New tx darwin Capabilities

Techex will exhibit at NAB Show 2026 (Booth W2267, April 19-23, Las Vegas Convention Center), demonstrating new tx darwin features including consumer multiview,...

10/04/2026

NAB 2026: NDI to Showcase Ecosystem and NDI 6.3

NDI will exhibit at NAB Show 2026, demonstrating its IP video ecosystem through live partner integrations, NDI 6.3 features, AI metadata workflows, and creator ...

10/04/2026

FOR-A Acquires Tamura Corporations Information Equipment Business

FOR-A has announced the acquisition of all shares of Tamu Radiance Corporation, a new company spun off from the Information Equipment Business of Tamura Corpora...

10/04/2026

NAB 2026: InSync Technology to Unveil New Video Processing and Frame Rate Conversion Products

InSync Technology will showcase new and updated video conversion products at NAB...

10/04/2026

TNT Sports and DAZN Announce Monthly Boxing Event Series in the United States

TNT Sports and DAZN have announced a partnership to air monthly boxing events in the United States under the brand The Fight. The series will be promoted in p...

10/04/2026

Panasonic Introduces SQ3 Series 4K LCD Displays for Professional Environments

Panasonic Projector and Display has announced the SQ3 Series of 4K LCD displays as part of its MEVIX professional display portfolio. All sizes will be available...

10/04/2026

Amagi Adds Agentic Capabilities to Its Media Operations Platform

Amagi has announced the addition of Agentic Media Operations to its Amagi NOW platform, integrating AI reasoning agents across its media supply chain workflows ...

10/04/2026

LTN Announces Network Enhancements Ahead of C-Band Spectrum Auction

LTN has announced enhancements to its global IP video network targeting broadcasters transitioning from satellite distribution. The updates come ahead of US fed...

10/04/2026

Daktronics Installs New LED Displays at Yankee Stadium

Daktronics has installed new LED displays at Yankee Stadium, upgrading the main centerfield board, two flanking boards, and two ribbon displays spanning the 200...

10/04/2026

NAB 2026: Harmonic Announces AI and Cloud Updates to Hybrid Streaming Solution

Harmonic has announced updates to its hybrid streaming solution, including Model Context Protocol (MCP) connectivity for AI applications, cloud-native deploymen...

10/04/2026

NAB 2026: MultiDyne to Debut FiberSaver-10G and VF-9100

MultiDyne Video and Fiber Optic Systems will introduce two new fiber transport products at NAB Show 2026 (Booth C4425, April 19-22): the FiberSaver-10G waveleng...

10/04/2026

NAB 2026: Telos Alliance and ip-studio to Demonstrate STUDIO ZERO

Telos Alliance and ip-studio will demonstrate STUDIO ZERO, a cloud-hosted virtual studio, at NAB Show 2026. First introduced at NAB Show 2023, STUDIO ZERO integ...

10/04/2026

Pixotope and d&b Solutions Announce Strategic Partnership for XR and Virtual Studio Production

d&b solutions, a London-based audio-visual, lighting, and media integration grou...

10/04/2026

ARRI and SmallHD Announce Lens Data Monitor Overlay License for Hi-5 and Hi-5 SX

ARRI and SmallHD have announced a new expansion license for ARRI's Hi-5 and Hi-5 SX hand units that displays lens data overlays on supported SmallHD monitor...

10/04/2026

Roku to Stream Exclusive Savannah Bananas Game Package on Roku Sports Channel

Roku and the Banana Ball Championship League (BBCL) have announced an exclusive streaming partnership to bring five BBCL games to the Roku Sports Channel in 202...

10/04/2026

Ratings Roundup: More Than 18 Million Fans Tune Into 2026 NCAA Mens March Madness on TNT and CBS Sports

Ratings Roundup is a rundown of recent rating news and is derived from press rel...

10/04/2026

No Other Land, Mr. Nobody Against Putin,and More Sundance Institute-Supported Films Nominated for Peabody Awards

The Peabody Awards don't just recognize great storytelling, they spotlight t...

10/04/2026

Fans Crown Winners at the First Spotify Podcast Awards in France

After launching the Spotify Podcast Awards in Mexico last year, we brought the fan-voted celebration to Paris this week for its first edition in France. Hosted ...

10/04/2026

Yamaha launch the DXR/DXS & CXR/CXS Mk3

Powered and unpowered live PA ranges upgraded Yamaha have just refreshed four of their hugely popular PA speaker ranges, delivering significant improvements...

10/04/2026

UJAM open Gorilla Engine to third-party developers

Underlying plug-in & VI technology now available to others UJAM's latest announcement sees the company open up' Gorilla Engine, the development pla...

10/04/2026

2026 NAB Show Exhibitor Insight: Bitcentral

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Bitcentral To Feature Connected Media Workflows At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Bitcentral to Showcase Connected Media Workflows and Inte...

NEWPORT BEACH, Calif., April 10, 2026 Bitcentral, a leading provider of professional media solutions for broadcast and digital video, will showcase its latest...

10/04/2026

Ikegami to Introduce Expanded Range of Broadcast Production Solutions at NAB 2026

Ikegami to Introduce Expanded Range of Broadcast Production Solutions at NAB 202...

10/04/2026

AJA Debuts SMPTE ST 2110 and openGear Solutions Ahead of NAB 2026

AJA Debuts SMPTE ST 2110 and openGear Solutions Ahead of NAB 2026 Brie Clayton April 10, 2026 0 Comments New gear and updates address evolving hybrid ...

10/04/2026

Portland Fire+ Streaming Platform Launches

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Tod Musgrave Joins Proton as U.S. Sales & Marketing Director

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Proton Expands Minicam Portfolio With Proton Pro At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

FCC To Vote on Changes to Audible Crawl Rule

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Frequency Launches AI Platform for Streaming Television a...

Frequency, the engine behind the worlds leading streaming television channels, today launched its AI platform for Frequency Studio, powering the entire channel ...

10/04/2026

UKTV Highlights: Saturday April 25th - Friday May 1st 2026

What can I watch on UKTV and stream on U this week? This week on UKTV and the free streaming service U, viewers can watch a range of new and returning programm...

10/04/2026

Jnger Audio Joins EBU ADM Integration Group as Founding Member to Help Advance ADM/S-ADM Integration

J nger Audio Joins EBU ADM Integration Group as Founding Member to Help Advance...

10/04/2026

Multi-BAFTA-winning Chernobyl makes its free-to-air debut, marking 40 years since the disaster

Five-part Sky Original drama airs nightly on Sky Mix and Sky Atlantic from 20 Ap...

09/04/2026

Yospace surpasses 10 billion ads stitched in a single month, as ad-supported streaming surges

Staines-upon-Thames, UK, 09, April, 2026 - Yospace, the trusted leader in Dynam...

09/04/2026

just:play pro 2026 and just:live pro 2026 Sneak Preview News for NAB 2026

just:play pro 2026 and just:live pro 2026 Sneak Preview News for NAB 2026 More Details:At NAB 2026, ToolsOnAir will showcase just:play pro 2026 and just:live p...

09/04/2026

just:in mac pro 2026 - The Next Level of Professional Recording on macOS at NAB 2026

just:in mac pro 2026 - The Next Level of Professional Recording on macOS at NAB ...

09/04/2026

NAB 2026: Zixi to Demonstrate Live Video Workflows and Satellite Replacement

Zixi will demonstrate IP-based live video workflow solutions at NAB Show 2026 (Booth W2057). The industry is moving quickly toward IP-based distribution as br...

09/04/2026

Deloitte Research: Women's Elite Sports Revenues Expected to Reach at Least $3 Billion in 2026

Global women's elite sports revenues are expected to reach at least $3 billi...

09/04/2026

Monitor Engineer Gavin Tempany Mixes Kylie Minogue's Tension Tour on Solid State Logic L550 Plus

Monitor engineer Gavin Tempany mixed Kylie Minogue s Tension Tour on a Solid Sta...

09/04/2026

NAB 2026: KOKUSAI DENKI Electric America to Debut New 4K Camera and Remote Control Panel

KOKUSAI DENKI Electric America will exhibit at NAB Show 2026 (Booth C5507), debu...

09/04/2026

NBC Sports Reviews Innovations and Milestones from Its 2025-26 NBA Regular Season

With the 2025-26 NBA regular season concluded and the playoffs beginning next we...