Sony Pixel Power calrec Sony

Scaling to New Heights: NVIDIA MLPerf Training Results Showcase Unprecedented Performance and Elasticity

12/06/2024

The full-stack NVIDIA accelerated computing platform has once again demonstrated exceptional performance in the latest MLPerf Training v4.0 benchmarks.

NVIDIA more than tripled the performance on the large language model (LLM) benchmark, based on GPT-3 175B, compared to the record-setting NVIDIA submission made last year. Using an AI supercomputer featuring 11,616 NVIDIA H100 Tensor Core GPUs connected with NVIDIA Quantum-2 InfiniBand networking, NVIDIA achieved this remarkable feat through larger scale - more than triple that of the 3,584 H100 GPU submission a year ago - and extensive full-stack engineering.

Thanks to the scalability of the NVIDIA AI platform, Eos can now train massive AI models like GPT-3 175B even faster, and this great AI performance translates into significant business opportunities. For example, in NVIDIA's recent earnings call, we described how LLM service providers can turn a single dollar invested into seven dollars in just four years running the Llama 3 70B model on NVIDIA HGX H200 servers. This return assumes an LLM service provider serving Llama 3 70B at $0.60/M tokens, with an HGX H200 server throughput of 24,000 tokens/second.

NVIDIA H200 GPU Supercharges Generative AI and HPC The NVIDIA H200 Tensor GPU builds upon the strength of the Hopper architecture, with 141GB of HBM3 memory and over 40% more memory bandwidth compared to the H100 GPU. Pushing the boundaries of what's possible in AI training, the NVIDIA H200 Tensor Core GPU extended the H100's performance by up to 47% in its MLPerf Training debut.

NVIDIA Software Drives Unmatched Performance Gains Additionally, our submissions using a 512 H100 GPU configuration are now up to 27% faster compared to just one year ago due to numerous optimizations to the NVIDIA software stack. This improvement highlights how continuous software enhancements can significantly boost performance, even with the same hardware.

This work also delivered nearly perfect scaling. As the number of GPUs increased by 3.2x - going from 3,584 H100 GPUs last year to 11,616 H100 GPUs with this submission - so did the delivered performance.

Learn more about these optimizations on the NVIDIA Technical Blog.

Excelling at LLM Fine-Tuning As enterprises seek to customize pretrained large language models, LLM fine-tuning is becoming a key industry workload. MLPerf introduced a new LLM fine-tuning benchmark this round, based on the popular low-rank adaptation (LoRA) technique applied to Meta Llama 2 70B.

The NVIDIA platform excelled at this task, scaling from eight to 1,024 GPUs, with the largest-scale NVIDIA submission completing the benchmark in a record 1.5 minutes.

Accelerating Stable Diffusion and GNN Training NVIDIA also accelerated Stable Diffusion v2 training performance by up to 80% at the same system scales submitted last round. These advances reflect numerous enhancements to the NVIDIA software stack, showcasing how software and hardware improvements go hand-in-hand to deliver top-tier performance.

On the new graph neural network (GNN) test based on R-GAT, the NVIDIA platform with H100 GPUs excelled at both small and large scales. The H200 delivered a 47% boost on single-node GNN training compared to the H100. This showcases the powerful performance and high efficiency of NVIDIA GPUs, which make them ideal for a wide range of AI applications.

Broad Ecosystem Support Reflecting the breadth of the NVIDIA AI ecosystem, 10 NVIDIA partners submitted results, including ASUS, Dell Technologies, Fujitsu, GIGABYTE, Hewlett Packard Enterprise, Lenovo, Oracle, Quanta Cloud Technology, Supermicro and Sustainable Metal Cloud. This broad participation, and their own impressive benchmark results, underscores the widespread adoption and trust in NVIDIA's AI platform across the industry.

MLCommons' ongoing work to bring benchmarking best practices to AI computing is vital. By enabling peer-reviewed comparisons of AI and HPC platforms, and keeping pace with the rapid changes that characterize AI computing, MLCommons provides companies everywhere with crucial data that can help guide important purchasing decisions.

And with the NVIDIA Blackwell platform, next-level AI performance on trillion-parameter generative AI models for both training and inference is coming soon.
LINK: https://blogs.nvidia.com/blog/mlperf-training-benchmarks/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

27/03/2026

FCC Opposes Emergency Motion to Stay Nexstar/Tegna Merger

Share Copy link Facebook X Linkedin Bluesky Email...

27/03/2026

NAB Pans Netflix's Coverage of MLB Opening Day

Share Copy link Facebook X Linkedin Bluesky Email...

27/03/2026

2026 NAB Show Exhibitor Insight: Netgear

Share Copy link Facebook X Linkedin Bluesky Email...

27/03/2026

Marshall Electronics Showcases New Feature-Rich CV320 and CV520 IP and 3G-SDI POV Cameras at NAB 2026

Marshall Electronics Showcases New Feature-Rich CV320 and CV520 IP and 3G-SDI PO...

27/03/2026

Sony Electronics Inc. Elevates Professional Video Workflows with Powerful Updates to Monitor & Control and Catalyst Tools

Sony Electronics Inc. Elevates Professional Video Workflows with Powerful Update...

27/03/2026

GatesAir Extends AirWatch365 Managed Service with Edge Gateway Site Appliance

GatesAir Extends AirWatch365 Managed Service with Edge Gateway Site Appliance Brie Clayton March 27, 2026 0 Comments NAB marks global launch of servic...

27/03/2026

GatesAir Expands AirWatch365 Managed Service

Share Copy link Facebook X Linkedin Bluesky Email...

27/03/2026

Dejero, Eutelsat, Others to Offer 'Field to Air' Demo at NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

27/03/2026

Global NextGen TV Takes Center Stage at 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

27/03/2026

USSI Global Opens Media Transport Solutions Lab

Share Copy link Facebook X Linkedin Bluesky Email...

27/03/2026

Net Insight Elevates Nimbra Edge to Govern Live Media Sys...

At NAB Show 2026, Net Insight will showcase the next evolution of Nimbra Edge, its orchestration and control layer designed to manage live media services across...

27/03/2026

Harmonic Delivers Winning Live Sports Streaming Innovatio...

Harmonic (NASDAQ: HLIT) today announced powerful new innovations that further elevate the company's sports streaming solution. The advanced capabilities enh...

27/03/2026

Bitmovin s Player Web X Expands with Performance First Ad...

Bitmovin, the leading provider of video streaming solutions, today announced significant new capabilities for Player Web X, its next-generation web video player...

27/03/2026

CS live Builds Future Ready OB Van With Riedel MediorNet...

Riedel Communications today announced that Czech-based production company CS live has equipped its newest outside broadcast (OB) van with an integrated Riedel i...

27/03/2026

MPTS 2026 Unveils First Wave of Speakers Featuring BBC Sk...

130 Industry Experts Confirmed as Show Celebrates its 10th Anniversary MPTS, the UK's largest and most influential event for the media, production and tech...

27/03/2026

Grass Valley and NVP Elevate LALIGA Broadcast Production...

Grass Valley today announced the successful completion of a major live production deployment with NVP, a leading European media company specializing in live spo...

27/03/2026

Appear Business Momentum Drives Leadership in Live Produc...

Showcases resilient solutions for satellite-to-IP migration, REMI and hybrid live production Appear ASA (Appear, OSE:APR), a global leader in live production t...

27/03/2026

Lightware MX2 delivers rock-solid 4K60 HDMI 10-bit signal...

San Francisco, California, March 2026 - Microsoft Ignite, a major annual conference hosted by Microsoft for developers, IT professionals, partners and business ...

27/03/2026

Clear-Coms EQUIP Wireless System Keeps Youth With A Missi...

Clear-Com is proud to highlight its support for worship teams through professional communication solutions, with the deployment of its EQUIP wireless system a...

27/03/2026

Orban Collaborates with Sage on Virtualized EAS Technology Demo

Orban Collaborates with Sage on Virtualized EAS Technology Demo Brie Clayton March 26, 2026 0 Comments Sage Alerting Systems (SAS), in cooperation wit...

27/03/2026

Berklee Alums Bring GIANT to Broadway

Berklee Alums Bring GIANT to Broadway Brian and Dayna Lee BM '13 met and fell in love at Berklee, and now their production companys first Broadway debut i...

27/03/2026

RT is Supporting 20 Arts and Cultural Events all over Ireland this April

RT Supporting the Arts RT is Supporting 20 Arts and Cultural Events all over Ireland this April RT is proud to support an exciting month of arts and cul...

27/03/2026

Average of 1.37 million viewers watched Irish World Cup heartbreak on RT2

Average of 1.37 million viewers watched Irish World Cup heartbreak on RT 2 Over one million streams on RT Player Highest number of streams for any single eve...

27/03/2026

Dolby Unveils Dolby House Shanghai, its First Global Flagship Experience Center

March 27 2026, 01:30 (PDT) Dolby Unveils Dolby House Shanghai, its First Global Flagship Experience Center A New Global Platform Showcasing Dolbys Innovatio...

27/03/2026

Dolby and QQ Music Advance Streaming Music with Enhancements to Dolby AC-4

March 27 2026, 03:30 (PDT) Dolby and QQ Music Advance Streaming Music with Enhancements to Dolby AC-4 Delivering greater fidelity, clarity and detail to Dol...

26/03/2026

Audio-Technica and Extron Announce Microsoft Teams-Certified Enterprise Solutions

Extron and Audio-Technica have announced that their Design Solutions devices, Ce...

26/03/2026

SPORTEL Asia Returns to Singapore with 370 Participants from 35 Countries

SPORTEL Asia took place March 24-25 at the Orchard Hotel Singapore, drawing 370 participants representing more than 220 companies from 35 countries. Approximate...

26/03/2026

NAB 2026: LTN to Showcase Satellite-to-IP Migration Solutions

LTN will demonstrate satellite-to-IP migration solutions for broadcasters, cable networks, MVPDs, and station groups at NAB Show 2026 (Booth W2100). The C-Ban...

26/03/2026

Seattle Mariners Upgrade Center Field Video Display at T-Mobile Park with Daktronics

The Seattle Mariners have partnered with Daktronics to install an upgraded outfi...

26/03/2026

Esports Nations Cup 2026 Names National Team Partners Across 100+ Countries

The Esports Foundation (EF) has awarded National Team Partner status to organizations and individuals across more than 100 countries and territories for the Esp...

26/03/2026

NAB 2026: Telestream Expands Vantage Integration with Adobe Premiere, Media Encoder, and Frame.io

Telestream has announced expanded integration between its Vantage workflow platf...

26/03/2026

NAB 2026: FOR-A Brings Nippon TV's viztrick AiDi AI Production Tool to Market with NVIDIA

FOR-A Corporation has announced a partnership with Nippon TV and NVIDIA to distr...

26/03/2026

CBS Sports Reaches New Long-Term Multiplatform Rights Agreement with the WNBA

CBS Sports and the WNBA have expanded their partnership through a new long-term, multiplatform rights agreement. CBS Sports will feature up to 20 regular-season...

26/03/2026

Zac Brown Band to Perform Sunday Night Baseball Open on NBC and Peacock

Zac Brown Band will star in the Sunday Night Baseball show open on NBC and Peacock, NBC Sports announced. The open was filmed at American Family Field, home of ...

26/03/2026

2026 MLB Season Opens With Full Slate of National Games

Major League Baseball opens the 2026 season with nationally televised games across multiple media partners. Fans can visit MLB.com/Watch for a complete schedule...

26/03/2026

DIRECTV Adds Nine MLB Team Channels for 2026 Season

DIRECTV has announced an agreement to carry nine new MLB-operated team channels, covering the Cincinnati Reds, Detroit Tigers (Detroit SportsNet), Kansas City R...

26/03/2026

Nationals.TV Now Available on Charter, Comcast, Cox, DIRECTV, Verizon Fios and More

Nationals.TV, the Washington Nationals' local broadcast channel, is now avai...

26/03/2026

MLB.TV Launches on the ESPN App Starting Opening Day

Starting Thursday, March 26, MLB.TV will be available on the ESPN App, giving fans access to more than 2,000 out-of-market regular season games. Fans can also r...

26/03/2026

What's Next for Spotify and NAVER in Korea After a Stockholm Summit

Spotify's partnership with NAVER, Korea's leading digital platform, is built around one simple goal: making it easier for listeners in Korea to discover...

26/03/2026

EastWest DrumX: Early access

Exclusive to ComposerCloud for first month Sample library and virtual instrument gurus EastWest have announced the upcoming release of a new virtual drum i...

26/03/2026

Dynamic Range Day 2026

Friday 27 March Friday 27 March will mark Dynamic Range Day 2026, the annual day of online activity that was started in 2010 by mastering engineer Ian Sheph...

26/03/2026

ASMR Choir from SRM Sounds

Includes four separate instruments Max Richter's SRM Sounds venture have just announced their fifth release, ASMR Choir. Captured at the main live room ...

26/03/2026

IMPORTANT FEEDBACK: PESP 6

The National Film and Video Foundation (NFVF), an agency of the Department of Sport, Arts and Culture (DSAC), has been tasked with administering the Sixth Presi...

26/03/2026

Why YEP Changed My Career (And Why It Might Change Yours)

By Matt Klein Hello to everyone reading this - whether you're a current YEP, HPA member, or just YEP-curious. I'm Matt Klein, one of the co-chairs of t...

26/03/2026

CBS Sports Inks New WNBA Deal

Share Copy link Facebook X Linkedin Bluesky Email...