As large language models (LLMs) grow larger, they get smarter, with open models from leading developers now featuring hundreds of billions of parameters. At the same time, today's leading models are also capable of reasoning, which means that they generate many intermediate reasoning tokens before delivering a final response to the user. The combination of these two trends-larger models that think using more tokens-drives the need for significantly higher compute performance.
Delivering the highest performance on production workloads takes a state-of-the-art technology stack-spanning chips, systems, and software-and an expansive developer ecosystem that is constantly building on that stack.
MLPerf Inference v5.1 is the latest version of the MLPerf Inference industry standard benchmark. With benchmark rounds held twice per year, the benchmark features many tests of AI inference performance and is regularly updated with new models and scenarios. This round features:
DeepSeek-R1 - a popular 671-billion parameter mixture-of-experts (MoE) reasoning model, developed by DeepSeek. In the server scenario, the time-to-first-token (TTFT) threshold is 2 seconds with a 12.5 tokens/second/user (TPS/user) target. All TPS/user targets are 99th percentile, meaning that 99% of tokens meet or exceed that TPS/user speed.
Llama 3.1 405B - MLPerf Inference v5.1 adds a new interactive scenario for the largest of the Llama 3.1 series of models, providing a faster 12.5 TPS/user threshold with a shorter 4.5 second TTFT requirement compared to the existing server scenario.
Llama 3.1 8B - an 8-billion parameter member of the Llama 3.1 series of models with offline, server (2 second TTFT, 10 TPS/user), and interactive (0.5 second TTFT, 33 TPS/user) scenarios. This replaces the GPT-J benchmark used in prior rounds.
Whisper - a popular speech recognition model that recently saw nearly 5 million downloads in a month on HuggingFace. This replaces RNN-T, which was featured in prior editions of the MLPerf Inference benchmark suite.
This round, NVIDIA submitted the first results using the new Blackwell Ultra architecture, announced in March. It came just six months after Blackwell made its debut in the available category in MLPerf Inference v5.0, setting new inference performance records. Additionally, the NVIDIA platform set new performance records on all newly added benchmarks this round-DeepSeek-R1, Llama 3.1 405B, Llama 3.1 8B, and Whisper-and continues to hold per-GPU performance records on all other MLPerf inference benchmarks.
MLPerf Inference Per-Accelerator Records
Benchmark Offline Server Interactive
DeepSeek-R1 5,842 tokens/second/GPU 2,907 tokens/second/GPU **
Llama 3.1 405B 224 tokens/second/GPU 170 tokens/second/GPU 138 tokens/second/GPU
Llama 2 70B 99.9% 12,934 tokens/second/GPU 12,701 tokens/second/GPU 7,856 tokens/second/GPU
Llama 2 70B 99% 13,015 tokens/second/GPU 12,701 tokens/second/GPU 7,856 tokens/second/GPU
Llama 3.1 8B 18,370 tokens/second/GPU 16,099 tokens/second/GPU 15,284 tokens/second/GPU
Stable Diffusion XL 4.07 samples/second/GPU 3.59 queries/second/GPU **
Mixtral 8x7B 16,099 tokens/second/GPU 16,131 tokens/second/GPU **
DLRMv2 99% 87,228 samples/second/GPU 80,515 samples/second/GPU **
DLRMv2 99.9% 48,666 samples/second/GPU 46,259 queries/second/GPU **
Whisper 5,667 tokens/second/GPU ** **
R-GAT 81,404 samples/second/GPU ** **
Retinanet 1,875 samples/second/GPU 1,801 queries/second/GPU **
Table 1. Performance records per GPU based on submissions powered by the NVIDIA platform. MLPerf Inference v5.0 and v5.1, Closed Division. Results retrieved from www.mlcommons.org on September 9, 2025. NVIDIA platform results from the following entries: 5.0-0072, 5.1-0007, 5.1-0053, 5.1-0079, 5.1-0028, 5.1-0062, 5.1-0086, 5.1-0073, 5.1-0008, 5.1-0070,5.1-0046, 5.1-0009, 5.1-0060, 5.1-0072. 5.1-0071, 5.1-0069 Per chip performance derived by dividing total throughput by number of reported chips. Per-chip performance is not a primary metric of MLPerf Inference v5.0 or v5.1.The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.
NVIDIA also made extensive use of NVFP4 acceleration across all DeepSeek-R1 and Llama model submissions using the Blackwell and Blackwell Ultra architectures.
In this post, we take a closer look at these performance results and the full-stack technologies that enabled them.
Blackwell Ultra sets reasoning records in MLPerf debut This round, NVIDIA submitted results in the available category using the GB300 NVL72 rack-scale system, the first-ever MLPerf submissions using the Blackwell Ultra architecture. Blackwell Ultra builds upon the many advances in the NVIDIA Blackwell architecture, with several key enhancements:
1.5x higher peak NVFP4 AI compute
2x higher attention-layer compute
1.5x higher HBM3e capacity
Compared to the GB200 NVL72 submission, GB300 NVL72 delivered up to 45% higher performance per GPU, setting the standard on the new DeepSeek-R1 benchmark. And compared to unverified results collected on a Hopper-based system, Blackwell Ultra delivered about 5x higher throughput per GPU-translating into significantly higher AI factory throughput and much lower cost per token.
DeepSeek-R1 Performance
Architecture Offline Server
Hopper 1,253 tokens/second/GPU 556 tokens/second/GPU
Blackwell Ultra 5,842 tokens/second/GPU 2,907 tokens/second/GPU
Blackwell Ultra Advantage 4.7x 5.2x
Table 2. Per-GPU performance on DeepSeek-R1. MLPerf Inference v5.1, Closed. Blackwell Ultra results based on results in entry 5.1-0072. Hopper results not verified by MLCommons Association. Per-GPU performance is not a primary metric of MLPerf Inference v5.1 and is calcu
Most recent headlines
09/11/2025
Dalet today announced a transformative leap forward for media operations: Agentic Artificial Intelligence (AI) that unifies the Dalet ecosystem under one natura...
28/10/2025
ESPN Announces Monsters Funday Football', Its Latest Real-Time Animated Bro...
28/10/2025
SVG All-Stars: Catherine Chalfant, Manager, Remote Operations, ESPNThe Ole Miss alum is an operational force behind ESPN's extensive college-football catalo...
28/10/2025
Elevating the experience: AI and data take Ryder Cup to the next level By Joe OHalloran
Tuesday, October 28, 2025 - 10:25
Print This Story
NBC produced th...
28/10/2025
Conquering the Air (waves): Taking a close up look at the IBC Accelerator Priva...
28/10/2025
World Series 2025: Spectrum SportsNet LA Brings Dodgers Fans Closer to the Actio...
28/10/2025
Dylan Southern and Benedict Cumberbatch at the premiere of The Thing with Feathers (photo by George Pimentel / Shutterstock for Sundance Film Festival)...
28/10/2025
For three weeks in Lagos, Spotify's Greasy Tunes Caf pop-up brought the cit...
28/10/2025
Once a niche subculture, German rap has evolved into an influential cultural movement. Now, Spotify is giving the genre a new home with OFFCULT, a playlist dedi...
28/10/2025
Shane Delia's Malta serves up the Mediterranean this summer
28 October, 2025
Media releases
Feast on 9,000 years of culinary history Mondays from 24 No...
28/10/2025
SBS's global sporting festival continues with the FIVB Beach Volleyball Worl...
28/10/2025
Bilbao, October 28, 2025 - AgileTV, a leading technology solutions company for t...
28/10/2025
Disney, NBCUniversal, FOX, Paramount Each Achieve Double-Digit Monthly Growth
...
28/10/2025
CINCINNATI The E.W. Scripps Company has announced an agreement to sell WRTV, its local ABC-affiliated station in Indianapolis, to Circle City Broadcasting for $...
28/10/2025
Berklee College of Music and Berklee Valencia Named to Billboards 2025 Top Music...
28/10/2025
NEW YORK As AI usage continues to spike, a new study from IAB delves into an important aspect of how AI is transforming the advertising business with new data s...
28/10/2025
Charlie Jablonski, a broadcast tech pioneer who helped shape the modern era of Olympics television coverage, died Oct. 25 at his home in Lake George N.Y., the N...
28/10/2025
VIENNA, Austria Bitmovin has launched Bitmovin Observability, a new stand-alone video data solution that delivers real-time insights into video playback. The so...
28/10/2025
LOS ANGELES LucidLink, the file streaming platform, has announced a Frame.io integration and expanded mobile capabilities at Adobe Max....
28/10/2025
Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, today announced that it has joined the Ama...
28/10/2025
Lightware, an industry leader in connectivity and signal management solutions, has announced a major update to its Taurus
platform, which now delivers flawless...
28/10/2025
Following a successful mid-September International Broadcasting Convention in Amsterdam, Hiltron Communications will promote its full range of satellite communi...
28/10/2025
Open Broadcast Systems has chosen MC&S (Media Consulting & Services) as a reseller to help strengthen its presence in France.
With over twenty years of experi...
28/10/2025
Bitmovin, leading provider of video streaming solutions, has launched Bitmovin Observability, a new stand-alone video data solution that delivers real-time insi...
28/10/2025
Ease Live, the leader in interactive TV technology, today announced the successful launch of interactive graphical overlays for UEFA Champions League matches fo...
28/10/2025
LucidLink, the file streaming platform, today at Adobe MAX announced a Frame.io integration and expanded mobile capabilities, streamlining collaboration and hel...
28/10/2025
ATLANTA Gray Media has promoted Nick Hasenecz to general manager of WNDU, its NBC affiliate in the South Bend-Elkhart, Ind., market....
28/10/2025
Applications Open for Berklee Fenway Neighborhood Improvement Grant Boston nonprofits can apply by December 12 for funding to support community projects that ...
28/10/2025
28 Oct 2025
VEON's JazzCash Wins Silver Award for Innovation in Lending at ...
28/10/2025
Guests include Wayne Rooney, Maya Jama, Dame Laura Kenny, Chloe Kelly, Chris McC...
28/10/2025
The two-hour special, recorded live from the iconic Dolby Theatre in LA, will ai...
28/10/2025
Eutelsat upgrades teleports with Rohde & Schwarz satellite uplink amplifiers High efficiency and resilient Ku-band amplifiers for excellent RF performance
...
28/10/2025
These appointments come at a pivotal time for ABC, as the organisation continues to evolve to meet the changing needs of a digital-first media ecosystem. The ne...
28/10/2025
Scripps Research awarded $4 million to advance platform for neurodevelopmental disorders The California Institute for Regenerative Medicine (CIRM) grant support...
27/10/2025
You can touch this: Haptics becoming central to the virtual live experience By Adrian Pennington
Friday, October 24, 2025 - 09:12
Print This Story
The vid...
27/10/2025
A tale of two trailers: France's Stop & Go doubles up with its new hybrid tr...
27/10/2025
Pro Padel League Stages City's Cup Finals Inside NYC's Hammerstein Ballr...
27/10/2025
World Series 2025: Sportsnet Delivers Made-in-Canada' Moment for a Nation U...
27/10/2025
ESPN Extends Partnership With Sony's Beyond Sports To Expand Animated Altern...
27/10/2025
Reid Davenport attends the 2025 Sundance Film Festival premiere of Life After at The Ray Theatre on January 27, 2025, in Park City, UT. (Photo by Robin Marsha...
27/10/2025
As the days grow shorter and the nights get darker, there's nothing like getting swept up in a story that sends shivers down your spine. In honor of spooky ...
27/10/2025
UPDATE: Both parties have reached a new carriage agreement....
27/10/2025
LONDON As Hollywood jumps into the production of mini-dramas, a new study from Ampere Analysis finds that more than one in 10 internet users have watched drama ...
27/10/2025
LONDON Yealink, a provider of unified communication and collaboration solutions, has joined the NDI ecosystem with the availability of its SmartVision 80 premiu...
27/10/2025
ATLANTA Gray Media has named Chris Conroy as general manager of its stations in Cleveland, leading WOIO, a CBS affiliate, The CW station WUAB and Telemundo outl...
27/10/2025
ESPOO, Finland European connectivity leaders Nokia and Ericsson, have partnered with Berlin's Fraunhofer Heinrich Hertz Institute (HHI), to shape and drive ...
27/10/2025
PHILADELPHIA Comcast has expanded NOW TV Latino, its Spanish-language live TV and streaming offering, adding five more channels from Univision, ViX Premium with...
27/10/2025
Test & measurement innovator, Leader Electronics, will present its latest products and solutions at InterBEE 2025 (Hall 5, Booth 5218) Makuhari Messe in Chiba, ...
27/10/2025
Back to All News
City of Shadows, the New Netflix Thriller Arrives on December 12
Entertainment
27 October 2025
GlobalSpain
Link copied to clipboard
Downl...
As large language models (LLMs) grow larger, they get smarter, with open models from leading developers now featuring hundreds of billions of parameters. At the same time, today's leading models are also capable of reasoning, which means that they generate many intermediate reasoning tokens before delivering a final response to the user. The combination of these two trends-larger models that think using more tokens-drives the need for significantly higher compute performance. 
