Acing the Test: NVIDIA Turbocharges Generative AI Training in MLPerf Benchmarks
08/11/2023
Among many new records and milestones, one in generative AI stands out: NVIDIA Eos - an AI supercomputer powered by a whopping 10,752 NVIDIA H100 Tensor Core GPUs and NVIDIA Quantum-2 InfiniBand networking - completed a training benchmark based on a GPT-3 model with 175 billion parameters trained on one billion tokens in just 3.9 minutes.
That's a nearly 3x gain from 10.9 minutes, the record NVIDIA set when the test was introduced less than six months ago.
The benchmark uses a portion of the full GPT-3 data set behind the popular ChatGPT service that, by extrapolation, Eos could now train in just eight days, 73x faster than a prior state-of-the-art system using 512 A100 GPUs.
The acceleration in training time reduces costs, saves energy and speeds time-to-market. It's heavy lifting that makes large language models widely available so every business can adopt them with tools like NVIDIA NeMo, a framework for customizing LLMs.
In a new generative AI test this round, 1,024 NVIDIA Hopper architecture GPUs completed a training benchmark based on the Stable Diffusion text-to-image model in 2.5 minutes, setting a high bar on this new workload.
By adopting these two tests, MLPerf reinforces its leadership as the industry standard for measuring AI performance, since generative AI is the most transformative technology of our time.
System Scaling Soars The latest results were due in part to the use of the most accelerators ever applied to an MLPerf benchmark. The 10,752 H100 GPUs far surpassed the scaling in AI training in June, when NVIDIA used 3,584 Hopper GPUs.
The 3x scaling in GPU numbers delivered a 2.8x scaling in performance, a 93% efficiency rate thanks in part to software optimizations.
Efficient scaling is a key requirement in generative AI because LLMs are growing by an order of magnitude every year. The latest results show NVIDIA's ability to meet this unprecedented challenge for even the world's largest data centers.
The achievement is thanks to a full-stack platform of innovations in accelerators, systems and software that both Eos and Microsoft Azure used in the latest round.
Eos and Azure both employed 10,752 H100 GPUs in separate submissions. They achieved within 2% of the same performance, demonstrating the efficiency of NVIDIA AI in data center and public-cloud deployments.
NVIDIA relies on Eos for a wide array of critical jobs. It helps advance initiatives like NVIDIA DLSS, AI-powered software for state-of-the-art computer graphics and NVIDIA Research projects like ChipNeMo, generative AI tools that help design next-generation GPUs.
Advances Across Workloads NVIDIA set several new records in this round in addition to making advances in generative AI.
For example, H100 GPUs were 1.6x faster than the prior-round training recommender models widely employed to help users find what they're looking for online. Performance was up 1.8x on RetinaNet, a computer vision model.
These increases came from a combination of advances in software and scaled-up hardware.
NVIDIA was once again the only company to run all MLPerf tests. H100 GPUs demonstrated the fastest performance and the greatest scaling in each of the nine benchmarks.
Speedups translate to faster time to market, lower costs and energy savings for users training massive LLMs or customizing them with frameworks like NeMo for the specific needs of their business.
Eleven systems makers used the NVIDIA AI platform in their submissions this round, including ASUS, Dell Technologies, Fujitsu, GIGABYTE, Lenovo, QCT and Supermicro.
NVIDIA partners participate in MLPerf because they know it's a valuable tool for customers evaluating AI platforms and vendors.
HPC Benchmarks Expand In MLPerf HPC, a separate benchmark for AI-assisted simulations on supercomputers, H100 GPUs delivered up to twice the performance of NVIDIA A100 Tensor Core GPUs in the last HPC round. The results showed up to 16x gains since the first MLPerf HPC round in 2019.
The benchmark included a new test that trains OpenFold, a model that predicts the 3D structure of a protein from its sequence of amino acids. OpenFold can do in minutes vital work for healthcare that used to take researchers weeks or months.
Understanding a protein's structure is key to finding effective drugs fast because most drugs act on proteins, the cellular machinery that helps control many biological processes.
In the MLPerf HPC test, H100 GPUs trained OpenFold in 7.5 minutes. The OpenFold test is a representative part of the entire AlphaFold training process that two years ago took 11 days using 128 accelerators.
A version of the OpenFold model and the software NVIDIA used to train it will be available soon in NVIDIA BioNeMo, a generative AI platform for drug discovery.
Several partners made submissions on the NVIDIA AI platform in this round. They included Dell Technologies and supercomputing centers at Clemson University, the Texas Advanced Computing Center and - with assistance from Hewlett Packard Enterprise (HPE) - Lawrence Berkeley National Laboratory.
Benchmarks With Broad Backing Since its inception in May 2018, the MLPerf benchmarks have enjoyed broad backing from both industry and academia. Organizations that support them include Amazon, Arm, Baidu, Google, Harvard, HPE, Intel, Lenovo, Meta, Microsoft, NVIDIA, Stanford University and the University of Toronto.
MLPerf tests are transparent and objective, so users can rely on the results to make informed buying decisions.
All the software NVIDIA used is available from the MLPerf repository, so all developers can get the same world-class results. These software optimizations get continuously folded into containers available on NGC, NVIDIA's software hub fo
LINK: | https://blogs.nvidia.com/blog/2023/11/08/scaling-ai-training-mlperf/... |
See more stories from nvidia |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
30/05/2024
When to Upgrade Software or Firmware
When to Upgrade Software or Firmware This post is from our blog and news archive. The information may be out of date. Please contact us for further information ...
30/05/2024
The City of San Jose Chooses Utah ScientificAgain!
The City of San Jose Chooses Utah Scientific Again! This post is from our blog and news archive. The information may be out of date. Please contact us for furth...
30/05/2024
Empowering Media Leaders: Transformative Insights from the Executive Learning Series
Navigating Strategy, Ethics, and Innovation in the evolving European media lands...
30/05/2024
Give Me the Backstory: Get to Know Thea Hvistendahl, the Filmmaker Behind Handling the Undead
By Bailey Pennick One of the most exciting things about the Sundance Film Festi...
30/05/2024
CHOOSE CHICAGO, CITY OF CHICAGO ANNOUNCE PROGRAMMING DETAILS OF SUNDANCE INSTITUTE X CHICAGO 2024 (June 28 - 30)
Tickets are now available for the first-of-its-kind event in the United States f...
30/05/2024
UNICEF and Spotify's Award-Winning Mental Health Hub, Our Minds Matter, Comes to Latin America
Mental health and well-being are fundamental components of a child's healthy...
30/05/2024
After 64 days, the winner of Alone Australia has been revealed
After 64 days, the winner of Alone Australia has been revealed 29 May, 2024 Media releases *Contains Spoilers* After 64 days alone in the extreme and wild...
30/05/2024
Alone Australia finale delivers strongly as SBS confirms it will join VOZ streaming
Alone Australia finale delivers strongly as SBS confirms it will join VOZ stream...
30/05/2024
Alone Australia finale delivers strongly for national broadcaster SBS with 4.2m tuning in across the season
Alone Australia finale delivers strongly for national broadcaster SBS with 4.2m ...
30/05/2024
Lockheed Martin Canada Awards L3Harris the Integrated Communications System Contract
Photo credit: BAE...
30/05/2024
The New Standard for Unmanned Ground Vehicle ISR
L3Harris' WESCAM MX -10 RSTA provides advanced ISR and targeting capabilities for land-based platforms including Unmanned Ground Vehicles....
30/05/2024
AccuWeather Inks Deal With Comcast Technology Solutions For Channel Origination
DENVER AccuWeather has selected Comcast Technology Solutions' (CTS') Managed Channel Origination (MCO) to create, manage and distribute linear TV, on-de...
30/05/2024
Samsung, LG Adopt IAB Software Kit
NEW YORK The global body that sets technical standards for digital advertising has expanded the reach of its measurement software development kit to include Sam...
30/05/2024
Amazon Celebrates 10th Anniversary of Fire TV Devices with an AI Upgrade
As Amazon celebrates the 10th anniversary of the launch of the first Fire TV devices in 2014, the company has unveiled new AI-powered search capabilities for th...
30/05/2024
Finding the sustainability balance with personalised video streaming
As the expansion of personalised video streaming continues apace, JUMPs CEO and co-founder, Jer nimo Macan s and Fran ois Polarczyk, sustainability director at ...
30/05/2024
EditShare boosts sales direction with alumnus Grant Carro...
EditShare, the technology leader that enables storytellers to create and manage collaborative workflows at every stage from storyboard to screening, has appoint...
30/05/2024
COW Featured Resume: Paula Zimmerman - Video Editor - Motion Designer
COW Featured Resume: Paula Zimmerman - Video Editor - Motion Designer Brie Clayton May 30, 2024 0 Comments Paula Zimmerman Looking for work? Sign up...
30/05/2024
Palme d'Or Winner Anatomy of a Fall Finished with DaVinci Resolve Studio
Palme d'Or Winner Anatomy of a Fall Finished with DaVinci Resolve Studio Brie Clayton May 30, 2024 0 Comments Winner of the Palme d'Or at the ...
30/05/2024
Real-Time Workflow Masterclass
Real-Time Workflow Masterclass Michael Cioni May 30, 2024 0 Comments Would you rather have your workflow move at the same speed it always has, or woul...
30/05/2024
Ending a Loop Expression in After Effects
Ending a Loop Expression in After Effects Graham Quince May 30, 2024 0 Comments This question comes up quite a bit on forums and it seems to trip peop...
30/05/2024
Parks: Prime Video Has Lowest Churn Rate
DALLAS Consumers who subscribe to streaming services are the least likely to cancel Prime Video among all major providers, according to Parks Associates' St...
30/05/2024
SMPTE/RIS-OSVP tests model for circle of confusion'
According to Camera and Lens Metadata committee, modern digital cinema cameras are not accurately representing the usable depth of field achieved with various l...
30/05/2024
Meet the vice president of product management
Dave MacKinnon, vice president of product management at Clear-Com, explains the value of collaboration and networking in building a media career By Matthew Cor...
30/05/2024
That Station Launches New App, Website, Filled with Features, Music, Interviews and More
That Station's app and website just got a major upgrade. The station crew co...
30/05/2024
SWR Deploys Rohde & Schwarz Pixel Power Software Playout Solution
MUNICH, Germany Regional German broadcaster S dwestrundfunk (SWR) has deployed the Rohde & Schwarz Pixel Power graphics and playout solution....
30/05/2024
Hollyland Technology Unveils Pyro Wireless Video Transmission Series
IRVINE, Calif. Hollyland Technology has launched Pyro, a wireless video transmission system designed for the multi-person, mobile transmission and monitoring re...
30/05/2024
AIMS Opens Call for Presentations for Media-Over-IP Pavilion at AES New York
AIMS has announced it will once again collaborate with the Audio Engineering Society (AES) to bring the popular Media-Over-IP Pavilion to the AES New York show,...
30/05/2024
FCC Announces Opportunity for LPTV Stations to Change Channels
WASHINGTON, D.C. The Federal Communications Commission (FCC) Media Bureau has announced that beginning on August 20, 2024, Class A television, Low Power Televis...
30/05/2024
Comcast's StreamSaver Streaming Bundle Goes Live
Comcast has officially launched its discounted $15-a-month StreamSaver streaming bundle of Netflix, Peacock and Apple TV+ services. The previously announced bun...
30/05/2024
AMG Launches TV Stations, Local Now FAST Channels on Amazon's Fire TV Channels
LOS ANGELES Allen Media Group (AMG) is launching three of its streaming brands o...
30/05/2024
Viant Integrates with Google Cloud's BigQuery Data Clean Rooms
IRVINE, Calif. The ad tech company Viant Technology Inc. has announced a new integration with Google Cloud's BigQuery data clean rooms that enables the seam...
30/05/2024
Marshall Brings Selection of New and Proven AV Solutions...
Marshall Electronics will highlight several of its newest product offerings at InfoComm 2024 (Booth C8982), including the CV612 auto-tracking PTZ camera and the...
30/05/2024
Triveni Digital Streamlines TV Operations at the 2024 ATS...
Triveni Digital today announced that the company will showcase its end-to-end ATSC 3.0 offering at the 2024 ATSC NEXTGEN Broadcast Conference, June 12-14 in Was...
30/05/2024
Riedel Bolero Empowers Student Production Excellence at O...
Riedel Communications today announced that Orange County School of the Arts (OCSA) has successfully integrated the Bolero wireless intercom system into their st...
30/05/2024
SWR moves to software playout with integrated Pixel Power...
Rohde & Schwarz has implemented its Pixel Power graphics and playout solution at S dwestrundfunk (SWR), the regional German broadcaster based in Baden-Baden. SW...
30/05/2024
Media Asset Management vs Inventory Management
From their inception, media asset management (MAM) systems have been a crucial component of the media supply chain. Traditionally MAM systems service customers ...
30/05/2024
Disguise launches new generation of EX media servers to p...
Disguise has announced the next evolution of its hardware solutions with the launch of the new generation of EX media servers, following the recently launched R...
30/05/2024
PlayBox Neo Highlights Smart Media Playout Innovations at...
PlayBox Neo, a globally active producer of broadcast media playout and channel branding solutions, sustained its industry event visibility at three events durin...
30/05/2024
EMG Gravity Media Announces Martyn Edwards as General Man...
EMG / Gravity Media, the leading force in production and content, media services and facilities, is pleased to announce the appointment of Martyn Edwards as the...
30/05/2024
Screen Australia, Australians in Film and VicScreen announce FUTURE VISION
30 05 2024 - Media release Screen Australia, Australians in Film and VicScreen announce FUTURE VISION Lee Sung Jin and Joanna Calo Australians in Film, Scree...
30/05/2024
AI Benefits Come at a Cost
AI Benefits Come at a Cost Brie Clayton May 29, 2024 0 Comments After this, there is no turning back. You take the blue pill the story ends, you wa...
30/05/2024
Morgan Spurlock, Super Size Me' Director, Has Died
Morgan Spurlock, director of Super Size Me, died May 24 in New York. He was 53 and had been battling cancer....
30/05/2024
Departing Longtime Local Anchors Share Their Lessons Learned
A pair of beloved local anchors with extraordinary runs at their stations are finally stepping down. Tom Wills signs off at WJXT Jacksonville May 31, 49 years a...
30/05/2024
Inscape Founder Zeev Neumeier Launches GraySwan To Optimize CTV Ads
Zeev Neumeier, who founded Inscape and was a pioneer in automatic content recognition, is beta-testing his new venture, GraySwan....
30/05/2024
Robert De Niro To Get Service to America Award From NABLF
Robert De Niro will get the 2024 Service to America Leadership Award from the National Association of Broadcasters Leadership Foundation. The award is presented...
30/05/2024
AGT' is Back, With More Golden Buzzers and Youngest Contestant Ever
Season 19 of America's Got Talent is on NBC starting May 28. The new season features more Golden Buzzers, which send a standout act directly to the live sho...
30/05/2024
The Quiz With Balls', Game Show That Pits Brains Against Boldness, Starts on Fox
Jay Pharoah hosts The Quiz with Balls, a game show that Fox says pits brains ag...
30/05/2024
Netflix's Bridgerton' Keeps Top Spot in TVision Power Score Rankings
Netflix's Bridgerton (season three) repeated as the top show in TVision's Power Score ranking of programs on connected TV for the week of May 20....