Sony Pixel Power calrec Sony

Scaling to New Heights: NVIDIA MLPerf Training Results Showcase Unprecedented Performance and Elasticity

12/06/2024

The full-stack NVIDIA accelerated computing platform has once again demonstrated exceptional performance in the latest MLPerf Training v4.0 benchmarks.

NVIDIA more than tripled the performance on the large language model (LLM) benchmark, based on GPT-3 175B, compared to the record-setting NVIDIA submission made last year. Using an AI supercomputer featuring 11,616 NVIDIA H100 Tensor Core GPUs connected with NVIDIA Quantum-2 InfiniBand networking, NVIDIA achieved this remarkable feat through larger scale - more than triple that of the 3,584 H100 GPU submission a year ago - and extensive full-stack engineering.

Thanks to the scalability of the NVIDIA AI platform, Eos can now train massive AI models like GPT-3 175B even faster, and this great AI performance translates into significant business opportunities. For example, in NVIDIA's recent earnings call, we described how LLM service providers can turn a single dollar invested into seven dollars in just four years running the Llama 3 70B model on NVIDIA HGX H200 servers. This return assumes an LLM service provider serving Llama 3 70B at $0.60/M tokens, with an HGX H200 server throughput of 24,000 tokens/second.

NVIDIA H200 GPU Supercharges Generative AI and HPC The NVIDIA H200 Tensor GPU builds upon the strength of the Hopper architecture, with 141GB of HBM3 memory and over 40% more memory bandwidth compared to the H100 GPU. Pushing the boundaries of what's possible in AI training, the NVIDIA H200 Tensor Core GPU extended the H100's performance by up to 47% in its MLPerf Training debut.

NVIDIA Software Drives Unmatched Performance Gains Additionally, our submissions using a 512 H100 GPU configuration are now up to 27% faster compared to just one year ago due to numerous optimizations to the NVIDIA software stack. This improvement highlights how continuous software enhancements can significantly boost performance, even with the same hardware.

This work also delivered nearly perfect scaling. As the number of GPUs increased by 3.2x - going from 3,584 H100 GPUs last year to 11,616 H100 GPUs with this submission - so did the delivered performance.

Learn more about these optimizations on the NVIDIA Technical Blog.

Excelling at LLM Fine-Tuning As enterprises seek to customize pretrained large language models, LLM fine-tuning is becoming a key industry workload. MLPerf introduced a new LLM fine-tuning benchmark this round, based on the popular low-rank adaptation (LoRA) technique applied to Meta Llama 2 70B.

The NVIDIA platform excelled at this task, scaling from eight to 1,024 GPUs, with the largest-scale NVIDIA submission completing the benchmark in a record 1.5 minutes.

Accelerating Stable Diffusion and GNN Training NVIDIA also accelerated Stable Diffusion v2 training performance by up to 80% at the same system scales submitted last round. These advances reflect numerous enhancements to the NVIDIA software stack, showcasing how software and hardware improvements go hand-in-hand to deliver top-tier performance.

On the new graph neural network (GNN) test based on R-GAT, the NVIDIA platform with H100 GPUs excelled at both small and large scales. The H200 delivered a 47% boost on single-node GNN training compared to the H100. This showcases the powerful performance and high efficiency of NVIDIA GPUs, which make them ideal for a wide range of AI applications.

Broad Ecosystem Support Reflecting the breadth of the NVIDIA AI ecosystem, 10 NVIDIA partners submitted results, including ASUS, Dell Technologies, Fujitsu, GIGABYTE, Hewlett Packard Enterprise, Lenovo, Oracle, Quanta Cloud Technology, Supermicro and Sustainable Metal Cloud. This broad participation, and their own impressive benchmark results, underscores the widespread adoption and trust in NVIDIA's AI platform across the industry.

MLCommons' ongoing work to bring benchmarking best practices to AI computing is vital. By enabling peer-reviewed comparisons of AI and HPC platforms, and keeping pace with the rapid changes that characterize AI computing, MLCommons provides companies everywhere with crucial data that can help guide important purchasing decisions.

And with the NVIDIA Blackwell platform, next-level AI performance on trillion-parameter generative AI models for both training and inference is coming soon.
LINK: https://blogs.nvidia.com/blog/mlperf-training-benchmarks/...
See more stories from nvidia

Most recent headlines

04/08/2024

Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation

Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....

14/07/2024

Gray Television Stations To Air Seattle Kraken Games in Alaska

Gray Television has made a deal with Tegna and the Seattle Kraken to air Kraken National Hockey League games in two Alaska markets....

14/07/2024

Chili's Sponsors Summer Movies on Vizio's WatchFree Plus

Chili's has signed up as the exclusive presenting sponsor of a Hot Summer Movies collection of films available on demand on Vizio's WatchFree Plus str...

14/07/2024

Spacecom Secures $4.2 Million Satellite Communication Services Deal

BACK TO PRESS RELEASES Spacecom Secures $4.2 Million Satellite Communication Services Deal 14.07.2024 Spacecom Secures $4.2 Million Satellite Communication ...

13/07/2024

Skywalkers: A Love Story Takes Love to Unimaginable Heights

Jeff Zimbalist, Ivan Beerkus, Angela Nikolau, and Maria Bukhonina at the premiere of Skywalkers: A Love Story at The Ray Theatre in Park City, Utah, on Januar...

13/07/2024

Invitation for South African filmmakers to submit films for the 97th Annual Academy Awards (Oscars) International Feature Film category.

The National Film and Video Foundation (NFVF), an agency of the Department of Sp...

13/07/2024

VideoAmp Integrates Snap Inc. Ad Inventory

NEW YORK VideoAmp has announced a wide-ranging collaboration with Snap Inc. that will see the social technology platform integrate its first-party data, video, ...

13/07/2024

RAI Moves up Launch of DVB-T2 HEVC Broadcasts

The Italian public broadcaster RAI will start broadcasting on Multiplex B in the DVB-T2 HEVC standard on August 28, according to Marco Rossignoli, president of ...

13/07/2024

Mile-High AI: NVIDIA Research to Present Advancements in Simulation and Gen AI at SIGGRAPH

Mile-High AI: NVIDIA Research to Present Advancements in Simulation and Gen AI a...

13/07/2024

Future Today Adds Studio Content to Fawesome

Future Today said it has signed licencing deals with Sony Pictures Entertainment, Samuel Goldwyn Films and Gravitas Ventures that will add fresh content to its ...

13/07/2024

VideoAmp To Integrate Data, Inventory From Snap Into Ad-Planning Tools

VideoAmp said it made a deal enabling the measurement company to integrate Snap's first-party data, video and augmented-reality ad inventory into VideoAmp&#...

13/07/2024

ABC Renews Bachelor in Paradise' for Season 10

ABC has ordered a tenth season of Bachelor in Paradise. The season will air in 2025 on the network and stream the next day on Hulu....

13/07/2024

Bill Belichick Joins Inside the NFL' for Season 2 on The CW

Bill Belichick, the legendary head coach who led the New England Patriots to eight Super Bowl wins, is joining Inside the NFL, which starts its second season on...

13/07/2024

Allen Media Group's HBCU Go Set To Kick Off College Football Season

Allen Media Group's HBCU Go said it will televise 26 college football games played by Historically Black Colleges and Universities....

13/07/2024

Backed By Top Producers, Exploding Kittens' Card Game Adapted to Netflix Series

Exploding Kittens, an animated series based on the card game, debuts on Netflix ...

13/07/2024

ABC Owned Stations, Ava DuVernay Partner on Our America' Special

ABC Owned Television Stations premieres Our America: Hidden Stories with Ava DuVernay starting Sunday, July 14. The one-hour special sheds light on the interco...

13/07/2024

Shania Twain To Host People's Choice Country Awards' on NBC

Shania Twain will host the People's Choice Country Awards, which happens Thursday, September 26 on NBC and Peacock. The two-hour special will be live at the...

12/07/2024

My byte-size week at Calrec

My Byte-size week at Calrec By Erin Crowther My name is Erin, and I am a student at Ryburn Sixth Form. This week I completed my work experience at Calrec Audi...

12/07/2024

Fox Sports to Create Virtual Reality Experience for Copa Amrica Final

CAMPBELL, Calif. Fox Sports and YBVR have announced that they will be creating an immersive virtual reality experience for the Copa Am rica Final on Meta Quest,...

12/07/2024

Gray TV Stations in Alaska to Air Seattle Kraken Games

TYSONS, Va. Tegna Inc., the Seattle Kraken and Gray Media have announced that Gray's KAUU in Anchorage and KYEX in Juneau will expand the Kraken broadcast n...

12/07/2024

Hallmark To Launch New Subscription Streaming Service

PASADENA, Calif. Hallmark has unveiled Hallmark+, a subscription streaming television service that will combine the company's brand of lifestyle experience ...

12/07/2024

Going the extra Milo: MRMC unveils new rig

The Super Milo can achieve speeds of up to 3 m/s on track and 5 m/s with combined camera movement By Matthew Corrigan Published: July 12, 2024 The Super M...

12/07/2024

TGI Sport acquires virtual advertising company Supponor

The deal is scheduled to close mid-summer, financial details have not been revealed By Jenny Priestley Published: July 12, 2024 The deal is scheduled to c...

12/07/2024

What the Clearhaven Deal Means for Zixi's Future

In recent years, as TV companies and producers pushed to launch streaming services and deploy IP infrastructures, Zixi has been at the center of that tectonic t...

12/07/2024

Ateme To Focus on Maximizing Output, Minimizing Costs At IBC 2024

PARIS Ateme will feature technologies pushing the boundaries of transformation, monetization and experience at IBC 2024, Sept. 13-16, at the RAI Convention Cent...

12/07/2024

DNEG Group Completes Prime Focus Technologies Acquisition

The DNEG Group has completed acquisition of Prime Focus Technologies....

12/07/2024

Wimbledon and Hawk-Eye Innovations Extend Tech Agreement

LONDON The All England Lawn Tennis Club and Sony's Hawk-Eye Innovations have announced a new multi-year agreement that will see Hawk-Eye continue to provide...

12/07/2024

Vision2 Marketing to Rep FOR-A as Part of Acceleration in...

As part of a more expansive push into the U.S. market, FOR-A announces its partnership with Vision2 Marketing, a manufacturers' rep for pro audio, video, li...

12/07/2024

KTRK Optimizes Look of Live Streams and Broadcasts with S...

Houston broadcaster KTRK-TV has been on the scene for 70 years and is known for community outreach as well as investigative journalism. Journalist alumni includ...

12/07/2024

QuickLink Expands its US Presence With Two New Seasoned T...

QuickLink, the leading global provider of multi-camera video productions and remote contributions, announces a significant growth strategy with two high-level p...

12/07/2024

Imagine Communications Names Don Durand SVP Global Sales...

Imagine Communications has appointed international media industry executive Don Durand to the position of senior vice president of global sales for its ad tech ...

12/07/2024

BH Telecom selects Appear to deliver Moja TV service to s...

Appear, the global leader in live production technology, has been selected by BH Telecom to enhance the live channel processing software for its Moja TV service...

12/07/2024

Ikegami Announces New Additions to its Range of Broadcast...

Ikegami announces two new additions to its range of broadcast quality picture monitors: the 24-inch HLM-2460WA and 18.5 inch HLM-1860WR. These will make their E...

12/07/2024

Vizrt focuses on BroadcastAV market with PTZ3 PLUS camera...

Vizrt, the leader in real-time graphics and live production solutions for content creators, brings the BroadcastAV market sharply into focus with today's la...

12/07/2024

Czech TV Live Streams Prague Marathon with Resilient Deje...

Local production company Livecast counts on Dejero Smart Blending Technology for connectivity on the move Dejero recently provided Czech production company Li...

12/07/2024

LTN playout solution powers European Football Championshi...

LTN, the industry leader in transformative media technology and video transport solutions, announces the successful delivery of live and pre-recorded coverage t...

12/07/2024

Ateme to Showcase Cutting-Edge Solutions to Maximise Outp...

Ateme's solutions deliver unprecedented video experiences and content monetisation while optimizsng workflows through hybrid on-premises and cloud systems. ...

12/07/2024

Lightware Announces New Leadership in the Enterprise Team

Lightware Visual Engerineering, a leading manufacturer of connectivity solutions for the professional integrated systems market and a pioneer in signal manageme...

12/07/2024

Anton Bauer Salt-E Dog Wins Innovation Impact Award at Ro...

Anton/Bauer has announced that its pioneering sustainable power product, Salt-E Dog, has been awarded the prestigious Innovation Impact Award at the Royal Telev...

12/07/2024

Teatro TV Browser by iWedia Surpasses 40 million Devices...

iWedia, a leading innovator in software solutions for connected TV devices, is thrilled to announce a significant milestone for its flagship product, the Teatro...

12/07/2024

Tim Belcher joins Light Iron as new Managing Director

Light Iron, the post-production creative-services division of Panavision, has named industry veteran Tim Belcher the company's new Managing Director. Belche...

12/07/2024

Screen Australia and ABC announce new comedy series Optics

12 07 2024 - Media release Screen Australia and ABC announce new comedy series Optics Optics. Photo credit: Joel Pratley ABC and Screen Australia have annou...

12/07/2024

COW Job Listing: Need a Drone Shot in Boardman, Oregon

COW Job Listing: Need a Drone Shot in Boardman, Oregon Brie Clayton July 11, 2024 0 Comments Need a Drone shot July 9, 2024COW Job Listing: Freelanc...

12/07/2024

American Underground Celebrates Pride & Juneteenth with Free, Public Learning Events

American Underground, Capitol Broadcasting's co-working community division, ...

12/07/2024

5 Session-Saving Tips for Studio One

By Craig Anderton I admit it: the following tips are based on personally embarrassing experiences. I like to work fast to keep the creative juices flowing, but...

12/07/2024

Fox Sports Heads to Berlin for Euro Final; Wraps Up Massive Month of International Soccer Coverage

Fox Sports Heads to Berlin for Euro Final; Wraps Up Massive Month of Internation...

12/07/2024

Euro 2024: UEFA Gets Set for Spain v England Final

Euro 2024: UEFA gets set for Spain v England final By George Bevir Friday, July 12, 2024 - 11:19 Print This Story When England take on Spain on Sunday in ...

12/07/2024

Euro 2024: Inside UEFA's International Broadcast Centre

Euro 2024: Inside UEFA's International Broadcast Centre By George Bevir Friday, July 12, 2024 - 08:00 Print This Story The IBC in Leipzig, Germany (Ph...

12/07/2024

NBC Sports Bay Area & California's Rusty West on Vendor Partnerships Bringing Broadcast Elements to Life

NBC Sports Bay Area & California's Rusty West on Vendor Partnerships Bringin...

12/07/2024

Long-time PSSI Broadcast/Mechanical Engineer Paul Blomstrom Dead at 44

Long-time PSSI Broadcast/Mechanical Engineer Paul Blomstrom Dead at 44 By Ken Kerschbaumer, Editorial Director Friday, July 12, 2024 - 12:55 pm Print This ...