NVIDIA Takes Inference to New Heights Across MLPerf Tests
05/04/2023
Three years ago when we introduced A100, the AI world was dominated by computer vision. Generative AI has arrived, said NVIDIA founder and CEO Jensen Huang.
This is exactly why we built Hopper, specifically optimized for GPT with the Transformer Engine. Today's MLPerf 3.0 highlights Hopper delivering 4x more performance than A100.
The next level of Generative AI requires new AI infrastructure to train large language models with great energy efficiency. Customers are ramping Hopper at scale, building AI infrastructure with tens of thousands of Hopper GPUs connected by NVIDIA NVLink and InfiniBand.
The industry is working hard on new advances in safe and trustworthy Generative AI. Hopper is enabling this essential work, he said.
The latest MLPerf results show NVIDIA taking AI inference to new levels of performance and efficiency from the cloud to the edge.
Specifically, NVIDIA H100 Tensor Core GPUs running in DGX H100 systems delivered the highest performance in every test of AI inference, the job of running neural networks in production. Thanks to software optimizations, the GPUs delivered up to 54% performance gains from their debut in September.
In healthcare, H100 GPUs delivered a 31% performance increase since September on 3D-UNet, the MLPerf benchmark for medical imaging.
Powered by its Transformer Engine, the H100 GPU, based on the Hopper architecture, excelled on BERT, a transformer-based large language model that paved the way for today's broad use of generative AI.
Generative AI lets users quickly create text, images, 3D models and more. It's a capability companies from startups to cloud service providers are rapidly adopting to enable new business models and accelerate existing ones.
Hundreds of millions of people are now using generative AI tools like ChatGPT - also a transformer model - expecting instant responses.
At this iPhone moment of AI, performance on inference is vital. Deep learning is now being deployed nearly everywhere, driving an insatiable need for inference performance from factory floors to online recommendation systems.
L4 GPUs Speed Out of the Gate NVIDIA L4 Tensor Core GPUs made their debut in the MLPerf tests at over 3x the speed of prior-generation T4 GPUs. Packaged in a low-profile form factor, these accelerators are designed to deliver high throughput and low latency in almost any server.
L4 GPUs ran all MLPerf workloads. Thanks to their support for the key FP8 format, their results were particularly stunning on the performance-hungry BERT model.
In addition to stellar AI performance, L4 GPUs deliver up to 10x faster image decode, up to 3.2x faster video processing and over 4x faster graphics and real-time rendering performance.
Announced two weeks ago at GTC, these accelerators are already available from major systems makers and cloud service providers. L4 GPUs are the latest addition to NVIDIA's portfolio of AI inference platforms launched at GTC.
Software, Networks Shine in System Test NVIDIA's full-stack AI platform showed its leadership in a new MLPerf test.
The so-called network-division benchmark streams data to a remote inference server. It reflects the popular scenario of enterprise users running AI jobs in the cloud with data stored behind corporate firewalls.
On BERT, remote NVIDIA DGX A100 systems delivered up to 96% of their maximum local performance, slowed in part because they needed to wait for CPUs to complete some tasks. On the ResNet-50 test for computer vision, handled solely by GPUs, they hit the full 100%.
Both results are thanks, in large part, to NVIDIA Quantum Infiniband networking, NVIDIA ConnectX SmartNICs and software such as NVIDIA GPUDirect.
Orin Shows 3.2x Gains at the Edge Separately, the NVIDIA Jetson AGX Orin system-on-module delivered gains of up to 63% in energy efficiency and 81% in performance compared with its results a year ago. Jetson AGX Orin supplies inference when AI is needed in confined spaces at low power levels, including on systems powered by batteries.
For applications needing even smaller modules drawing less power, the Jetson Orin NX 16G shined in its debut in the benchmarks. It delivered up to 3.2x the performance of the prior-generation Jetson Xavier NX processor.
A Broad NVIDIA AI Ecosystem The MLPerf results show NVIDIA AI is backed by the industry's broadest ecosystem in machine learning.
Ten companies submitted results on the NVIDIA platform in this round. They came from the Microsoft Azure cloud service and system makers including ASUS, Dell Technologies, GIGABYTE, H3C, Lenovo, Nettrix, Supermicro and xFusion.
Their work shows users can get great performance with NVIDIA AI both in the cloud and in servers running in their own data centers.
NVIDIA partners participate in MLPerf because they know it's a valuable tool for customers evaluating AI platforms and vendors. Results in the latest round demonstrate that the performance they deliver today will grow with the NVIDIA platform.
Users Need Versatile Performance NVIDIA AI is the only platform to run all MLPerf inference workloads and scenarios in data center and edge computing. Its versatile performance and efficiency make users the real winners.
Real-world applications typically employ many neural networks of different kinds that often need to deliver answers in real time.
For example, an AI application may need to understand a user's spoken request, classify an image, make a recommendation and then deliver a response as a spoken message in a human-sounding voice. Each step requires a different type
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
28/05/2024
Pixotope Enables Remote Intercontinental Camera Tracking for NEP the Netherlands
Pixotope Enables Remote Intercontinental Camera Tracking for NEP the Netherlands Brie Clayton May 28, 2024 0 Comments Remote markerless through-the-le...
28/05/2024
Picture Shop's Mark Kueper Grades Billy the Kid With DaVinci Resolve Studio
Picture Shop's Mark Kueper Grades Billy the Kid With DaVinci Resolve Studio Brie Clayton May 28, 2024 0 Comments Blackmagic Design today announced...
28/05/2024
Profoto enters the cinema market with L1600D
Profoto enters the cinema market with L1600D Brie Clayton May 28, 2024 0 Comments Profoto enters the cinema market with uncompromising speed of use an...
28/05/2024
After Effects cameras and Unreal Engine
After Effects cameras and Unreal Engine Graham Quince May 28, 2024 0 Comments Welcome to my series on learning Unreal Engine for video production, esp...
28/05/2024
Apple Motion: Understanding Fixed Resolution
Apple Motion: Understanding Fixed Resolution Simon Ubsdell May 28, 2024 0 Comments An overview of this tricky but important topic which can hit you wi...
28/05/2024
OBS Taps Alibaba Cloud for AI-Enhanced MultiCamera Replays at Paris 2024
LONDON Olympic Broadcasting Services recently tested AI-enhanced multcamera replay tech from Alibaba Cloud at the Olympic Qualifier Series in Shanghai in prepar...
28/05/2024
Dune Part 2 and Avatar colourists to take part in DaVinci Resolve Live Tour
The events are for filmmakers, editors, colourists, and visual effects artists, whether theyre beginners and experienced users By Jenny Priestley Published: ...
28/05/2024
Meet the head of sound
1185 Films Mark Hodgkin explains his journey from studying classic guitar and piano to working on the sound of TV adverts, films and documentaries By TVBEurope...
28/05/2024
Alfalite presents its LED displays at InfoComm 2024
Alfalite, the European LED display manufacturer, returns for the second consecutive year to InfoComm with its LED displays for the rental, fixed installation an...
28/05/2024
Leader Electronics Corporation Appoints AV Group Technolo...
Leader Electronics Corporation, globally active innovator of broadcast-quality test and measurement instrumentation, announces the appointment of Sydney-based A...
28/05/2024
Advanced 3D qualifier in DaVinci Resolve
Advanced 3D qualifier in DaVinci Resolve Kasia Jarco May 27, 2024 0 Comments In today's advanced tutorial, I want to show you how and why to use 3...
28/05/2024
Deadline Approaching for 2024 Emerging Leaders Intern Program
Applications for CBC & Leadership Triangle Due May 31 Only a few days remain for HBCU students in the Triangle to apply for the 2024 Emerging Leadership Intern...
28/05/2024
Thales' FlytEDGE digitally remasters the inflight entertainment experience
Facebook Twitter LinkedIn Live personalization for a journey filled with unique experiences Instantly stream favorites and never miss a beat, continue wa...
28/05/2024
AI-backbone for FCAS operational
AI-backbone for FCAS operational The HIS consortium and partners provide BAAINBw and industry with a cross-sectional AI development platform for FCAS (AI-back...
27/05/2024
Experience inspiration. Master challenges. With SIGRAFLEX and SIGRAFINE at ACHEMA 2024
It will soon be that time again: ACHEMA, the worlds most important trade show fo...
27/05/2024
How L3Harris Evolved into Canada's Trusted Tanker Aircraft In-Service Support Provider
L3Harris has been maintaining Canada's CC-150 Polaris fleet for over a decad...
27/05/2024
Tech Lifestyle Influencer Shelby Church Uses Blackmagic Cloud Storage with DaVinci Resolve Studio
Tech Lifestyle Influencer Shelby Church Uses Blackmagic Cloud Storage with DaVin...
27/05/2024
Bridge Technologies Introduce StreamOverview to the VB330
Bridge Technologies Introduce StreamOverview to the VB330 Brie Clayton May 27, 2024 0 Comments Single page diagnostics overview gives first-line engin...
27/05/2024
Intelligent Video Effects from Film Impact
Intelligent Video Effects from Film Impact Colin Smith May 27, 2024 0 Comments Take an incredible trip though the many unbelievable transitions from F...
27/05/2024
Midwest Regional Broadcasters Clinic Announces Agenda
The Midwest Regional Broadcasters Clinic (MRBC) announced its agenda for the clinic being held Tuesday, Sept. 10, and Wednesday, Sept. 11, in Middleton, Wis....
27/05/2024
NVIDIA Scoops Up Wins at COMPUTEX Best Choice Awards
Building on more than a dozen years of stacking wins at the COMPUTEX trade show's annual Best Choice Awards, NVIDIA was today honored with BCAs for its late...
27/05/2024
Live From NCAA Men's Lacrosse National Championship: ESPN Travels Down I-95 to Familiar Lincoln Financial Field
Live From NCAA Men's Lacrosse National Championship: ESPN Travels Down I-95 ...
27/05/2024
Rohde & Schwarz presents its solutions for next generation wide bandgap device test and debug at PCIM Europe
Rohde & Schwarz presents its solutions for next generation wide bandgap device t...
27/05/2024
Hierarchy' Trailer Teases a Dark Scandal and Social Upheaval at Jooshin High School
Back to All News Hierarchy' Trailer Teases a Dark Scandal and Social Uphea...
27/05/2024
SKY Perfect JSAT selects Thales Alenia Space to build a new cutting-edge software-defined satellite JSAT-31
Facebook Twitter LinkedIn Tokyo / Cannes, May 27th 2024 - Asia's large...
26/05/2024
Vizrt to showcase state-of-the-art proAV solutions at Inf...
Vizrt, the leader in real-time graphics and live production solutions for content creators, will be present at InfoComm for the first time since unifying with N...
26/05/2024
Alfredo Valdes Named Noticiero Telemundo Arizona' Meteorologist
Alfredo Valdes has been named meteorologist for Noticiero Telemundo Arizona weekday morning newscasts, which run on KTAZ Phoenix and KHRR Tucson. Both stations ...
26/05/2024
Paramount, Charter Reach Carriage Deal That Includes Linear Networks, TV Stations and Streaming Services
Paramount Global and Charter Communications said they reached a new carriage agr...
26/05/2024
Daytime Emmys To Again Be Hosted by ET's Kevin Frazier, Nischelle Turner
Entertainment Tonight's Kevin Frazier and Nischelle Turner are returning to host the 51st annual Daytime Emmys, CBS and the National Academy of Television A...
25/05/2024
Get to Know This Summer's Filmmakers Through These 12 Sundance Films
(L-R) Writer-director Hannah Pearl Utt and co-writer Jen Tullock star as sisters in Before You Know It, which premiered at the 2019 Sundance Film Festival....
25/05/2024
Study: Digital Media Ad Spend Grew 18% in Q1 24
NEW YORK A new study from Guideline indicates that In Q1 2024, large US advertisers expanded their overall ad spend by 7% compared to the year prior and that di...
25/05/2024
Accedo Helps ITV Expand ITVX to Sony PlayStation 4 and 5
STOCKHOLM Global video solutions provider, Accedo has announced that it worked with ITV in the U.S. to expand the reach of the broadcasters streaming service, I...
25/05/2024
Broadband Forum Celebrates 20th Anniversary of TR-069 Standard
Broadband Forum has announced that it is celebrating the 20-year anniversary of its groundbreaking TR-069 standard that has paved the way for the open standards...
25/05/2024
TV Tech Weekly Tech Wrap-Up
Missed any of our coverage of new products, services and deployments during your busy week? The TV Tech weekly wrap-up provides links to all of our product cove...
25/05/2024
HBO Original Series 30 Coins Season Two Finished with DaVinci Resolve Studio
HBO Original Series 30 Coins Season Two Finished with DaVinci Resolve Studio Brie Clayton May 24, 2024 0 Comments Blackmagic Design announced today th...
25/05/2024
VideoProc Converter AI: Your Answer to Video Format Challenges and Quality Enhancement
VideoProc Converter AI: Your Answer to Video Format Challenges and Quality Enhan...
24/05/2024
The Hives Celebrate 50 Years of Sweden's Global Music Success With Spotify Singles Cover
On April 6, 1974, the Swedish pop quartet ABBA won the Eurovision Song Contest w...
24/05/2024
The U.K. Holds Firm in the Fight for Fair Competition With the DMCC Act, But It's Not Over Yet
For more than a year, the U.K. government has been working to redefine how the i...
24/05/2024
Alone Australia continues to build as it moves towards finale
Alone Australia continues to build as it moves towards finale 23 May, 2024 Media releases The program continues to deliver for SBS with significant uplifts...
24/05/2024
EditShare Introduces Expanded Product Line-Up at BroadcastAsia
EditShare Introduces Expanded Product Line-Up at BroadcastAsia Transforming innovations in workflow, server and delivery from storyboard to screen Boston, MA...
24/05/2024
ZEISS CinCraft Scenario Camera Tracking Now Compatible wi...
Scenario 2.0 introduces pre-calibrated lens templates and the Lens Template Finetuner, increasing flexibility and compability while also saving a great amount o...
24/05/2024
Chyron Unlocks a Complete Newsroom in the Cloud With News...
Based on a long-term, coordinated development effort, Chyron today announced sweeping improvements across its news workflow portfolio that empower broadcasters ...
24/05/2024
Cobalt Expands its Reach into the AV Market with Plans to...
Cobalt Digital, known for its vast array of signal processing products, is strengthening its position in the Pro AV market by exhibiting at InfoComm 2024 for th...
24/05/2024
IHSE USA Earns Coveted Awards at the 2024 NAB Show
HSE USA today announced that the company s JPEG-XS IP Core for KVM and kvm-tec Scalable Pro Line 5K were honored with three awards at this year's NAB Show i...
24/05/2024
Aputure Gears Up for the 2024 Cine Gear Expo
Aputure, creators of LED lighting for filmmakers, is excited to showcase its award-winning lineup of professional lighting solutions at the upcoming Cine Gear E...
24/05/2024
EVERTZAV JOINS GPA GLOBAL PARTNER PROGRAM
EvertzAV (https://av.evertz.com), a division of Evertz, the global leader in providing professional A/V over IP solutions, is proud to announce its partnership ...
24/05/2024
Metropolis Studios Upgrades To Prism Sound Dream ADA-128...
With 25 Prism Sound ADA-8XR multichannel converters already in use across its five studios, the internationally acclaimed Metropolis Studios in London is no str...
24/05/2024
Leader Expands LVB440 IP Analyzer with New Measurement To...
Leader Electronics Corporation, globally active innovator of broadcast-quality test and measurement instrumentation, announces an expansion to the capabilities ...