
NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).
The open model combined with NVIDIA accelerated computing equips developers, researchers and businesses to innovate responsibly across a wide variety of applications.
Trained on NVIDIA AI Meta engineers trained Llama 3 on computer clusters packing 24,576 NVIDIA H100 Tensor Core GPUs, linked with RoCE and NVIDIA Quantum-2 InfiniBand networks.
To further advance the state of the art in generative AI, Meta recently described plans to scale its infrastructure to 350,000 H100 GPUs.
Putting Llama 3 to Work Versions of Llama 3, accelerated on NVIDIA GPUs, are available today for use in the cloud, data center, edge and PC.
From a browser, developers can try Llama 3 at ai.nvidia.com. It's packaged as an NVIDIA NIM microservice with a standard application programming interface that can be deployed anywhere.
Businesses can fine-tune Llama 3 with their data using NVIDIA NeMo, an open-source framework for LLMs that's part of the secure, supported NVIDIA AI Enterprise platform. Custom models can be optimized for inference with NVIDIA TensorRT-LLM and deployed with NVIDIA Triton Inference Server.
Taking Llama 3 to Devices and PCs Llama 3 also runs on NVIDIA Jetson Orin for robotics and edge computing devices, creating interactive agents like those in the Jetson AI Lab.
What's more, NVIDIA RTX and GeForce RTX GPUs for workstations and PCs speed inference on Llama 3. These systems give developers a target of more than 100 million NVIDIA-accelerated systems worldwide.
Get Optimal Performance with Llama 3 Best practices in deploying an LLM for a chatbot involves a balance of low latency, good reading speed and optimal GPU use to reduce costs.
Such a service needs to deliver tokens - the rough equivalent of words to an LLM - at about twice a user's reading speed which is about 10 tokens/second.
Applying these metrics, a single NVIDIA H200 Tensor Core GPU generated about 3,000 tokens/second - enough to serve about 300 simultaneous users - in an initial test using the version of Llama 3 with 70 billion parameters.
That means a single NVIDIA HGX server with eight H200 GPUs could deliver 24,000 tokens/second, further optimizing costs by supporting more than 2,400 users at the same time.
For edge devices, the version of Llama 3 with eight billion parameters generated up to 40 tokens/second on Jetson AGX Orin and 15 tokens/second on Jetson Orin Nano.
Advancing Community Models An active open-source contributor, NVIDIA is committed to optimizing community software that helps users address their toughest challenges. Open-source models also promote AI transparency and let users broadly share work on AI safety and resilience.
Learn more about how NVIDIA's AI inference platform, including how NIM, TensorRT-LLM and Triton use state-of-the-art techniques such as low-rank adaptation to accelerate the latest LLMs.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
01/06/2026
January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026
Throughout the week, Dolby brings to life the latest innovatio...
02/05/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
01/05/2026
January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...
02/04/2026
HBO and NFL Films have announced Hard Knocks: Training Camp with the Seattle Sea...
02/04/2026
Haivision has announced the Makito ONE, a single-blade video encoding and decoding platform, at NAB Show 2026. The platform combines dual-channel video encoding...
02/04/2026
Telestream has introduced UP.Lens, a cloud-based multiviewer and monitoring serv...
02/04/2026
Mark Roberts Motion Control (MRMC) will exhibit at NAB Show 2026 (Booth C5220, April 19-22, Las Vegas Convention Center), marking the company's 60th anniver...
02/04/2026
Net Insight has introduced programmable Trust Boundaries, a feature integrated i...
02/04/2026
Bitmovin has announced support for SGAI (Server-Guided Ad Insertion) in its playback products, using HLS interstitials. SGAI combines elements of client-side an...
02/04/2026
Riedel Communications' SimplyLive RiMotion R12 replay system is supporting B...
02/04/2026
LTN, a managed IP video transport company, and Ateme, a video compression and de...
02/04/2026
Harmonic has announced that TDF, a broadcast infrastructure operator in France, has deployed Harmonic's XOS Advanced Media Processor and ProStream X Video S...
02/04/2026
Eluvio and the United Rugby Championship (URC) have announced first-year results...
02/04/2026
The Kansas City Current and Scripps Sports have announced that ION will broadcas...
02/04/2026
ESPN will debut Courtside at the Women's Final Four Presented by AT&T, an alt-cast airing Friday, April 3 at 7 p.m. and 9:30 p.m. ET on ESPN2, and Sunday, A...
02/04/2026
The Professional Audio Manufacturers Alliance (PAMA) and Shure Incorporated are ...
02/04/2026
ESPN's MegaCast Coverage of 2026 NCAA Women's Final Four Begins Friday, ...
02/04/2026
DAZN has announced the launch of DAZN Playmakers, a global influencer program designed to build a network of sports content creators. The programme will give cr...
02/04/2026
Tony Cole, Jessica Lee shift into new roles reporting to ESPN SVP/Content Operations Chris Calcinari....
02/04/2026
Michigan's Fab Five will reunite for an alternate presentation of the Mich...
02/04/2026
Real-time tracking, virtual production, and Pixar storytelling converge for Apri...
02/04/2026
NASA's long-awaited Artemis II mission has launched four astronauts on a 10-day journey around the moon, marking the first manned launch toward the moon sin...
02/04/2026
(L-R) Molly Belle Wright, Wyatt Solis, and John Magaro appear in Omaha by Cole Webley, an official selection of the 2025 Sundance Film Festival. (Photo courte...
02/04/2026
4 - 11 April 2026
VEMIA's latest gear auction is just around the corner, and there's already a wealth of sought-after instruments and studio gear li...
02/04/2026
Little Labs emulation now available to all
The latest Universal Audio plug-in to become available outside of the company's DSP-powered UAD2 platform has...
02/04/2026
SBS Welcomes Back Cup Fever! For The Biggest FIFA World Cup Ever
2 April, 2026
Media releases
Santo Cilauro, Ed Kavalee and a host of special guests retur...
02/04/2026
Truth. Power. Perspective: NITV's Flagship Current Affairs Programs Return t...
02/04/2026
L3Harris has successfully powered the historic launch of the Artemis II mission, providing propulsion and avionics....
02/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/04/2026
Already deployed extensively by NBC Sports, FOR-A Corporation will demonstrate GoVertical! AiDi, the real-time 9:16 autocropping feature of viztrick AiDi, durin...
02/04/2026
Interra Systems, a provider of end-to-end quality assurance solutions for the digital media industry, announced that Elite Media Technologies has selected its B...
02/04/2026
Harmonic's Media Processing Solutions Maximize Bandwidth Efficiency for Terrestrial Broadcast Delivery
Harmonic (NASDAQ: HLIT) today announced that TDF, a...
02/04/2026
NBC Sports Deploys viztrick AiDi to Stream Live Events in 9:16 Mobile-First Formats with Auto Tracking, Development Signals Strategic Shift for FOR-A
Long reco...
02/04/2026
Evergent will showcase new innovations in subscriber lifecycle management and monetization at NAB Show 2026 (Las Vegas, April 18 22), including:
New advances i...
02/04/2026
Riedel Communications is proud to be part of Binghamton University, State University of New York, Athletics' milestone year, celebrating the university'...
02/04/2026
Encompass Digital Media and Techex have today announced new, fully managed, cloud-native Master Control services designed to meet the growing operational demand...
02/04/2026
Avid today announced it will showcase new innovations designed to help media companies win in the new media economy at NAB Show 2026 (April 18 22, Las Vegas Co...
02/04/2026
PlayBox Neo helps AIS PLAY kick-off premier football content direct to fans
PlayBox Neo has provided MIMO Tech with a brand-new major installation to extend it...
02/04/2026
Globo has transitioned its primary content distribution to Secure Reliable Transport over a fully IP-based managed backbone using Synamedia's Quortex PowerV...
02/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/04/2026
Student Spotlight: Al-Fadl Salem The Danish singer recently performed for the queen of Denmark.
April 1, 2026
By
Editorial Staff
Image by Junia Morrow
Wh...
02/04/2026
Taku Hirano's Career Is Defined by Identity Whether he's performing, composing, teaching, or developing instruments, the do-it-all percussionist sees ...
02/04/2026
Back to All News
Netflix Unveils Official Trailer for The Trials of Winnie Mandela
Entertainment
02 April 2026
GlobalSouth Africa
Link copied to clipboard
...
02/04/2026
Back to All News
Netflix Announces Start of Production and New Cast for Second ...
02/04/2026
Back to All News
The Trailer for Ronaldinho: The One and Only Features Never-Be...
02/04/2026
Red Seat Ventures Launches Speakeasy, a New Hosting, Distribution and Monetizati...
02/04/2026
GoVertical! AiDi Powers Real-Time 9:16 Autocropping for Immediate Streaming to M...