
NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).
The open model combined with NVIDIA accelerated computing equips developers, researchers and businesses to innovate responsibly across a wide variety of applications.
Trained on NVIDIA AI Meta engineers trained Llama 3 on computer clusters packing 24,576 NVIDIA H100 Tensor Core GPUs, linked with RoCE and NVIDIA Quantum-2 InfiniBand networks.
To further advance the state of the art in generative AI, Meta recently described plans to scale its infrastructure to 350,000 H100 GPUs.
Putting Llama 3 to Work Versions of Llama 3, accelerated on NVIDIA GPUs, are available today for use in the cloud, data center, edge and PC.
From a browser, developers can try Llama 3 at ai.nvidia.com. It's packaged as an NVIDIA NIM microservice with a standard application programming interface that can be deployed anywhere.
Businesses can fine-tune Llama 3 with their data using NVIDIA NeMo, an open-source framework for LLMs that's part of the secure, supported NVIDIA AI Enterprise platform. Custom models can be optimized for inference with NVIDIA TensorRT-LLM and deployed with NVIDIA Triton Inference Server.
Taking Llama 3 to Devices and PCs Llama 3 also runs on NVIDIA Jetson Orin for robotics and edge computing devices, creating interactive agents like those in the Jetson AI Lab.
What's more, NVIDIA RTX and GeForce RTX GPUs for workstations and PCs speed inference on Llama 3. These systems give developers a target of more than 100 million NVIDIA-accelerated systems worldwide.
Get Optimal Performance with Llama 3 Best practices in deploying an LLM for a chatbot involves a balance of low latency, good reading speed and optimal GPU use to reduce costs.
Such a service needs to deliver tokens - the rough equivalent of words to an LLM - at about twice a user's reading speed which is about 10 tokens/second.
Applying these metrics, a single NVIDIA H200 Tensor Core GPU generated about 3,000 tokens/second - enough to serve about 300 simultaneous users - in an initial test using the version of Llama 3 with 70 billion parameters.
That means a single NVIDIA HGX server with eight H200 GPUs could deliver 24,000 tokens/second, further optimizing costs by supporting more than 2,400 users at the same time.
For edge devices, the version of Llama 3 with eight billion parameters generated up to 40 tokens/second on Jetson AGX Orin and 15 tokens/second on Jetson Orin Nano.
Advancing Community Models An active open-source contributor, NVIDIA is committed to optimizing community software that helps users address their toughest challenges. Open-source models also promote AI transparency and let users broadly share work on AI safety and resilience.
Learn more about how NVIDIA's AI inference platform, including how NIM, TensorRT-LLM and Triton use state-of-the-art techniques such as low-rank adaptation to accelerate the latest LLMs.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
20/11/2026
The sports broadcaster's chairman sees the upcoming game as an opportunity t...
20/11/2026
In-venue and creative video staffers at the professional and collegiate level have one major thing in common: the intensity and attention to detail ramps up dur...
20/11/2026
A variety of LED displays, integrated broadcast infrastructure covers every angl...
20/11/2026
Real American Freestyle (RAF) has announced a distribution partnership with Migu Video, a Chinese digital sports platform, to carry the RAF15 main event between...
20/11/2026
Apple and F1 Academy have announced that the all-female racing series will stream on Apple TV in the U.S. beginning in 2027. Fans will be able to watch every qu...
20/11/2026
Regional broadcaster m nchen.tv has modernized its production infrastructure at a new location in northern Munich. Riedel MediorNet provides decentralized signa...
20/11/2026
Eutelsat has announced three appointments to its Executive Committee, effective November 2026, under CEO Jean-Fran ois Fallacher.
Jean-Hubert Lenotte is appoin...
20/11/2026
After more than two decades at Harbour Esplanade in Docklands, the Seven Network has relocated its Melbourne headquarters to a new, custom-built ST 2110 facilit...
20/11/2026
Brazil is among the first markets deploying Versatile Video Coding (VVC) as part...
20/11/2026
The Soccer Tournament (TST), the $1 million winner-take-all 7v7 world championsh...
20/11/2026
Enhancements include a larger compound, an elevated camera count, and a whole lo...
20/11/2026
Leaders from ESPN, iHeartMedia, and Just Women's Sports explore creating new...
20/11/2026
New EQ plug-in aimed at mastering duties
Lemur Audio's latest plug-in has just arrived, delivering a characterful EQ that's aimed primarily at maste...
20/11/2026
Discounts of up to 75% now available
Soundtoys' annual Black Friday sale is now live, with discounts of up to 75% applied across the company's range...
20/11/2026
New creative looping tool revealed
Described as a cycling audio fragmenter , the latest addition to the SoundGhost range draws its inspiration from tape ph...
20/11/2026
Fairchild recreation gets plug-in counterpart
Heritage Audio's Herchild 670 offers a meticulous hardware recreations of the much-loved Fairchild 670 and...
20/11/2026
BONDI: 7 Minutes That Changed Australia
16 November, 2026
Media releases
A landmark documentary chronicling the worst terror attack in modern Australian hi...
20/11/2026
Statement regarding SBS non-Executive Director Aaron Fa'Aoso
20 November, 2026
Media releases
Aaron Fa'Aoso has advised that he is stepping down a...
20/11/2026
INVITATION FOR PUBLIC COMMENTS AND OBJECTIONS ON NOMINEES FOR APPOINTMENT TO THE COUNCIL OF THE NATIONAL FILM AND VIDEO FOUNDATION (NFVF) The Department of Spor...
20/11/2026
UEFA Nations League football and autumn premieres propel TVP to number one as YouTube grows its TV screen share
Warsaw, Poland, November 19, 2026 - Nielsen, a...
20/11/2026
October's autumn weather clearly encouraged Poles to spend more time in front of their screens, resulting in both longer viewing times and a higher overall ...
20/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
20/11/2026
DigitalGlue releases creative.space Pulse, a free Mac app that shows creatives w...
20/11/2026
How NBCUniversal Telemundo Used Virtual Production for its FIFA World Cup 2026 C...
20/11/2026
What Film School Pays and What it Costs: The Federal Numbers
Mark Santos November 20, 2026
0 Comments
Every editor has had the argument about film sch...
20/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
20/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
20/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
20/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
20/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
20/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
20/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
20/11/2026
Beam Dynamics has introduced new Model Context Protocol (MCP) support, expanding how customers can work with the live operational data held in Beam. The new cap...
20/11/2026
Re:Vision Effects Cyber Week Sale Nov 24 to Nov 30
Pierre Jasmin November 19, 2026
0 Comments
Long time friends of the COW are up with a big sale!
Us...
20/11/2026
Avid is offering up to 33% off annual subscriptions!
Brie Clayton November 19, 2026
0 Comments
Hero image courtesy Deposit Photos
Pro Tools
Pro To...
19/11/2026
With a sports-bar vibe and all the teams onsite in a huge atrium in Ballpark Vil...
19/11/2026
The senior from Phoenix and England is balancing life as a Division I tennis player with a growing career behind the microphone
The future of live sports produ...
19/11/2026
RED Digital Cinema and Colorfront have partnered to integrate native Colorfront color management into RED Connect, letting productions apply Colorfront's co...
19/11/2026
SMPTE has published an engineering report summarizing an industry survey by its Study Group on Content Provenance and Authenticity in Media. The findings point ...
19/11/2026
CTech and Skyline Communications have announced a partnership to deliver integra...
19/11/2026
Euskal Irrati Telebista (EITB), the Basque Country's public media group, has completed Phase 1 of a five-studio renewal with the commissioning of a Lawo IP ...
19/11/2026
The Esports Foundation (EF) has announced an expanded Road to ENC 2027 for the E...
19/11/2026
All Women's Sports Network (AWSN) held its first awards show, VEON: Women W...
19/11/2026
Gracenote, the content intelligence business unit of Nielsen, has extended its relationship with DIRECTV. Gracenote will continue to supply visual imagery and p...
19/11/2026
The Audio Engineering Society (AES) Technical Committee on Broadcast and Online ...
19/11/2026
NDI has released NDI 6.4, which adds Recording and Playback APIs, support for HEVC 4:4:4 and a configurable receiver buffer for Wi-Fi. The update also includes ...
19/11/2026
Lemonlight, the video production company behind more than 50,000 brand videos, has announced Hero, an AI production operating system, along with a new AI agent ...
19/11/2026
In a first-of-its-kind deployment at XFINITY Center, the digital signage/display...
19/11/2026
For the first time on a US streaming service, real-time player identification, contextual stats, and visual insights appear right in the live stream, available ...