NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).
The open model combined with NVIDIA accelerated computing equips developers, researchers and businesses to innovate responsibly across a wide variety of applications.
Trained on NVIDIA AI Meta engineers trained Llama 3 on computer clusters packing 24,576 NVIDIA H100 Tensor Core GPUs, linked with RoCE and NVIDIA Quantum-2 InfiniBand networks.
To further advance the state of the art in generative AI, Meta recently described plans to scale its infrastructure to 350,000 H100 GPUs.
Putting Llama 3 to Work Versions of Llama 3, accelerated on NVIDIA GPUs, are available today for use in the cloud, data center, edge and PC.
From a browser, developers can try Llama 3 at ai.nvidia.com. It's packaged as an NVIDIA NIM microservice with a standard application programming interface that can be deployed anywhere.
Businesses can fine-tune Llama 3 with their data using NVIDIA NeMo, an open-source framework for LLMs that's part of the secure, supported NVIDIA AI Enterprise platform. Custom models can be optimized for inference with NVIDIA TensorRT-LLM and deployed with NVIDIA Triton Inference Server.
Taking Llama 3 to Devices and PCs Llama 3 also runs on NVIDIA Jetson Orin for robotics and edge computing devices, creating interactive agents like those in the Jetson AI Lab.
What's more, NVIDIA RTX and GeForce RTX GPUs for workstations and PCs speed inference on Llama 3. These systems give developers a target of more than 100 million NVIDIA-accelerated systems worldwide.
Get Optimal Performance with Llama 3 Best practices in deploying an LLM for a chatbot involves a balance of low latency, good reading speed and optimal GPU use to reduce costs.
Such a service needs to deliver tokens - the rough equivalent of words to an LLM - at about twice a user's reading speed which is about 10 tokens/second.
Applying these metrics, a single NVIDIA H200 Tensor Core GPU generated about 3,000 tokens/second - enough to serve about 300 simultaneous users - in an initial test using the version of Llama 3 with 70 billion parameters.
That means a single NVIDIA HGX server with eight H200 GPUs could deliver 24,000 tokens/second, further optimizing costs by supporting more than 2,400 users at the same time.
For edge devices, the version of Llama 3 with eight billion parameters generated up to 40 tokens/second on Jetson AGX Orin and 15 tokens/second on Jetson Orin Nano.
Advancing Community Models An active open-source contributor, NVIDIA is committed to optimizing community software that helps users address their toughest challenges. Open-source models also promote AI transparency and let users broadly share work on AI safety and resilience.
Learn more about how NVIDIA's AI inference platform, including how NIM, TensorRT-LLM and Triton use state-of-the-art techniques such as low-rank adaptation to accelerate the latest LLMs.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
03/07/2026
New bundle & three Single Packs
Continuing to expand their already sizeable orchestral collection, VSL's latest release introduces three new Single Pack...
03/07/2026
Venue installs new ATC-equipped control room
LSO St Luke's, a unique music venue that also serves as the home of the London Symphony Orchestra, have rec...
03/07/2026
Caden Pearson appointed new Commissioning Editor for NITV
3 July, 2026
Media releases
National Indigenous Television (NITV) has strengthened its commitment...
03/07/2026
1 February 2023
SHARE Facebook Twitter Linkedin Email
Munich, Germany, 1st February 2023: Cinegy GmbH, the premier provider of software technology for digit...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Death Star
Andy Marken July 2, 2026
0 Comments
Look, I can't get involved. I've got work to do. It's not that I like the Empire; I hate i...
03/07/2026
Berklee's New Visual Identity: Honoring Our History, Building for What's...
03/07/2026
Scripps Research scientists awarded $2M to advance global disease surveillance Two Gates Foundation grants will expand wastewater surveillance and AI-driven dis...
03/07/2026
Joan Pulupa joins Scripps Research faculty to study the organization of DNA in brain cells and its links to neurodegeneration Using smell-sensing neurons and ad...
02/07/2026
Entering her senior year, this hometown girl is paving a career in live sports production gaining experience in replay and audio and as a TD
In the live-sports...
02/07/2026
In-venue and creative video staffers at the professional and collegiate level ha...
02/07/2026
BLAST, a competitive entertainment company focused on esports, has announced more than $133 million in revenue for 2025, representing more than 40% year-over-ye...
02/07/2026
Riedel Communications has announced official SKAARHOJ panel support for SimplyLive production workflows, enabled through the SimplyLive 2.1 release. The integra...
02/07/2026
The Fire Rescue Service of the Czech Republic has deployed LiveU video-over-bond...
02/07/2026
Gravity Media USA has announced the appointment of Brittney Boston as Head of Business Development, effective July 1, 2026. Based in Nashville, Tennessee, Bosto...
02/07/2026
TwelveLabs, a video intelligence company, has announced $100 million in Series B funding co-led by NEA and NAVER Ventures, with participation from Amazon, Radic...
02/07/2026
The Pro Padel League (PPL) has announced a broadcast partnership with USA Sports that will air five PPL championship matches on CNBC during the 2026 season, the...
02/07/2026
LiveLike, a digital fan engagement platform, has announced eight confirmed FIFA ...
02/07/2026
Cobalt Digital has received Future's Best of Show Award, presented by AV Technology at InfoComm 2026, for its blueCORE family of standalone signal processor...
02/07/2026
Synamedia has announced the appointment of Dr. Tzvi Gerstl as Chief Executive Officer. Paul Segre, who has served as CEO for the past six years, will transition...
02/07/2026
The Esports Foundation (EF) and Sony Group Corporation have announced an expanded collaboration for the Esports World Cup 2026 (EWC), taking place in Paris, Fra...
02/07/2026
Zee Entertainment Enterprises Ltd. ( Z') has announced exclusive broadcast and digital rights for the Bundesliga in India for five years, beginning with the...
02/07/2026
An effort uniting News, Sports, Local, and Telemundo, the 50+-camera live produc...
02/07/2026
Zoey Deutch, John Slattery, Ken Marino, Miles Gutierrez-Riley, and Ben Wang appe...
02/07/2026
Stammering, stuttering, strangulated tones
The Crow Hill Company's latest creation promises to be the most original sound set they've produced to d...
02/07/2026
A new era in unmixing and spectral editing
The latest version of Steinberg's spectral audio-editing software has just arrived, building on the strength...
02/07/2026
Aims to simplify additive synthesis
Sine Machine is the debut launch from Melatonin, a Vienna-based developer who have spent the past six years creating wha...
02/07/2026
Products to remain fully active & supported
Following the news of Native Instruments joining the inMusic brand line-up, Academy and Emmy Award-winning visua...
02/07/2026
What you missed!
Last weekend, Saturday 27 June 2026, saw the debut of Sound On Sounds new GearExpo UK event, the largest dedicated pro-audio event to take ...
02/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/07/2026
Following the successful launch of its inaugural APAC Mentoring Programme last month, the Rise AV APAC Regional Council will bring the conversation around mento...
02/07/2026
Blackmagic PYXIS 6K Used to Shoot Director Takahisa Zeze's Cry Out
Brie Clayton July 2, 2026
0 Comments
Highly mobile camera supports tense and de...
02/07/2026
Broadcast Solutions acquires BFE, expanding its lead in European broadcast, medi...
02/07/2026
Berklee Alum and Faculty Perform at Boston Public Library's 250th Anniversar...
02/07/2026
Broadcast Solutions GmbH, a leading systems integrator and provider of innovative solutions for the broadcast media industry, is acquiring BFE Studio und Medien...
02/07/2026
Cinegy GmbH, the premier provider of software-defined television technology, has extended the ingest facility at leading Brazilian sports company LiveMode, work...
02/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/07/2026
Standalone processors acknowledged for the innovation and value they bring to Pro AV
Cobalt Digital, a leading designer and manufacturer of signal processing ...
02/07/2026
Synamedia announced today the appointment of Dr Tzvi Gerstl as Chief Executive Officer. Paul Segre, who has served as CEO for the past six years, will transitio...
02/07/2026
Screen Australia backs audience-led filmmaking with new insight-driven initiativ...
02/07/2026
Screen Australia refines guidelines for Narrative Content Development and Docume...
02/07/2026
Maxon Autograph: Introduction to working with Tables
Simon Ubsdell July 1, 2026
0 Comments
An overview of Autograph's ridiculously powerful tables...
02/07/2026
Boston Conservatory's Soir e Breaks Records to Fund Student Scholarships The event achieved 127 percent of its fundraising goal in an evening celebrating ...
NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).
