NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).
The open model combined with NVIDIA accelerated computing equips developers, researchers and businesses to innovate responsibly across a wide variety of applications.
Trained on NVIDIA AI Meta engineers trained Llama 3 on computer clusters packing 24,576 NVIDIA H100 Tensor Core GPUs, linked with RoCE and NVIDIA Quantum-2 InfiniBand networks.
To further advance the state of the art in generative AI, Meta recently described plans to scale its infrastructure to 350,000 H100 GPUs.
Putting Llama 3 to Work Versions of Llama 3, accelerated on NVIDIA GPUs, are available today for use in the cloud, data center, edge and PC.
From a browser, developers can try Llama 3 at ai.nvidia.com. It's packaged as an NVIDIA NIM microservice with a standard application programming interface that can be deployed anywhere.
Businesses can fine-tune Llama 3 with their data using NVIDIA NeMo, an open-source framework for LLMs that's part of the secure, supported NVIDIA AI Enterprise platform. Custom models can be optimized for inference with NVIDIA TensorRT-LLM and deployed with NVIDIA Triton Inference Server.
Taking Llama 3 to Devices and PCs Llama 3 also runs on NVIDIA Jetson Orin for robotics and edge computing devices, creating interactive agents like those in the Jetson AI Lab.
What's more, NVIDIA RTX and GeForce RTX GPUs for workstations and PCs speed inference on Llama 3. These systems give developers a target of more than 100 million NVIDIA-accelerated systems worldwide.
Get Optimal Performance with Llama 3 Best practices in deploying an LLM for a chatbot involves a balance of low latency, good reading speed and optimal GPU use to reduce costs.
Such a service needs to deliver tokens - the rough equivalent of words to an LLM - at about twice a user's reading speed which is about 10 tokens/second.
Applying these metrics, a single NVIDIA H200 Tensor Core GPU generated about 3,000 tokens/second - enough to serve about 300 simultaneous users - in an initial test using the version of Llama 3 with 70 billion parameters.
That means a single NVIDIA HGX server with eight H200 GPUs could deliver 24,000 tokens/second, further optimizing costs by supporting more than 2,400 users at the same time.
For edge devices, the version of Llama 3 with eight billion parameters generated up to 40 tokens/second on Jetson AGX Orin and 15 tokens/second on Jetson Orin Nano.
Advancing Community Models An active open-source contributor, NVIDIA is committed to optimizing community software that helps users address their toughest challenges. Open-source models also promote AI transparency and let users broadly share work on AI safety and resilience.
Learn more about how NVIDIA's AI inference platform, including how NIM, TensorRT-LLM and Triton use state-of-the-art techniques such as low-rank adaptation to accelerate the latest LLMs.
Most recent headlines
02/01/2026
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
02/01/2026
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
02/01/2026
Sequins, chat shows, live sporting action, ground-breaking docuseries and brand-new Irish drama to kick off 2026
New Year, New Content Coming Soon across RT
...
01/01/2026
Latin Grammy Cultural Foundation Announces 2026 Noel Schajris Scholarship The scholarship will cover tuition and housing for one Berklee College of Music stud...
01/01/2026
New year, new games, all with RTX 5080-powered cloud energy. GeForce NOW is kicking off 2026 by looking back at an unforgettable year of wins and wildly high fr...
31/12/2025
Back to All News
NFL Christmas Gameday on Netflix Scores Again With the Lions-V...
30/12/2025
As the College Football Playoff Enters the Quarterfinals, ESPN Blows Out Its Meg...
30/12/2025
SVG's Best of 2025: Original ArticlesTake a look back at all our coverage of big-time productions, game-changing technologies, and state-of-the-art new faci...
30/12/2025
MELBOURNE, Fla., Dec. 30, 2025 - L3Harris Technologies (NYSE: LHX) will release its fourth quarter 2025 financial results before the market opens on Thursday, J...
30/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
30/12/2025
Your live countdown to 2026 with Inhaler, David Gray, Lyra, Garron Noone, Sharon...
30/12/2025
It marked the first civilian operational authorization for a HAPS flight in Europe, led by Space42's subsidiary, Mira Aerospace
The flight demonstrated HAP...
29/12/2025
San Francisco 49ers Strike Gold With Halftime Laser SpectacularStunning display caps $200 million renovation of Levi's Stadium techBy Dan Daley, Audio Edito...
29/12/2025
The Cup's Around the Corner: An Inside Look at Broadcast Preparations for th...
29/12/2025
SVG's Best of 2025: Longform VideoWatch the standout keynote conversations, deep dives, and panel discussions from the year for free on SVG PLAY!By Brandon ...
29/12/2025
From crisper Lossless audio and immersive music videos in beta to new Audiobooks+ plans, custom transitions between tracks, and in-app Messages, we keep levelin...
29/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
27/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
26/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
26/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
25/12/2025
Holiday lights are twinkling, hot cocoa's on the stove and gamers are settling in for a well-earned break.
Whether staying in or heading on a winter getawa...
24/12/2025
What is AI good for? Posted by MTI Film on December 24, 2025
What is AI good for?
What is AI good for?
It's been three years since ChatGPT first cap...
24/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
24/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
24/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
24/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
24/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
24/12/2025
RT has unveiled an exclusive first look at the new Dancing with the Stars promo...
24/12/2025
Back to All News
The Boyfriend' Season 2 Unveils Heartwarming Trailer, Key...
24/12/2025
Back to All News
Love, Fights, and Everything in Between: Badly in Love' Returns for Season 2
Entertainment
24 December 2025
GlobalJapan
Link copied t...
24/12/2025
Scripps Research study links sleep variability with sleep apnea and hypertension How consumers' digital activity trackers could enable personalized health s...
23/12/2025
How guilas Cibae as Dominican Winter League Games Are Locally Produced for Glob...
23/12/2025
BitFire's Jim Akimchuk on Supplying Scalability and Customization in the Clo...
23/12/2025
CAMB.AI Enables European Athletics to Offer Multi-Language SupportPlan is to eventually offer translation into all languages spoken in EuropeBy Ken Kerschbaumer...
23/12/2025
Analysis: As sports media values trend negative, scarcity and quality are king By Callum McCarthy, Editor-at-Large
Monday, December 22, 2025 - 14:08
Print ...
23/12/2025
ESPN, Disney, and NBA Return to the Animated Altcast Fray With Second Edition of...
23/12/2025
End the Year on a High Note and Donate to the Sports Broadcasting Fund Today!By Ken Kerschbaumer, Editorial Director
Tuesday, December 23, 2025 - 12:25 pm
P...
23/12/2025
The year is winding down, the weather outside is frightful, and it's the perfect time to escape into a story that warms the heart. For listeners looking for...
23/12/2025
A Zeus motor is hot fire tested at L3Harris' Camden, Arkansas, solid rocket ...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Lightware will exhibit several major product innovations at ISE 2026, including the new USB-C BOOSTER-V1, Google Meet. integration for various Taurus UCX models...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Taking the Stage at Carnegie Hall-On a Global Scale Boston Conservatory Orchestra students reflect on their epic concert marking the 80th session of the UN Gene...
23/12/2025
Back to All News
Netflix's The Great Flood and Culinary Class Wars 2 Top Gl...
NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).
