
NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).
The open model combined with NVIDIA accelerated computing equips developers, researchers and businesses to innovate responsibly across a wide variety of applications.
Trained on NVIDIA AI Meta engineers trained Llama 3 on computer clusters packing 24,576 NVIDIA H100 Tensor Core GPUs, linked with RoCE and NVIDIA Quantum-2 InfiniBand networks.
To further advance the state of the art in generative AI, Meta recently described plans to scale its infrastructure to 350,000 H100 GPUs.
Putting Llama 3 to Work Versions of Llama 3, accelerated on NVIDIA GPUs, are available today for use in the cloud, data center, edge and PC.
From a browser, developers can try Llama 3 at ai.nvidia.com. It's packaged as an NVIDIA NIM microservice with a standard application programming interface that can be deployed anywhere.
Businesses can fine-tune Llama 3 with their data using NVIDIA NeMo, an open-source framework for LLMs that's part of the secure, supported NVIDIA AI Enterprise platform. Custom models can be optimized for inference with NVIDIA TensorRT-LLM and deployed with NVIDIA Triton Inference Server.
Taking Llama 3 to Devices and PCs Llama 3 also runs on NVIDIA Jetson Orin for robotics and edge computing devices, creating interactive agents like those in the Jetson AI Lab.
What's more, NVIDIA RTX and GeForce RTX GPUs for workstations and PCs speed inference on Llama 3. These systems give developers a target of more than 100 million NVIDIA-accelerated systems worldwide.
Get Optimal Performance with Llama 3 Best practices in deploying an LLM for a chatbot involves a balance of low latency, good reading speed and optimal GPU use to reduce costs.
Such a service needs to deliver tokens - the rough equivalent of words to an LLM - at about twice a user's reading speed which is about 10 tokens/second.
Applying these metrics, a single NVIDIA H200 Tensor Core GPU generated about 3,000 tokens/second - enough to serve about 300 simultaneous users - in an initial test using the version of Llama 3 with 70 billion parameters.
That means a single NVIDIA HGX server with eight H200 GPUs could deliver 24,000 tokens/second, further optimizing costs by supporting more than 2,400 users at the same time.
For edge devices, the version of Llama 3 with eight billion parameters generated up to 40 tokens/second on Jetson AGX Orin and 15 tokens/second on Jetson Orin Nano.
Advancing Community Models An active open-source contributor, NVIDIA is committed to optimizing community software that helps users address their toughest challenges. Open-source models also promote AI transparency and let users broadly share work on AI safety and resilience.
Learn more about how NVIDIA's AI inference platform, including how NIM, TensorRT-LLM and Triton use state-of-the-art techniques such as low-rank adaptation to accelerate the latest LLMs.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
01/07/2026
Broadcast Management Group (BMG) has announced the appointment of Kathy Samuels ...
01/07/2026
Shade has announced Custom Objects and Automations, a platform expansion releasing June 29, 2026, that adds database and workflow automation capabilities direct...
01/07/2026
FOR-A America has announced the addition of Jaz Wray and Fernando Cruz to its U.S. sales team. Both report to Ernie Leon, Senior VP and Head of Sales and Strate...
01/07/2026
NBC Sports will air all 15 MLB games nationally on Sunday, July 5, across NBC, P...
01/07/2026
Clear-Com has announced a wireless communications upgrade for Jeopardy! and Wheel of Fortune, deploying FreeSpeak II and FreeSpeak Icon systems across both prod...
01/07/2026
England's performance team will use Sony's STATSports APEX GPS tracking system to monitor player physical data in real time during FIFA World Cup 2026 m...
01/07/2026
Adder Technology has announced the appointment of Neil Hillier as Chief Executive Officer, effective July 1, 2026. Hillier succeeds Adrian Dickens, who transiti...
01/07/2026
Bitcentral, Inc. has announced a strategic transaction creating two separate companies. The Production and Playout business will continue as Bitcentral, now own...
01/07/2026
DAZN has announced results from DAZN48, its creator initiative for the FIFA World Cup 2026. Launched in April 2026, the program received thousands of applicatio...
01/07/2026
Sarah Rose, VP, global services, Daktronics (NASDAQ: DAKT), will be inducted into the Information Display and Entertainment Association (IDEA) Hall of Fame at t...
01/07/2026
Gravity Media and the World Economic Forum's production team provided broadc...
01/07/2026
Insight Productions has announced the launch of Insight Storm, a 53-foot mobile broadcast unit built for esports production. The truck is built around a Ross Vi...
01/07/2026
ESPN has announced several content initiatives marking America's 250th anniversary, as part of The Walt Disney Company's Disney Celebrates America pro...
01/07/2026
Eleven production kits, REMI workflows, and cloud distribution bring 40 World Cu...
01/07/2026
The conference also discussed the opportunities offered an industry that is endu...
01/07/2026
The Mountain West Conference has announced the launch of MW , a direct-to-consumer streaming platform powered by Kiswe. The platform will carry live Mountain We...
01/07/2026
New AI Assistant, Multi-channel Audio, ARA2 improvements & more
Tracktion's DAW software has just received its latest major update, gaining a selection ...
01/07/2026
Stammering, stuttering, strangulated tones
The Crow Hill Company's latest creation promises to be the most original sound set they've produced to d...
01/07/2026
Exclusive run of limited-edition modelling pedals
Sweetwater and Andertons M...
01/07/2026
The National Film and Video Foundation (NFVF) is pleased to announce that the ca...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Manfrotto Introduces UNCOVER, the new premium camera bag collection for modern h...
01/07/2026
Blackmagic Design Powers Houston Tamil Sangam Literacy Competition
Brie Clayton July 1, 2026
0 Comments
Volunteers use ATEM Mini Pro, Blackmagic Desig...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
PlayBox Technology has published State of Broadcast Infrastructure 2026, an in-depth industry research report examining the technologies, operational challenges...
01/07/2026
LONDON, UK, 1 JULY Jigsaw24 has appointed Alan Henry as Head of Sales for Media and Entertainment, reinforcing its continued investment in helping broadcaster...
01/07/2026
Content Vault, the patent-pending secure content distribution platform protecting high-value media from disclosures, theft and unauthorised access, today announ...
01/07/2026
Bitcentral, Inc. a leading provider of enterprise software and digital media solutions for news, sports and entertainment broadcasters, as well as streaming pla...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/07/2026
Groundbreaking First Nations Screen Business Accelerator launched through nation...
01/07/2026
Chyron Launches the All-New Chyron Academy: A Reimagined, Hands-On Learning Expe...
01/07/2026
Amplium Captures Kawasaki Brave Thunders Game with Blackmagic URSA Cine Immersiv...
01/07/2026
Boris FX Optics Expands Plugin Support to Apple Photos, Capture One, and Affinit...
01/07/2026
UKTV has today announced the appointment of Matt Berry to the newly created role of General Manager - Marketing, effective 1 July.
Matt will take on this senio...
01/07/2026
Sky Zero Footprint Fund-backed TV campaign featuring Deborah Meaden challenges consumers to rethink everyday bathroom wasteWednesday 1 July 2026
Fussy asks Bri...
01/07/2026
Constituency-level analysis reveals where girls miss out most on sport - and where targeted action could unlock more than £640 million in economic and health be...
01/07/2026
Wuppertal July 1, 2026
Riedel and SKAARHOJ Expand Collaboration With SimplyLive IntegrationRiedel Communications today announced an expanded collaboration wit...
01/07/2026
Apple today introduced power-packed updates to Apple Creator Studio, a groundbre...