Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
04/06/2026
American Underground (AU), the Startup Hub of the South and a community of mor...
04/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/06/2026
NVIDIA and Microsoft Reinvent Windows PCs for the Age of Personal AI
Brie Clayton June 3, 2026
0 Comments
RTX Spark - a 1-Petaflop Superchip, the Full...
04/06/2026
Inside the Peaky Blinders: The Immortal Man Grade
Brie Clayton June 3, 2026
0 Comments
Simone Grattarola discusses shaping the look in DaVinci Resolve...
04/06/2026
Cine Gear Expo Announces 2026 Awards of Excellence Recipients
Brie Clayton June 3, 2026
0 Comments
Ed Lachman ASC, Caleb Deschanel ASC, and M. David M...
04/06/2026
St. Vincents Live with Orchestra Tour to Feature Berklee Alum Ruby Plume Berklee alumni St. Vincent and Ruby Plume will appear on the same bill across seven d...
04/06/2026
BackStory follows four Irish young people as they travel back to their parents' homelands
Modern Irish identity is enriched by cultures and influences from...
04/06/2026
June's forecast with GeForce NOW: 100% chance of gaming.
GeForce NOW is lining up new adventures for the month, from big-name blockbusters to quirky indies...
03/06/2026
SES and Viva, Mexico's ultra low-cost airline, have launched multi-orbit satellite inflight connectivity on Viva's Airbus aircraft. A total of 60 A320s ...
03/06/2026
The College Football Playoff, ESPN, and TNT Sports have announced kick times and...
03/06/2026
RED Digital Cinema will exhibit at Cine Gear Expo 2026 (Booth 33, June 5-6, Universal Studios Lot), hosting three panels and hands-on product demonstrations.
P...
03/06/2026
Roku has announced the Soccer Zone, a dedicated hub for FIFA World Cup 2026 content available across the United States, Canada, Mexico, Brazil, Colombia, Argent...
03/06/2026
Liverpool FC and Wasabi Technologies have announced a multi-year extension of their partnership, with Wasabi continuing as the club's official cloud storage...
03/06/2026
Grass Valley has announced that Australian News Channel (ANC), operator of Sky News Australia, has deployed Grass Valley AMPP as part of the relocation of its n...
03/06/2026
The Riedel Group has announced the appointment of Gudrun Scharler as CEO of Riedel Networks. She succeeds Michael Martens, who has led Riedel Networks since 201...
03/06/2026
Clear-Com has announced the deployment of its intercom solutions at Itaka Arena ...
03/06/2026
Scott Coker has announced six senior executive appointments for his new global MMA promotion, which launched earlier this year with $60 million in financing. Ad...
03/06/2026
Telemundo, the exclusive Spanish-language home of the FIFA World Cup 2026, has announced its digital and social media programming for the tournament, running Ju...
03/06/2026
The Bundesliga has announced the launch of Captain, an AI assistant built into t...
03/06/2026
Anthony James Partners (AJP) is serving as technology consultant for a moderniza...
03/06/2026
Audio-Technica has announced the recipients of its annual sales rep firm awards for the 2025-26 fiscal year. The awards were presented by Jim Schanz, Executive ...
03/06/2026
iodyne will exhibit at Cine Gear Expo 2026 alongside RED Digital Cinema and Adob...
03/06/2026
The broadcaster will deploy 39 cameras for game coverage, six cameras for studio...
03/06/2026
The 2026 FIFA World Cup is providing a great opportunity for not only Fox Sports...
03/06/2026
Sony Electronics is introducing the SRG-AS10, a 4K 60p compatible PTZ Auto Framing camera that uses Sony's proprietary AI to automatically recognize and tra...
03/06/2026
Game Creek Video's Flagship A, B, C, and D unit will wind up its first year ...
03/06/2026
According to Caretta Research, the sports-rights market may be hitting the brake...
03/06/2026
Retrieval still courtesy of Tribeca.
By Jessica Herndon
This year, the Tribeca...
03/06/2026
Step sequencer-style panning tool revealed
Alongside their flagship self-titled sound-design platform, Sound Particles offer an array of creative effects an...
03/06/2026
Now features full H90 algorithm library
Eventide have announced the upcoming launch of the H9 Harmonizer Gen 2, a new and improved version of their hugely p...
03/06/2026
Significant discount available until 1 October 2026
Aim Audio have just announced a promotion that sees a significant discount applied to their Essence micr...
03/06/2026
Clarification: SBS's position on definitions of antisemitism
3 June, 2026
Media releases
Statement by Mandi Wicks, SBS Director of News and Current Aff...
03/06/2026
Rohde & Schwarz to supply CERTIUM advanced communications system to Memmingen Ai...
03/06/2026
eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...
03/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/06/2026
For more than three decades, Re-recording Mixer Andrew Wilson, AMPS, CAS, has helped bring the natural world to the screen with exceptional audio enjoyed by mil...
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
