Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
13/11/2026
The UIM E1 Series wraps its third season this week with the Bombay Sapphire E1 Miami GP, Nov. 11-14, on Biscayne Bay at the Miami Seaplane Base on Watson Island...
13/11/2026
In-venue and creative video staffers at the professional and collegiate level ha...
13/11/2026
Singer-songwriter Benson Boone will headline entertainment for the Discover NHL Winter Classic between the Utah Mammoth and Colorado Avalanche at Rice-Eccles St...
13/11/2026
Blackmagic Design has announced the Blackmagic Ambisonic Microphone, a 7th order ambisonic microphone array that captures spatial audio for immersive video prod...
13/11/2026
SMPTE has announced that The Studio B&H and Riedel Communications will provide the video infrastructure and communications technology for the Solutions Hub at t...
13/11/2026
The Hollywood Professional Association (HPA) has announced the theme and program focus for the 2027 HPA Tech Retreat, which takes place Feb. 14-18, 2027, in La ...
13/11/2026
Football Manager 27 Touch is available on Apple Arcade starting today. The game ...
13/11/2026
MotoGP has announced a two-year media rights agreement with ESPN covering the Un...
13/11/2026
Apple Sports, the free iPhone app, has added the Ultimate Fighting Championship (UFC) to its lineup of supported sports, in time for Saturday's UFC 334. The...
13/11/2026
The NBA and Fanatics Studios, the joint venture between Fanatics and entertainme...
13/11/2026
ESPN held its annual EDGE Innovation Conference last night and offered up an ove...
13/11/2026
Regional sports network gives Ithaca College, SUNY Cortland students a platform to thrive
One of the biggest games in Division III college football will be pla...
13/11/2026
Studer-inspired 500-series PWM compressor unveiled
The HLabs range already includes a couple of Studer-inspired offerings in the form of the PRE169 and EQ16...
13/11/2026
Download Now!
To view the white paper, please enter the following details:
--
First Name* First Name
Company*
Country* Please select your countryA...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Blue Lucy, the workflow orchestration and integration platform for broadcasters, rights owners and localisation companies, today announced an integration with c...
13/11/2026
Krotos announces Black Friday offers on Studio Pro and Studio Max
Brie Clayton November 12, 2026
0 Comments
First-year annual plans start at US$195.99...
13/11/2026
Blackmagic Design Announces Blackmagic Ambisonic Microphone
Brie Clayton November 12, 2026
0 Comments
New 7th order ambisonic microphone with 108 MEMS...
13/11/2026
Meet the Berklee Student Setting Planetary Motion to Music With his compositional platform SoniKosmos, Javier Roqueni aims to make the universe the ultimate i...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , er...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , er...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Amagi, the media industry cloud company, today announced a partnership with Golden Times Media Technology (stock code 920021.BJ), a Beijing-based media and tech...
13/11/2026
CGI (TSX: GIB.A) (NYSE: GIB), one of the largest independent IT and business consulting services firms in the world, together with its Malaysian partner, Cedar ...
13/11/2026
SMPTE, the home for media professionals, technologists and engineers, today announced that The Studio B&H and Riedel Communications will provide the video infra...
13/11/2026
iWedia, a global leader in connected TV software solutions, will present its software-first approach to DRAM optimization at Android TV APAC Summit 2026 in Bang...
13/11/2026
Beginning today, November 13, Apple HomeView is available for purchase at Apple Store locations, on the Apple Store online, and in the Apple Store app in select...
13/11/2026
From pediatric research to smart hospitals, AI is helping care teams spend less time on routine tasks and more time with patients.
Health systems and startups...
12/11/2026
Industry leaders examine how Dynamic Media Facility, Media eXchange Layer, and c...
12/11/2026
The junior Tar Heel is embracing the technical and creative challenges of live production while emerging as a leader and mentor in the control room...
12/11/2026
Gracenote, Nielsen's content intelligence business, has released its fifth a...
12/11/2026
RTL Deutschland will bring the NHL to free-to-air TV in Germany starting Nov. 30...
12/11/2026
Clear-Com supported On Air 2026, a 24-hour student-led broadcast connecting 45 u...
12/11/2026
uniqFEED has opened a Remote Operations Center (ROC) in Madrid, Spain. It is already operational, delivering virtual advertising for LaLiga matches. The ROC is ...
12/11/2026
Disney, ESPN and the NBA will present Dunk the Halls, the third real-time animated NBA game using Sony's Beyond Sports data visualization technology, on Chr...
12/11/2026
Van Wagner has launched Van Wagner RELAY, a digital platform for official athlet...
12/11/2026
Deal brings together two of the industry's most established graphics brands and significantly expands Vizrt's footprint in U.S. sports and regional broa...
12/11/2026
Euro TV Production GmbH of Austria has upgraded its outside broadcast vehicle with Lawo technology. The mc 36 Classic console in service since 2019 was replaced...
12/11/2026
Tagboard has launched Spark Resize, an AI-assisted tool now in beta that takes an existing graphic and produces a new version for a different aspect ratio or sc...
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
