
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
01/06/2026
January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026
Throughout the week, Dolby brings to life the latest innovatio...
02/05/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
01/05/2026
January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...
09/04/2026
Staines-upon-Thames, UK, 09, April, 2026 - Yospace, the trusted leader in Dynam...
09/04/2026
just:play pro 2026 and just:live pro 2026 Sneak Preview News for NAB 2026
More Details:At NAB 2026, ToolsOnAir will showcase just:play pro 2026 and just:live p...
09/04/2026
just:in mac pro 2026 - The Next Level of Professional Recording on macOS at NAB ...
09/04/2026
Zixi will demonstrate IP-based live video workflow solutions at NAB Show 2026 (Booth W2057).
The industry is moving quickly toward IP-based distribution as br...
09/04/2026
Global women's elite sports revenues are expected to reach at least $3 billi...
09/04/2026
Monitor engineer Gavin Tempany mixed Kylie Minogue s Tension Tour on a Solid Sta...
09/04/2026
KOKUSAI DENKI Electric America will exhibit at NAB Show 2026 (Booth C5507), debu...
09/04/2026
With the 2025-26 NBA regular season concluded and the playoffs beginning next we...
09/04/2026
Telestream and Mimir have announced an integration connecting Telestream's V...
09/04/2026
Bitmovin has expanded its Live Encoding and Observability solutions to provide r...
09/04/2026
The Nashville Predators and Scripps Sports have announced a multi-year media rights agreement covering local preseason, regular season, and first-round playoff ...
09/04/2026
Advanced Systems Group, LLC has announced a partnership with Beam Dynamics to offer the Beam Asset and License Intelligence Platform to its clients. The platfor...
09/04/2026
Lawo has unveiled Edge One, a combined video and audio stagebox for broadcast and Pro AV workflows. The device will be on display at NAB Show (Booth C2108, Apri...
09/04/2026
The Society of Motion Picture and Television Engineers (SMPTE) will host the SMPTE ST 2110 IP Media Roadshow on Tuesday, April 21, 2026, at the Las Vegas Conven...
09/04/2026
The Atlanta Braves have completed upgrades to video displays in and around Truist Park ahead of the 2026 MLB season. The upgrades include the Delta Out-of-Town ...
09/04/2026
The University of Southern California has contracted Daktronics (NASDAQ: DAKT) of Brookings, South Dakota, to manufacture and install 22 LED displays across fou...
09/04/2026
Backlight, the media technology company behind Iconik and Wildmoka, will showcase its Creative Operations Platform at NAB Show 2026 (Booth N2829, April 19-22). ...
09/04/2026
MotoAmerica and V10 Entertainment have announced a partnership to broadcast MotoAmerica Superbike racing on VICE TV for the 2026 season. Coverage begins live on...
09/04/2026
Proton Camera Innovations has announced the appointment of Tod Musgrave as US Sa...
09/04/2026
Designed specifically for live sports broadcasting, new platform features IP-nat...
09/04/2026
Blending 1990s DNA, modern motion theory, and a distinctly colorful brand identi...
09/04/2026
Technical capability is essential, but long-term success often depends on how we...
09/04/2026
15 feature films, including fiction and documentaries, along with six short film...
09/04/2026
Spotify has always been about putting listeners in the driver's seat. Today, people don't just want more ways to spend their time; they want that time t...
09/04/2026
Our Chief Public Affairs Officer Dustee Jenkins shares how we're building a more positive experience for families on Spotify.
As Spotify's Chief Public...
09/04/2026
Festival season is upon us. From sun-soaked weekends out west to iconic stages in Chicago and New York, fans are getting ready to see their favorite artists liv...
09/04/2026
O Spotify sempre teve como foco colocar os ouvintes no controle. Hoje, as pessoas n o querem apenas mais formas de passar o tempo - elas querem que esse tempo s...
09/04/2026
Read the original note in English here.
Nossa Chief Public Affairs Officer, Dustee Jenkins, compartilha como estamos construindo uma experi ncia mais positiva ...
09/04/2026
New synth focuses on sci-fi scoring
Following their formation and debut releases in December 2025, Smokestack Sounds - the brainchild of composer and produc...
09/04/2026
Latest version set for May 2026 launch
Reason Studios have revealed that the latest version of their DAW software will be launching in May 2026. Currently a...
09/04/2026
New EQ aimed at mix-bus & mastering duties
Shy Audio's first two releases focused on the past, delivering recreations of the budget mixers that were com...
09/04/2026
Based on John Galsworthy's novels known collectively as The Forsyte Saga a...
09/04/2026
The Forsytes has been renewed for a third season before the period drama has even premiered on PBS Masterpiece.
The adaptation of John Galsworthy's novel, ...
09/04/2026
IThe BBC has commissioned new drama The Witch Farm, starring Gabrielle Creevy (T...
09/04/2026
The next big thing To help broadcasters fully embrace dynamic hybrid workflows, Calrec will make a major announcement that unlocks even more freedom for broadca...
09/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
09/04/2026
Purpose Built Monitoring From Live Production to Master Control to OTT, Across On Prem and Cloud Environments
At the 2026 NAB Show (April 19-22, Las Vegas Con...
09/04/2026
Purpose Built Monitoring From Live Production to Master Control to OTT, Across On Prem and Cloud Environments
At the 2026 NAB Show (April 19-22, Las Vegas Con...
09/04/2026
New advances meet surge in demand for broadcast-grade IP migration as C-band spectrum auctions approach
LTN announces major enhancements to its purpose-built g...
09/04/2026
Hitomi Broadcast has expanded its sales team with the addition of Nicola Milburn as Technical Sales Manager. In this role, Nicola will work with customers and p...
09/04/2026
Revolutionary combined solution brings sub-second latency, resilient delivery, and workflow orchestration to global broadcasters and digital platforms
Layercak...
09/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...