
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
North America Stories
16/11/2026
The ESPN Edge Innovation Conference once again gave key ESPN technology partners as a chance to take to the stage and discuss the future of sports fan experienc...
16/11/2026
As rights holders rethink who tells their stories and where, IMG is preparing th...
16/11/2026
From the sidelines, sonic savants advocate for greater emphasis on sound as sports and entertainment increasingly merge
SVG Live!, a one-day event on Nov. 10 i...
16/11/2026
Deal combines Yamaha's audio technology with Disguise's visual-productio...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Hiltron Communications has received a repeat order from one of Europes longest established communications satellite operators. Central to this project is a Hilt...
16/11/2026
Fabric Insights and Fonn Group today announce a partnership to connect editorial planning and media operations across their complementary product portfolios.
...
16/11/2026
Density 1.0 brings a complete color grading workflow to Final Cut Pro
Brie Clayton November 16, 2026
0 Comments
Filmmaker Pascal Basel's plugin pr...
16/11/2026
Berklee Students to Perform at Boston's CITGO Sign Relighting Kuper Walker, Ollie Mann, and dreamcorp. will perform at the free public celebration in Kenm...
16/11/2026
The Hollywood Reporter Names Berklee Among World's Best Music Schools The annual list spotlights Berklee's screen scoring programs and new master'...
16/11/2026
Five years ago, Lightware launched the original Taurus UCX with a clear vision: USB-C would become the central connection in meeting rooms. The goal was simple:...
16/11/2026
Sonniss: bundles over 80% below their libraries' prices, plus up to 60% off ...
16/11/2026
Singaporean Comedy Luck My Life Shot with Blackmagic URSA Cine 17K 65
Brie Clayton November 16, 2026
0 Comments
Blackmagic PYXIS 12K and DaVinci Resol...
16/11/2026
November 16th, 2026
TRIBECA FESTIVAL LISBOA UNVEILS 2026 FILM LINEUP AND TALE...
16/11/2026
Announced today at SC26, NVIDIA Rubin HPC is a new GPU in the NVIDIA Vera Rubin ...
16/11/2026
This week at SC26, NVIDIA is marking two decades of CUDA by honoring the researchers, developers and engineers who helped make GPU computing a foundation for mo...
16/11/2026
With expanded NVIDIA CUDA-X libraries, including cuPhoton, cuPotential and cuSpa, researchers can sift through vast amounts of data to accelerate research, from...
16/11/2026
In its MLPerf benchmarks debut, with results released today, the NVIDIA Vera Rub...
16/11/2026
Quantum processor deployments at supercomputing centers have grown nearly sevenfold over the past five years, with nearly 30 supercomputing centers around the w...
16/11/2026
To train an AI model, thousands of GPUs have to work together, requiring a network that's fast, efficient and resilient.
Lambda is tackling the network pie...
15/11/2026
Can VideoProc AI Really Reduce Noise and Enhance Blurry, Old Footage? Real-World...
15/11/2026
Over two days at COEX, more than 2,000 attendees converged for AI Day Seoul, bri...
14/11/2026
Longtime MLB Network employee David Maybrown passed away Thursday at the age of 57. David joined MLB Network in 2009 shortly after its launch, beginning as an i...
14/11/2026
Marcelo Dai's Live Session Captured with URSA Cine Immersive
Brie Clayton November 14, 2026
0 Comments
Artist takes viewers on an immersive journe...
14/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
14/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
14/11/2026
Newest Lenses, Horizon Anamorphic Seminar & Guided Lens Character Experience Highlight Creative Possibilities
At the 2026 Toru CAMERIMAGE Film Festival, ZEISS...
14/11/2026
Sonnet Announces Black Friday Sale on Select Products
Brie Clayton November 14, 2026
0 Comments
WHAT: Sonnet Technologies is having a sale on a select...
14/11/2026
Berklee Alumni Shine at the 2026 Latin Grammy Awards This year's winners included songwriters, engineers, and performers who contributed to acclaimed reco...
13/11/2026
The UIM E1 Series wraps its third season this week with the Bombay Sapphire E1 Miami GP, Nov. 11-14, on Biscayne Bay at the Miami Seaplane Base on Watson Island...
13/11/2026
In-venue and creative video staffers at the professional and collegiate level ha...
13/11/2026
Singer-songwriter Benson Boone will headline entertainment for the Discover NHL Winter Classic between the Utah Mammoth and Colorado Avalanche at Rice-Eccles St...
13/11/2026
Blackmagic Design has announced the Blackmagic Ambisonic Microphone, a 7th order ambisonic microphone array that captures spatial audio for immersive video prod...
13/11/2026
SMPTE has announced that The Studio B&H and Riedel Communications will provide the video infrastructure and communications technology for the Solutions Hub at t...
13/11/2026
The Hollywood Professional Association (HPA) has announced the theme and program focus for the 2027 HPA Tech Retreat, which takes place Feb. 14-18, 2027, in La ...
13/11/2026
Football Manager 27 Touch is available on Apple Arcade starting today. The game ...
13/11/2026
MotoGP has announced a two-year media rights agreement with ESPN covering the Un...
13/11/2026
Apple Sports, the free iPhone app, has added the Ultimate Fighting Championship (UFC) to its lineup of supported sports, in time for Saturday's UFC 334. The...
13/11/2026
The NBA and Fanatics Studios, the joint venture between Fanatics and entertainme...
13/11/2026
ESPN held its annual EDGE Innovation Conference last night and offered up an ove...
13/11/2026
Regional sports network gives Ithaca College, SUNY Cortland students a platform to thrive
One of the biggest games in Division III college football will be pla...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...