
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
20/08/2026
TV, BritBox and Mammoth Screen have announced that filming is now underway on th...
20/08/2026
New York August 19, 2026 Nielsen today shared a number of key enhancements t...
20/08/2026
New Gracenote Data Hub analysis finds sports program titles up 31.2% and individ...
20/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/08/2026
At IBC, Profuz Digital will showcase at stand 8.A88 the SubtitleNEXT platform delivering innovative AI transcription integrations and stunning new features boos...
20/08/2026
Riedel Communications today announced that Transformation Church, one of the fastest-growing and most influential digital ministries in the United States, has d...
20/08/2026
QuickLink Showcases New 4K Capabilities Across StudioEdge and StudioCall at IBC ...
20/08/2026
The Boys Season Five Graded with DaVinci Resolve Studio
Brie Clayton August 20, 2026
0 Comments
Colorist Siggy Ferstl uses Depth Map to bring a three ...
20/08/2026
Adobe Firefly expands its creative AI studio: generate music, speech, and sound ...
20/08/2026
Multiview available via SaaS for streaming platforms and operators through broadpeak.io, with live demonstrations at IBC2026
Broadpeak , a leader in streaming ...
20/08/2026
Techex, a leading provider of live broadcast infrastructure solutions, has appointed Stuart Almond as Chief Revenue Officer, further strengthening its executive...
20/08/2026
Avid today announced that MediaCity Studios has upgraded its production infrastructure with an enhanced end-to-end Avid production ecosystem, including the dep...
20/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
20/08/2026
IPC Expands Brazilian Presence with New Channel Partner CCATech
Industry veterans Carlos Abrah o and Bruno Lopes strengthen local support and expand IPCs prese...
20/08/2026
Zixi Enters Mauritius Market Through Strategic Deployment with MTC Jockey Club
Worlds Oldest Horse Racing Club in the Southern Hemisphere Selects Zixis Softwar...
20/08/2026
LA JOLLA, CA-Scripps Research assistant professor Tiantian Liu has received a five-year NIAID DP2 award from the National Institute of Allergy and Infectious Di...
20/08/2026
Audiences reveal appetite for multicultural Australia on-screen 20 August 2026
Dr Georgie McClean, Dr Rebecca Huntley (L-R)
By Dr Georgie McClean and Dr Rebec...
20/08/2026
How Mavis Studio Is Expanding Live Production Teaching at Sussex
Brie Clayton August 19, 2026
0 Comments
Hero image displays a student about to mix be...
20/08/2026
Vertical Unscripted Series Legal Queens Captured with Blackmagic Design
Brie Clayton August 19, 2026
0 Comments
Reality TV veteran Eric Gardner create...
20/08/2026
The music between the notes Coming together In the middle of 2026, as conversations around artificial intelligence continue to raise questions about how musi...
20/08/2026
Arvato Systems Brings Together Digital Transformation Leaders in the Manufacturi...
20/08/2026
It's a new way into the cloud.
GeForce NOW welcomes Firefox support to the cloud, opening up another way to jump into high-performance PC gaming straight ...
19/08/2026
As the cost of IP nears price parity with traditional SDI, Reynolds argues that ...
19/08/2026
A private 5G bubble was chosen instead of traditional broadcast RF for the race ...
19/08/2026
(L-R) Rafael Manuel and Isabel Sicat attend the Filipi ana premiere during the 2026 Sundance Film Festival at The Ray Theatre on January 23, 2026, in Park Cit...
19/08/2026
Studio owners invited to have their say on rates cuts
The Music Producers Guild (MPG) are inviting UK-based studios to take part in an in-depth survey that ...
19/08/2026
Vintage-inspired desktop interface revealed
Harrison's latest hardware offering sees them head into the world of audio interfaces, delivering a 10-in/12...
19/08/2026
Granular effect remains locked to scale and tempo
Described as your dream granular effect , Minimal Audio's latest creation is capable of turning any s...
19/08/2026
First polyphonic SWAM instrument incoming
Audio Modeling's latest physical-modelling orchestral instrument is just around the corner, and is now availab...
19/08/2026
World Cup Games Generate 84 Billion Minutes Across FOX, Fox Sports 1 and NBCU...
19/08/2026
July brought typical peak-holiday drops in viewership, driven primarily by viewers stepping away from screens and a visible decline in reach across both traditi...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Henry joins BCE as the company accelerates its commercial strategy and expands its international presence across the media, broadcast and technology markets. He...
19/08/2026
iWedia, a leading provider of software solutions for connected TV devices, today announced a major milestone in Android platform optimization with the successfu...
19/08/2026
NTP Technology announces that ST 2110-30 audio-over-IP connectivity with AES67 and RAVENNA capabilities is now available for the brand's world-class audio i...
19/08/2026
Net Insight has been selected to provide its Nimbra platform by the service provider Big Blue Marble, formerly ORS Group, as the core media transport solution f...
19/08/2026
LiveU has supported Coventry City FC and several other top-flight UK football clubs through one of their most demanding live production challenges of the year, ...
19/08/2026
Nature Seekers Documentary Shot with Blackmagic URSA Cine 12K LF
Brie Clayton August 18, 2026
0 Comments
Filming endangered turtles required camera...
19/08/2026
Neyrinck Brings D-Control Consoles Back to Life on Apple Silicon with D-Control ...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...