
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
North America Stories
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
The EZDRM approach to Revenue Security on streaming video can be visualized as a multi-dimensional set of solutions. A growing dimension of that approach is the...
03/09/2026
From field capture to management, control and production, Kiloview will demonstrate one connected Hybrid AV-over-IP workflow including a complete 4RU producti...
03/09/2026
Mediagenix, a global leader in smart content solutions to profitably connect the right content to the right audience, today announced Mediagenix Curation, a new...
03/09/2026
LucidLink, the Emmy-winning cloud-native file streaming platform for instant, secure access to large files, today announced the next step in its vision to make ...
03/09/2026
AJA Introduces FS8, Next-Gen 12G-SDI/SMPTE ST 2110 Media Processor
Brie Clayton September 2, 2026
0 Comments
AJA Video Systems unveiled FS8, a video a...
03/09/2026
Shooting Deadheading with Blackmagic URSA Cine 12K LF
Brie Clayton September 2, 2026
0 Comments
Bringing a small story to the big screen with dark com...
03/09/2026
OOONA Launches OOONA Go: Professional Localization Management for Small Companie...
03/09/2026
DURHAM, N.C. - East of Texas Durham has officially opened its doors at American ...
02/09/2026
Professional and collegiate clubs, leading vendors discuss current trends in stadium production
On. Aug. 12, the 2026 SVG Venues & Teams Summit explored the ev...
02/09/2026
SoFi Stadium, YouTube Theater, and Hollywood Park have completed an infrastructure upgrade expanding game presentation and broadcast production capabilities acr...
02/09/2026
Comcast Technology Solutions (CTS) has announced a suite of AI-powered workflow applications for broadcasters, content owners, and operators, built on its Video...
02/09/2026
The North American Broadcasters Association (NABA) and the Association of Professional Wireless Production Technologies (APWPT) have announced a partnership to ...
02/09/2026
Carolina Hurricanes, Anaheim Ducks, Columbus Blue Jackets, Dallas Stars, Minneso...
02/09/2026
New LED provides enhanced quality control, quicker show prep, detailed productio...
02/09/2026
Six-country survey finds 26% of sports fans frequently unable to determine where...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
KOKKEDAL, DENMARK, SEPTEMBER 1, 2026 DPA Microphones announces its new 6388 CORE Headset Microphone, a high-performance directional headset designed for dema...
02/09/2026
Pliant Technologies will highlight its CrewCom CXD 32CF 32x32 Dante /AES67 Digital Audio Network Interface, a powerful connectivity solution designed to expand ...
02/09/2026
iWedia, a leading provider of software solutions for connected TV devices, today announced two enhancements to Teatro Pulse, its end-to-end TV service operation...
02/09/2026
Signiant, trusted by the media industry for accelerated content movement for more than 20 years, will showcase the latest evolution of the Signiant Platform at ...
02/09/2026
With a contractual agreement to supply a leading UK sports broadcaster with outside broadcast services for 104 football matches per season over the next four ye...
02/09/2026
IBC2026 will put sport firmly at the centre of the global media and entertainment technology conversation with the strongest sports showing in the event's h...
02/09/2026
PTZOptics will bring a hands-on showcase of intelligent video production to IBC2026 (11-14 September, RAI Amsterdam), with demonstrations spanning new camera te...
02/09/2026
OOONA, a global provider of professional management and production tools for the media localization industry, today announced the launch of OOONA Go, a ready-to...
02/09/2026
EON Media will make its European debut at IBC2026 (September 11-14, RAI Amsterdam) in the show's Future Tech Zone (Hall 14.FT5). The company is launching it...
02/09/2026
Frequency, the engine behind many of the worlds leading streaming television channels, will demonstrate at IBC2026 how integrating intelligence throughout chann...
02/09/2026
On-device AI platform gives broadcasters and content creators purpose-built tools for real-time vertical video production and AI-enhanced sports coverage...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/09/2026
Next-Generation Freeform Control Surfaces Lay the Foundation for AI-Enabled Media Operations
At IBC2026 (Sept. 11 14, RAI Amsterdam, Stand #1.B73), Imagine Com...
02/09/2026
Friedrichstadt-Palast in Berlin, the world's largest theater stage, has modernized its backstage production infrastructure with a comprehensive Riedel solut...
02/09/2026
Women in Media announces that it will host the two-day Reel/Forward in-person conference devoted to navigating today's production scene. Guests can enjoy pa...
02/09/2026
The expanded partnership brings Teradek's Live Production solutions to New Media AV's European dealer and integrator network, opening new opportunities ...
02/09/2026
QuickLink, a leading provider of award-winning video production and remote guest contribution solutions, announces a major expansion to its award winning Studio...
02/09/2026
BCNEXXT, the developers of the advanced playout platform Vipe, today announced that premium entertainment brand Foxtel has selected Vipe to underpin playout for...
02/09/2026
SubtitleNEXT helps HOT Telecom's subtitling team keep a cool head with reliable QC capabilities
Profuz Digital, global developer of digital media and local...
02/09/2026
Case Study: Mission Digital Strengthens Battery Safety with Safe-ion
As productions become increasingly dependent on battery-powered equipment, the safe storag...
02/09/2026
LA JOLLA, CA-Influenza viruses constantly shapeshift to evade recognition by the immune system. This shapeshifting occurs in critical proteins like hemagglutini...
02/09/2026
LA JOLLA, CA-Clinicians often use a medical test called a 12-lead electrocardiogram (ECG) to diagnose heart problems, which uses electrodes placed on the chest ...
02/09/2026
LA JOLLA, CA-Proteins may look like rigid molecular structures, but inside our c...