Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
North America Stories
19/05/2026
Cisco and the USGA have announced a multiyear extension of their partnership, which began in 2018. Cisco serves as the Official Technology Partner of the USGA, ...
19/05/2026
Urban Edge Network (UEN), a streaming platform for NAIA sports, has announced a partnership with Spiideo to provide streaming and production tools to UEN's ...
19/05/2026
Warner Bros. Discovery (WBD) will provide live coverage of all 900 Roland-Garros matches across its platforms beginning with qualifiers on May 18. In Europe, 21...
19/05/2026
Tubi, Fox Corporation's free streaming service, has announced the launch of the FIFA World Cup 2026 FOX Hub, a dedicated destination for World Cup programmi...
19/05/2026
Telef nica, in collaboration with Sony, has conducted a 5G connectivity trial at the Movistar Arena in Spain using the 26 GHz millimetre wave (mmWave) band. The...
19/05/2026
Ross Production Services (RPS) has installed a Calrec Argo M console into its new Hypermax-1 remote production truck, replacing one of three Argo S consoles pre...
19/05/2026
Panasonic Projector and Display Corporation has announced the acquisition of 100% of the shares of UK-based media technology company Hive Media Control Ltd. (HI...
19/05/2026
Globecast has announced the completion of a nine-month renovation of its Singapore facility, converting it from a traditional linear broadcast operation into a ...
19/05/2026
Grass Valley has announced a three-year enterprise agreement with Phoenix Broadc...
19/05/2026
Bitmovin has announced that Watch Brasil, a streaming platform operating across Brazil and Europe since 2018, has replaced its legacy systems with Bitmovin'...
19/05/2026
Ateme has announced the migration of Dish Home Nepal's Nepal Premier League (NPL) streaming infrastructure to Ateme's TITAN Live solution deployed on Ak...
19/05/2026
CMSI provided workflow, media management, and HDR support for ESPN during coverage of the NCAA Gymnastics Semifinals and Championships. The company supported fi...
19/05/2026
In advance of this year's Sports Emmy Awards, SVG is taking a deep dive into the six production-technologies nominated for this year's George Wensel Tec...
19/05/2026
In advance of this year's Sports Emmy Awards, SVG is taking a deep dive into the six production-technologies nominated for this year's George Wensel Tec...
19/05/2026
Featuring a fully IP infrastructure, Supershooter 11 is intended for large-scale events. Enabling remote and distributed workflows, Supershooter 65 joins the RE...
19/05/2026
By Jessica Herndon
The line wrapped around the building outside Denver's La...
19/05/2026
When all companies in the market are allowed to compete under the same set of ru...
19/05/2026
Battle-proven technology withstands electronic warfare threats across air, maritime surface and subsea domains....
19/05/2026
WESCAM MX-Series EO/IR systems provide high-precision targeting across domains, including counter-UAS applications...
19/05/2026
Champaign, IL - March 16, 2026 Cobalt Digital, the leading designer and manufa...
19/05/2026
CHAMPAIGN, Ill. April 16, 2026 - Cobalt Digital today announced a partnership ...
19/05/2026
Las Vegas - April 18, 2026 Cobalt Digital, the leading designer and manufactur...
19/05/2026
LAS VEGAS April 18, 2026 - Advanced HDR by Technicolor and Cobalt Digital are ...
19/05/2026
LAS VEGAS - May 4, 2026 - Cobalt Digital, the leading designer and manufacturer of signal processing products, and a founding partner in the openGear initiativ...
19/05/2026
London - 11 May 2026 Cobalt Digital, the leading designer and manufacturer of award-winning signal processing products, and a founding partner in the openGear...
19/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/05/2026
Grass Valley has entered into a three-year enterprise agreement with Phoenix Broadcast Solutions as the Singapore-based company expands its live production oper...
19/05/2026
Clear-Com supported Musikal Perahu Kertas, a large-scale musical theater production in Indonesia adapted from the acclaimed novel by Dee Lestari, through the d...
19/05/2026
Actus Digital, a LiveU company, and a leading provider of intelligent media monitoring and broadcast compliance platforms, announced today that METEO CONSULT ha...
19/05/2026
Globecast, a leading provider of managed services for the broadcast and media industry, today announced the completion of a major renovation of its Singapore fa...
19/05/2026
Following a major upgrade to its Connecticut remote production centre with the installation of three 60-fader Argo S consoles in 2023, RPS has switched one of t...
19/05/2026
PlayBox Technology today announced the launch of Celebro Play, a browser-based media orchestration platform designed specifically for broadcast facilities and p...
19/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/05/2026
LYNX Technik, provider of modular signal processing solutions for broadcast, post, and professional AV will showcase its latest signal processing solutions at B...
19/05/2026
Media Links, a leading developer and manufacturer of IP media transport solutions, has announced the integration of its Xscend platform with Skyline Communicat...
19/05/2026
Collaboration expands regional partner ecosystem as Media Links showcases IP media transport solutions on stand 4M4-4 and alongside Clearcast Asia on stand
Me...
19/05/2026
Company Highlights Labor-Saving Control and Custom Branding Opportunities Designed to Increase Integrator Profitability
LynTec, a leading manufacturer of ele...
19/05/2026
Mago Studio Moves AI Video Beyond Prompts with New Production-Focused Tools
Brie Clayton May 18, 2026
0 Comments
New Capabilities Bring Controllable, ...
19/05/2026
It May be Time to Stop and Take an AI Breather
Brie Clayton May 19, 2026
0 Comments
If the rock you stay on starts to roll, jump clean. Or you'll...
19/05/2026
Back to All News
Netflix Unveils the Trailer for RAFA, the Documentary Series A...
19/05/2026
Back to All News
Netflix Expands Consumer Products Footprint with Strategic Con...
19/05/2026
Matt Derella Expands Leadership Across LinkedIn Ads and Microsoft Advertising Published on May 19, 2026 Categories: Company News
LinkedIn Corporate Commun...
18/05/2026
Despite a bumpy start to the year, the provider of event-production support and ...
18/05/2026
Daktronics has partnered with the University of North Carolina to manufacture and install 11 LED displays totaling more than 10,000 square feet and more than 14...
18/05/2026
CBS LA has announced a multi-year partnership with the Los Angeles Rams, covering exclusive local broadcasts of Rams preseason games, weekly year-round programm...
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
