
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
15/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
15/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
South end zone videoboard, cloud-based control system will be ready for 2026 football season...
14/07/2026
Redesigned website, enhanced mobile app unify content, ticketing, and personalized fan engagement...
14/07/2026
Agreement spans sponsorship sales, digital monetization, radio production, and new practice facility naming rights...
14/07/2026
Independent league adds 14 regional sports network affiliates, growing distribut...
14/07/2026
New strategy aims to unify competitions, content, fan engagement, and commercial...
14/07/2026
ESPN and Disney+ become exclusive home of PFL events in key international MMA market...
14/07/2026
Gill will oversee engineering, technology, and sports operations across the company's 64 local television stations...
14/07/2026
Submitted by North American Broadcasters Association (NABA)
As broadcasters con...
14/07/2026
Modernized mobile unit combines HOME Apps, mc 56 console, and IP infrastructure ...
14/07/2026
Every match of the 32-team tournament will air across ION and Scripps Sports platforms in English and Spanish...
14/07/2026
FloHockey to stream every game of the annual international tournament under four...
14/07/2026
Victory+ telecasts to be simulcast on TEGNA-owned station, expanding free local distribution...
14/07/2026
Plug-ins for heavy music
Avalanche Tones is the brainchild of Ava Toton, a 17-year-old musician and developer who says her goal is to make the lives of gui...
14/07/2026
Launched alongside new Singer Showcase purchase model
IK Multimedia's innovative vocal-synthesis software has just gained its latest voice add-on, the R...
14/07/2026
Registration open until 1 September 2026
The MIDI Association have revealed that the registration deadline for this year's MIDI Innovation Awards has no...
14/07/2026
88-note model completes MK4 range
Novation have just introduced the final model in their flagship MIDI controller keyboard range, the Launchkey MK4 88. Roun...
14/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
IBC2026 has unveiled a powerful Conference programme bringing together global media leaders, technology innovators, creators, sports organisations, broadcasters...
14/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
Broadcast Solutions, a leading systems integrator and provider of innovative solutions for the broadcast media industry, has delivered two highly capable outsid...
14/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
14/07/2026
Editor's note: This post is part of the Nemotron Labs blog series, which exp...
14/07/2026
Techtel Successfully Relocates AICD Broadcast Studio to New Sydney Headquarters
BroadcastBroadcast EquipmentLive StreamingBroadcast Studio2026
14 July Writ...
14/07/2026
Tuesday 14 July 2026
First look revealed for Friday the 13th prequel, Crystal ...
14/07/2026
When immersive audio dominates industry headlines, it's easy to assume that every broadcaster is preparing for an Atmos future. The reality is quite differ...
14/07/2026
Emma and Sophie from ICG's marketing team joined thousands of fellow marketers, brands and agencies at MAD//Fest London 2026, one of the UK's biggest ma...
14/07/2026
Why Trusted and Secure Media Operations Matter In this series, we explore the technologies, architectures and operational realities shaping modern media operati...
14/07/2026
Pilot Project Shows How Retailers Are Prepared for the Next Step in the Evolution of Digital Commerce
Arvato Systems Drives Agentic Commerce Forward
G terslo...
14/07/2026
As part of this commitment, weve joined the SME Climate Hub, publicly pledging to:
measure our greenhouse gas emissions
reduce them in line with a net zero p...
14/07/2026
Power is AI infrastructure's inescapable constraint. How many tokens an AI factory can generate within a fixed power budget determines its revenue and profi...
13/07/2026
The Braves opted to keep production in-house rather than hand it off to MLB...
13/07/2026
Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...
13/07/2026
Eurovision Sport is making live athletics more accessible to fans than ever befo...
13/07/2026
The Milwaukee Bucks will return to full-season over-the-air television for the 2...
13/07/2026
SMPTE has announced an expanded education pathway for media technology professio...
13/07/2026
Vizrt has announced that its graphics technology was used by broadcast design agency Girraphic for Netflix's debut MVP MMA event, broadcast live from the In...
13/07/2026
ARRI has announced an agreement to sell its global rental activities in Europe, the United Kingdom, and North America to H2 Equity Partners through a management...
13/07/2026
DAZN has announced a partnership with Premier Boxing Champions (PBC) to bring PBC fight nights to DAZN subscribers globally. The partnership begins Saturday, Ju...
13/07/2026
TikTok and WSC Sports have announced a strategic partnership that gives sports r...