
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
17/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
17/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
17/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
17/06/2026
Montr al, Quebec, June 11, 2026 Kiloview, a leading provider of AV-over-IP and NDI -based video transmission solutions, today announced a distribution partner...
17/06/2026
Changsha, China, June 15, 2026 Kiloview officially announced the launch of U4 IP Video Dock, a compact IP video decoder and output dock designed to bring prof...
17/06/2026
The RT Concert Orchestra will bring the timeless music of The Beach Boys to aud...
17/06/2026
Calibr-Skaggs awarded $5.1M by NIH to develop long-acting hepatitis B virus therapy A new program aims to replace a daily HBV drug with once-monthly or even qua...
16/06/2026
Thomson's highly regarded expert-led online learning courses are now easier to access on the go via our new App.
Available now on Google Play Store, the J...
16/06/2026
Neumann.Berlin has released firmware version 2.0 for the MT 48 audio interface, adding plugin compatibility, expanded Dante networking options, broadcast encode...
16/06/2026
TVNewsCheck has announced that nominations are now open for its 2027 Women in Technology Awards, to be presented at NAB Show 2027 on Tuesday, April 6 in the Med...
16/06/2026
Clear-Com has announced Avalon, a 1RU IP intercom platform for broadcast, live events, and production environments. Designed for IP-only workflows, Avalon suppo...
16/06/2026
SNS has published a guide to remote video editing workflows using its EVO shared storage platform and companion tools, covering use cases ranging from home edit...
16/06/2026
Grass Valley has announced that the Richmond Flying Squirrels, a Minor League Baseball affiliate of the San Francisco Giants, have deployed five Grass Valley LD...
16/06/2026
The Alliance for IP Media Solutions (AIMS) has announced the launch of the Official IPMX Training Series, a free online program covering the design, configurati...
16/06/2026
Swerve TV has announced distribution agreements with Fubo, Plex, Amazon Fire TV,...
16/06/2026
ATP and TikTok have announced an expansion of their global content partnership, extending the ATP's TikTok hub powered by TikTok GamePlan to cover all nine ...
16/06/2026
Network's LA facility serves as the heart of a sprawling operation built to ...
16/06/2026
At Pico, the network's media-management team is supporting a flood of HBS fe...
16/06/2026
The NHL will no longer air on CBC after the pulic broadcasters and national rights-holder Rogers Sportsnet were unable to come to agreement.
After a successfu...
16/06/2026
As live sports broadcasters continue to seek new ways to make complex action mor...
16/06/2026
Built with the 2026 FIFA World Cup in mind, these small but mighty IP-based tran...
16/06/2026
Boasts individual synths for each band
UVI's latest synth takes an interesting approach to synthesis, offering a trio of synth engines that each operate...
16/06/2026
New intelligent auto-fader plug-in unveiled
PSPaudioware's latest release offers automatic level adjustment and provides more detailed control than many...
16/06/2026
New performance-focused library announced
Crystal Pads is the latest addition to The Crow Hill Company's ever-growing product range, and according to th...
16/06/2026
Developed in partnership with Sequential
In recent years, GForce Software have branched into official emulations of classic hardware synths, delivering a ha...
16/06/2026
Designed specifically for live performance monitoring
beyerdynamic's latest announcement sees the company introduce an affordable in-ear monitoring syst...
16/06/2026
Official emulation celebrates iconic synth's 40th anniversary
Cherry Audio have just introduced Ensoniq ESQ-1, an official recreation of the 1986 polyph...
16/06/2026
Australians place growing trust in SBS News
16 June, 2026
Media releases
SBS has been recognised as one of Australia's most trusted news providers, ran...
16/06/2026
Rohde & Schwarz achieves highest number of GCF validated 3GPP NR NTN test cases ...
16/06/2026
Bydgoszcz to Become a Local Centre of Excellence for Advanced Rail Technologies....
16/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
16/06/2026
87 Percent of Creators Who Use Creative AI Say It Is Growing Their Business and ...
16/06/2026
Historic Zhuque-3 Reusable Rocket Test Mission Captured with URSA Cine Immersive
Brie Clayton June 16, 2026
0 Comments
Apple Immersive Video puts view...
16/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
16/06/2026
Rise WIB, the award-winning advocacy group championing gender diversity and career progression across the broadcast and media technology industry, today announc...
16/06/2026
Limecraft today announced the availability of Limecraft 2026.4, the fourth of eight planned platform releases this year. The update introduces Team-Based Access...
16/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
16/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
16/06/2026
Free Program Supports IPMX Education from Foundational Concepts Through System and Network Design
The Alliance for IP Media Solutions (AIMS) today announced t...
16/06/2026
Share your views on Screen Australia and the future of the industry 15 June 2026
Your feedback matters.
Following the instrumental insights provided in 2025,...
16/06/2026
Enterprises are moving agentic AI from proof of concept to production - and the next generation of AI factories are built for the era of agents.
At HPE Discove...
16/06/2026
AI runs at the speed of light. More and more, that light is made in Texas.
Cohe...
16/06/2026
Techtel Supports T-Motion RCCP-2A Controller Upgrade for Major Australian Broadc...
16/06/2026
Tuesday 16 June 2026
Record audiences tune in for opening weekend of ICC Women&...
16/06/2026
Every breakthrough AI model starts the same way: with a training run. The infrastructure running those training jobs shapes everything: how fast teams can itera...
15/06/2026
One of the more exciting internal video production divisions within a college at...