Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
NVIDIA's advancements in inference software optimization and the NVIDIA Hopper platform are helping industries serve the latest generative AI models, delivering excellent user experiences while optimizing total cost of ownership. The Hopper platform also helps deliver up to 15x more energy efficiency for inference workloads compared to previous generations.
AI inference is notoriously difficult, as it requires many steps to strike the right balance between throughput and user experience.
But the underlying goal is simple: generate more tokens at a lower cost. Tokens represent words in a large language model (LLM) system - and with AI inference services typically charging for every million tokens generated, this goal offers the most visible return on AI investments and energy used per task.
Full-stack software optimization offers the key to improving AI inference performance and achieving this goal.
Cost-Effective User Throughput Businesses are often challenged with balancing the performance and costs of inference workloads. While some customers or use cases may work with an out-of-the-box or hosted model, others may require customization. NVIDIA technologies simplify model deployment while optimizing cost and performance for AI inference workloads. In addition, customers can experience flexibility and customizability with the models they choose to deploy.
NVIDIA NIM microservices, NVIDIA Triton Inference Server and the NVIDIA TensorRT library are among the inference solutions NVIDIA offers to suit users' needs:
NVIDIA NIM inference microservices are prepackaged and performance-optimized for rapidly deploying AI foundation models on any infrastructure - cloud, data centers, edge or workstations.
NVIDIA Triton Inference Server, one of the company's most popular open-source projects, allows users to package and serve any model regardless of the AI framework it was trained on.
NVIDIA TensorRT is a high-performance deep learning inference library that includes runtime and model optimizations to deliver low-latency and high-throughput inference for production applications.
Available in all major cloud marketplaces, the NVIDIA AI Enterprise software platform includes all these solutions and provides enterprise-grade support, stability, manageability and security.
With the framework-agnostic NVIDIA AI inference platform, companies save on productivity, development, and infrastructure and setup costs. Using NVIDIA technologies can also boost business revenue by helping companies avoid downtime and fraudulent transactions, increase e-commerce shopping conversion rates and generate new, AI-powered revenue streams.
Cloud-Based LLM Inference To ease LLM deployment, NVIDIA has collaborated closely with every major cloud service provider to ensure that the NVIDIA inference platform can be seamlessly deployed in the cloud with minimal or no code required. NVIDIA NIM is integrated with cloud-native services such as:
Amazon SageMaker AI, Amazon Bedrock Marketplace, Amazon Elastic Kubernetes Service
Google Cloud's Vertex AI, Google Kubernetes Engine
Microsoft Azure AI Foundry coming soon, Azure Kubernetes Service
Oracle Cloud Infrastructure's data science tools, Oracle Cloud Infrastructure Kubernetes Engine
Plus, for customized inference deployments, NVIDIA Triton Inference Server is deeply integrated into all major cloud service providers.
For example, using the OCI Data Science platform, deploying NVIDIA Triton is as simple as turning on a switch in the command line arguments during model deployment, which instantly launches an NVIDIA Triton inference endpoint.
Similarly, with Azure Machine Learning, users can deploy NVIDIA Triton either with no-code deployment through the Azure Machine Learning Studio or full-code deployment with Azure Machine Learning CLI. AWS provides one-click deployment for NVIDIA NIM from SageMaker Marketplace and Google Cloud provides a one-click deployment option on Google Kubernetes Engine (GKE). Google Cloud provides a one-click deployment option on Google Kubernetes Engine, while AWS offers NVIDIA Triton on its AWS Deep Learning containers.
The NVIDIA AI inference platform also uses popular communication methods for delivering AI predictions, automatically adjusting to accommodate the growing and changing needs of users within a cloud-based infrastructure.
From accelerating LLMs to enhancing creative workflows and transforming agreement management, NVIDIA's AI inference platform is driving real-world impact across industries. Learn how collaboration and innovation are enabling the organizations below to achieve new levels of efficiency and scalability.
Serving 400 Million Search Queries Monthly With Perplexity AI Perplexity AI, an AI-powered search engine, handles over 435 million monthly queries. Each query represents multiple AI inference requests. To meet this demand, the Perplexity AI team turned to NVIDIA H100 GPUs, Triton Inference Server and TensorRT-LLM.
Supporting over 20 AI models, including Llama 3 variations like 8B and 70B, Perplexity processes diverse tasks such as search, summarization and question-answering. By using smaller classifier models to route tasks to GPU pods, managed by NVIDIA Triton, the company delivers cost-efficient, responsive service under strict service level agreements.
Through model parallelism, which splits LLMs across GPUs, Perplexity achieved a threefold cost reduction while maintaining low latency and high accurac
North America Stories
28/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Bluesky
Email...
28/01/2026
Boston Conservatory Orchestra Premieres a New Piano Concerto by Peter and Leonar...
27/01/2026
L-R: Jonathan Cuchacovich, Sonia Kennebeck, Alan Fischer, Daeil Kim, Andrew Sta...
27/01/2026
Nielsen will continue to drive growth and provide local audience measurement
in...
27/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
27/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
27/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
27/01/2026
Amagi Media Labs Limited, the only cloud-native SaaS provider offering end-to-end solutions across live production, content preparation, distribution, and monet...
27/01/2026
Miri Technologies Inc. today unveiled its V410 live 4K video encoder/decoder for streaming, IP-based production workflows and AV-over-IP distribution. Designed ...
27/01/2026
The Paul E. Tsongas Center at UMass Lowell in Massachusetts has chosen Ikegami cameras for incorporation into its broadcast-quality television production facili...
27/01/2026
SipRadius, specialists in secure, low latency media transport, will show the pro AV community its end-to-end solution for high quality content distribution and ...
27/01/2026
Hiltron will highlight the fruits of its 45-plus years of experience in satellite communications system integration and product development at the annual GovSat...
27/01/2026
Klvr will demonstrate how its Charger Pro solution is delivering cost reduction and performance benefits for live entertainment and pro AV markets at ISE 2026 (...
27/01/2026
Amino, a global media technology leader in enterprise video and digital signage, today announced that its Integrated Systems Europe (ISE) 2026 booth (4N700) wil...
27/01/2026
ANI, India's leading news agency and a major content provider across the Asia Pacific region, today announced a significant expansion of its long standing p...
27/01/2026
Independent analysis confirms faster encoding, lower CPU usage, and improved perceptual quality compared to CBR and CRF
VisualOn, a leader in multimedia playba...
27/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
27/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
27/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
27/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
27/01/2026
Back to All News
When It Feels Real, Is It Still a Lie? The Art of Sarah' ...
27/01/2026
Back to All News
That Night premiering on Netflix March 13
Entertainment
27 January 2026
GlobalSpain
Link copied to clipboard
DOWNLOAD THE KEY ART
DOWNLO...
27/01/2026
Back to All News
Netflix Revealed the Future of Mexican Stories Coming in 2026
Entertainment
27 January 2026
GlobalMexico
Link copied to clipboard
Among t...
27/01/2026
Back to All News
Four New Japan Titles Announced for 2026: Viral Hit', On...
27/01/2026
Back to All News
What's Next on Netflix from Japan: Expanding the Possibili...
27/01/2026
AI, graphics and virtual software power new production capabilities...
26/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
26/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
26/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
26/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
26/01/2026
Advanced Systems Group (ASG) today announced an exclusive partnership with Nu Studio, the company behind the first modular, portable studio designed for immersi...
26/01/2026
OpenDrives, the leader in high-end video data management and workflow solutions, today announced an add-on investment to its previous funding rounds, led by IAG...
26/01/2026
SmallHD today announced a major update to its award-winning PageOS software, introducing Fleet Control, Portrait Mode, expanded Camera Control features, and Dow...
26/01/2026
Whether rigging a PTZ on a truss, sliding a Blackmagic camera across a mezzanine railing, or capturing the mayhem from inside a drum, Filipic delivers visuals t...
26/01/2026
Back to All News
Netflix Unveils Official Trailer for Salvador
Entertainment
26 January 2026
GlobalSpain
Link copied to clipboard
DISCOVER THE TRAILER
DO...
26/01/2026
Back to All News
Netflix Presents Our 2026 Series and Films and Announces New Projects
Entertainment
26 January 2026
GlobalSpain
Link copied to clipboard
...
26/01/2026
Back to All News
Made in Louisiana: People We Meet on Vacation' Lights Up New Orleans
Emily Bader and Tom Blyth film on Royal Street in New Orleans' ...
26/01/2026
LinkedIn Gives Professionals the Edge with Verified Skills and Tools to Navigate the Job Search Show Proficiency of AI Tools such as Descript, Lovable, Relay.ap...
26/01/2026
Alfalite, Brainstorm, Dejero, Domo Broadcast Systems, FOR-A, KitPlus, Ontario So...
26/01/2026
Tyngsboro, Mass. - January 26, 2026 - Broadcast Pix is excited to exhibit at the Alliance for Community Media West Region Conference and Trade Show, taking plac...
25/01/2026
Back to All News
Sins of Kujo' Premieres April 2: Teaser Trailer, Art and ...
24/01/2026
Masami Kawai Selected as the 2026 Merata Mita Fellow; Isabella Madrigal and Tsanavi Spoonhunter Named 2026 Graton Fellows During Native Forum Celebration in Par...
24/01/2026
The MC-55A Peregrine aircraft will give the Royal Australian Air Force information superiority and serve as strategic assets for future Australian Defence Force...
24/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
24/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
24/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
23/01/2026
WWE's Virtual Production Playbook: How the Professional Wrestling Super Powe...
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using the NVIDIA AI inference platform - a full stack comprising world-class silicon, systems and software - is the key to delivering high-throughput and low-latency inference and enabling great user experiences while lowering cost.
