![](//www.4rfv.com/images/no_image.jpg)
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and which showcases new hardware, software, tools and accelerations for RTX PC users.
As generative AI advances and becomes widespread across industries, the importance of running generative AI applications on local PCs and workstations grows. Local inference gives consumers reduced latency, eliminates their dependency on the network and enables more control over their data.
NVIDIA GeForce and NVIDIA RTX GPUs feature Tensor Cores, dedicated AI hardware accelerators that provide the horsepower to run generative AI locally.
Stable Video Diffusion is now optimized for the NVIDIA TensorRT software development kit, which unlocks the highest-performance generative AI on the more than 100 million Windows PCs and workstations powered by RTX GPUs.
Now, the TensorRT extension for the popular Stable Diffusion WebUI by Automatic1111 is adding support for ControlNets, tools that give users more control to refine generative outputs by adding other images as guidance.
TensorRT acceleration can be put to the test in the new UL Procyon AI Image Generation benchmark, which internal tests have shown accurately replicates real-world performance. It delivered speedups of 50% on a GeForce RTX 4080 SUPER GPU compared with the fastest non-TensorRT implementation.
More Efficient and Precise AI TensorRT enables developers to access the hardware that provides fully optimized AI experiences. AI performance typically doubles compared with running the application on other frameworks.
It also accelerates the most popular generative AI models, like Stable Diffusion and SDXL. Stable Video Diffusion, Stability AI's image-to-video generative AI model, experiences a 40% speedup with TensorRT.
The optimized Stable Video Diffusion 1.1 Image-to-Video model can be downloaded on Hugging Face.
Plus, the TensorRT extension for Stable Diffusion WebUI boosts performance by up to 2x - significantly streamlining Stable Diffusion workflows.
With the extension's latest update, TensorRT optimizations extend to ControlNets - a set of AI models that help guide a diffusion model's output by adding extra conditions. With TensorRT, ControlNets are 40% faster.
TensorRT optimizations extend to ControlNets for improved customization. Users can guide aspects of the output to match an input image, which gives them more control over the final image. They can also use multiple ControlNets together for even greater control. A ControlNet can be a depth map, edge map, normal map or keypoint detection model, among others.
Download the TensorRT extension for Stable Diffusion Web UI on GitHub today.
Other Popular Apps Accelerated by TensorRT Blackmagic Design adopted NVIDIA TensorRT acceleration in update 18.6 of DaVinci Resolve. Its AI tools, like Magic Mask, Speed Warp and Super Scale, run more than 50% faster and up to 2.3x faster on RTX GPUs compared with Macs.
In addition, with TensorRT integration, Topaz Labs saw an up to 60% performance increase in its Photo AI and Video AI apps - such as photo denoising, sharpening, photo super resolution, video slow motion, video super resolution, video stabilization and more - all running on RTX.
Combining Tensor Cores with TensorRT software brings unmatched generative AI performance to local PCs and workstations. And by running locally, several advantages are unlocked:
Performance: Users experience lower latency, since latency becomes independent of network quality when the entire model runs locally. This can be important for real-time use cases such as gaming or video conferencing. NVIDIA RTX offers the fastest AI accelerators, scaling to more than 1,300 AI trillion operations per second, or TOPS.
Cost: Users don't have to pay for cloud services, cloud-hosted application programming interfaces or infrastructure costs for large language model inference.
Always on: Users can access LLM capabilities anywhere they go, without relying on high-bandwidth network connectivity.
Data privacy: Private and proprietary data can always stay on the user's device.
Optimized for LLMs What TensorRT brings to deep learning, NVIDIA TensorRT-LLM brings to the latest LLMs.
TensorRT-LLM, an open-source library that accelerates and optimizes LLM inference, includes out-of-the-box support for popular community models, including Phi-2, Llama2, Gemma, Mistral and Code Llama. Anyone - from developers and creators to enterprise employees and casual users - can experiment with TensorRT-LLM-optimized models in the NVIDIA AI Foundation models. Plus, with the NVIDIA ChatRTX tech demo, users can see the performance of various models running locally on a Windows PC. ChatRTX is built on TensorRT-LLM for optimized performance on RTX GPUs.
NVIDIA is collaborating with the open-source community to develop native TensorRT-LLM connectors to popular application frameworks, including LlamaIndex and LangChain.
These innovations make it easy for developers to use TensorRT-LLM with their applications and experience the best LLM performance with RTX.
Get weekly updates directly in your inbox by subscribing to the AI Decoded newsletter.
More from Nvidia
25/07/2024
Hey, you. You're finally awake.
It's the summer of Elder Scrolls - whe...
24/07/2024
AI is set to transform the workforce - and the Georgia Institute of Technology's new AI Makerspace is helping tens of thousands of students get ahead of the...
24/07/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...
23/07/2024
Generative AI applications have little, or sometimes negative, value without acc...
23/07/2024
Businesses seeking to harness the power of AI need customized models tailored to their specific industry needs.
NVIDIA AI Foundry is a service that enables ent...
22/07/2024
AI and accelerated computing - twin engines NVIDIA continuously improves - are d...
22/07/2024
Team NVIDIA has triumphed at the Amazon KDD Cup 2024, securing first place Friday across all five competition tracks.
The team - consisting of NVIDIANs Ahmet E...
19/07/2024
Research published earlier this month in the science journal Nature used NVIDIA-powered supercomputers to validate a pathway toward the commercialization of qua...
19/07/2024
Research published earlier this month in the science journal Nature used NVIDIA-powered supercomputers to validate a pathway toward the commercialization of qua...
19/07/2024
AI has seen unprecedented growth - spurring the need for new training and educat...
19/07/2024
Research published earlier this month in the science journal Nature used NVIDIA-powered supercomputers to validate a pathway toward the commercialization of qua...
18/07/2024
Mistral AI and NVIDIA today released a new state-of-the-art language model, Mist...
18/07/2024
It's time for a sweet treat - the GeForce NOW Summer Sale offers high-perfor...
17/07/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...
16/07/2024
Editor's note: This post is part of our In the NVIDIA Studio series, which c...
15/07/2024
NVIDIA founder and CEO Jensen Huang and Meta founder and CEO Mark Zuckerberg wil...
12/07/2024
NVIDIA is taking an array of advancements in rendering, simulation and generativ...
11/07/2024
Unlock new experiences every GFN Thursday. Whether post-apocalyptic survival adventures, narrative-driven games or vast, open worlds, GeForce NOW always has som...
11/07/2024
Enhancing Japan's AI sovereignty and strengthening its research and development capabilities, Japan's National Institute of Advanced Industrial Science ...
10/07/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, softwar...
10/07/2024
Improved cancer diagnostics - and improved patient outcomes - could be among the...
09/07/2024
Sphere, a new kind of entertainment medium in Las Vegas, is joining the ranks of legendary circular performance spaces such as the Roman Colosseum and Shakespea...
08/07/2024
Artificial intelligence is transforming the transportation industry, helping dri...
04/07/2024
GeForce NOW is bringing 22 new games to members this month.
Dive into the four titles available to stream on the cloud gaming service this week to stay cool an...
03/07/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...
28/06/2024
On a weekday afternoon, Ashwini Ashtankar sat on the bank of the Doodhpathri River, in a valley nestled in the Himalayas. Taking a deep breath, she noticed that...
27/06/2024
Editor's note: This post is part of Into the Omniverse, a series focused on ...
27/06/2024
Get ready to feel some chills, even amid the summer heat. Capcom's award-winning Resident Evil Village brings a touch of horror to the cloud this GFN Thursd...
26/06/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...
26/06/2024
Roblox is a colorful online platform that aims to reimagine the way that people ...
25/06/2024
Generative AI has revolutionized software development with prompt-based code generation - protein design is next.
EvolutionaryScale today announced the release...
24/06/2024
Multi-die chips, known as three-dimensional integrated circuits, or 3D-ICs, represent a revolutionary step in semiconductor design. The chips are vertically sta...
20/06/2024
Sit back and settle in for some epic storytelling. Tell Me Why and As Dusk Falls - award-winning, narrative-driven games from Xbox Studios - add to the 1,900+ g...
19/06/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, softwar...
18/06/2024
The electric grid and the utilities managing it have an important role to play in the next industrial revolution that's being driven by AI and accelerated c...
17/06/2024
NVIDIA contributed the largest ever indoor synthetic dataset to the Computer Vision and Pattern Recognition (CVPR) conference's annual AI City Challenge - h...
17/06/2024
Making moves to accelerate self-driving car development, NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Vision and Pattern Recognit...
17/06/2024
NVIDIA researchers are at the forefront of the rapidly advancing field of visual...
15/06/2024
NVIDIA founder and CEO Jensen Huang on Friday encouraged Caltech graduates to pu...
14/06/2024
When she was five years old, Veronica Miller (n e Teklai) and her family left their homeland of Eritrea, in the Horn of Africa, to escape an ongoing war with Et...
14/06/2024
NVIDIA today announced Nemotron-4 340B, a family of open models that developers ...
13/06/2024
Set sail for adventure, pirates. Sea of Thieves makes waves in the cloud this week. It's an adventure-filled GFN Thursday with four new games joining the Ge...
12/06/2024
Accelerated computing is transforming data processing and analytics for enterpri...
12/06/2024
The full-stack NVIDIA accelerated computing platform has once again demonstrated...
12/06/2024
Let's talk about NeRFs - no, not the neon-colored foam dart blasters, but ne...
12/06/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...
07/06/2024
Across industries, AI is supercharging innovation with machine-powered computation. In finance, bankers are using AI to detect fraud more quickly and keep accou...
06/06/2024
Capcom's latest entry in the iconic Street Fighter series, Street Fighter 6, punches its way into the cloud this GFN Thursday. The game, along with Ubisoft&...
05/06/2024
India's AI market is expected to be massive. Yotta Data Services is setting its sights on supercharging it. In this episode of NVIDIA's AI Podcast, Suni...
05/06/2024
NVIDIA launched NVIDIA Studio at COMPUTEX in 2019. Five years and more than 500 ...