Sony Pixel Power calrec Sony

NVIDIA TensorRT Boosts Stable Diffusion 3.5 Performance on NVIDIA GeForce RTX and RTX PRO GPUs

12/06/2025

Generative AI has reshaped how people create, imagine and interact with digital content.

As AI models continue to grow in capability and complexity, they require more VRAM, or video random access memory. The base Stable Diffusion 3.5 Large model, for example, uses over 18GB of VRAM - limiting the number of systems that can run it well.

By applying quantization to the model, noncritical layers can be removed or run with lower precision. NVIDIA GeForce RTX 40 Series and the Ada Lovelace generation of NVIDIA RTX PRO GPUs support FP8 quantization to help run these quantized models, and the latest-generation NVIDIA Blackwell GPUs also add support for FP4.

NVIDIA collaborated with Stability AI to quantize its latest model, Stable Diffusion (SD) 3.5 Large, to FP8 - reducing VRAM consumption by 40%. Further optimizations to SD3.5 Large and Medium with the NVIDIA TensorRT software development kit (SDK) double performance.

In addition, TensorRT has been reimagined for RTX AI PCs, combining its industry-leading performance with just-in-time (JIT), on-device engine building and an 8x smaller package size for seamless AI deployment to more than 100 million RTX AI PCs. TensorRT for RTX is now available as a standalone SDK for developers.

RTX-Accelerated AI NVIDIA and Stability AI are boosting the performance and reducing the VRAM requirements of Stable Diffusion 3.5, one of the world's most popular AI image models. With NVIDIA TensorRT acceleration and quantization, users can now generate and edit images faster and more efficiently on NVIDIA RTX GPUs.

Stable Diffusion 3.5 quantized FP8 (right) generates images in half the time with similar quality as FP16 (left). Prompt: A serene mountain lake at sunrise, crystal clear water reflecting snow-capped peaks, lush pine trees along the shore, soft morning mist, photorealistic, vibrant colors, high resolution. To address the VRAM limitations of SD3.5 Large, the model was quantized with TensorRT to FP8, reducing the VRAM requirement by 40% to 11GB. This means five GeForce RTX 50 Series GPUs can run the model from memory instead of just one.

SD3.5 Large and Medium models were also optimized with TensorRT, an AI backend for taking full advantage of Tensor Cores. TensorRT optimizes a model's weights and graph - the instructions on how to run a model - specifically for RTX GPUs.

FP8 TensorRT boosts SD3.5 Large performance by 2.3x vs. BF16 PyTorch, with 40% less memory use. For SD3.5 Medium, BF16 TensorRT delivers a 1.7x speedup. Combined, FP8 TensorRT delivers a 2.3x performance boost on SD3.5 Large compared with running the original models in BF16 PyTorch, while using 40% less memory. And in SD3.5 Medium, BF16 TensorRT provides a 1.7x performance increase compared with BF16 PyTorch.

The optimized models are now available on Stability AI's Hugging Face page.

NVIDIA and Stability AI are also collaborating to release SD3.5 as an NVIDIA NIM microservice, making it easier for creators and developers to access and deploy the model for a wide range of applications. The NIM microservice is expected to be released in July.

TensorRT for RTX SDK Released Announced at Microsoft Build - and already available as part of the new Windows ML framework in preview - TensorRT for RTX is now available as a standalone SDK for developers.

Previously, developers needed to pre-generate and package TensorRT engines for each class of GPU - a process that would yield GPU-specific optimizations but required significant time.

With the new version of TensorRT, developers can create a generic TensorRT engine that's optimized on device in seconds. This JIT compilation approach can be done in the background during installation or when they first use the feature.

The easy-to-integrate SDK is now 8x smaller and can be invoked through Windows ML - Microsoft's new AI inference backend in Windows. Developers can download the new standalone SDK from the NVIDIA Developer page or test it in the Windows ML preview.

For more details, read this NVIDIA technical blog and this Microsoft Build recap.

Join NVIDIA at GTC Paris At NVIDIA GTC Paris at VivaTech - Europe's biggest startup and tech event - NVIDIA founder and CEO Jensen Huang yesterday delivered a keynote address on the latest breakthroughs in cloud AI infrastructure, agentic AI and physical AI. Watch a replay.

GTC Paris runs through Thursday, June 12, with hands-on demos and sessions led by industry leaders. Whether attending in person or joining online, there's still plenty to explore at the event.

Each week, the RTX AI Garage blog series features community-driven AI innovations and content for those looking to learn more about NVIDIA NIM microservices and AI Blueprints, as well as building AI agents, creative workflows, digital humans, productivity apps and more on AI PCs and workstations.

Plug in to NVIDIA AI PC on Facebook, Instagram, TikTok and X - and stay informed by subscribing to the RTX AI PC newsletter.

Follow NVIDIA Workstation on LinkedIn and X.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/rtx-ai-garage-gtc-paris-tensorrt-rtx-nim...
See more stories from nvidia

Most recent headlines

06/10/2025

France Tlvisions Wins Prestigious 2025 EBU Technology & Innovation Award in Groundbreaking Collaboration with Dalet

France T l visions, France's leading broadcaster, has received the 2025 EBU ...

04/09/2025

Monumental Sports & Entertainment and Dalet Win Prestigious 2025 NAB Show Project of the Year Award

Monumental Sports & Entertainment (MSE), in collaboration with Dalet, has been a...

07/08/2025

Tata Motors & Dolby Bring Dolby Atmos to Harrier.ev, Redefining In-Car Entertainment Experience

July 8 2025, 22:30 (PDT) Tata Motors & Dolby Bring Dolby Atmos to Harrier.ev, R...

16/07/2025

Spotify's Editors Pick Their Best Podcasts of the Year (So Far)

Spotify's podcast editorial team is always on the hunt for shows and episodes that spark conversation, push boundaries, and keep us coming back for more. As...

16/07/2025

Clear-Com's EQUIP Boosts Ride Efficiency, Safety, and Communication at Merlin Theme Parks

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...

16/07/2025

IBC 2025

Join us at IBC this September and take control of your content. Discover how we're bringing a new level of purpose and precision to the application of AI in...

16/07/2025

May 2025 Spring with Miniscule drops in viewing figures in Poland

Warsaw - Poland, June 24, 2025 - Nielsen, the global leader in audience measurement, data and analytics, has released its latest May All Screens Video Landscape...

16/07/2025

Bluey and Squid Game Top Nielsen's Streaming Charts in First Half of 2025

Bluey Tops Nielsen's Overall Streaming List with More Than 25 Billion Minutes Streamed from January through June 2025 Big June Vaults Squid Game into Top O...

16/07/2025

TV SIM builds for the future with high efficiency transmi...

TV SIM, the SBT affiliate serving Esp rito Santo state in Brazil, has launched a transmitter upgrade initiative with the installation of a Rohde & Schwarz R&S T...

16/07/2025

Pebble looks to the future of playout automation at IBC20...

Pebble, the leading automation, content management and integrated channel specialist, is turning its vision to the future of automation with its demonstrations ...

16/07/2025

Ikegami Announces UHK-X700RF Wireless Television Camera

Ikegami announces a new addition to its UNICAM-XE product range with the introduction of the UHK-X700RF wireless portable camera: The new version has the same f...

16/07/2025

TwelveLabs video understanding models now avail in Amazon...

Today, Amazon Web Services added TwelveLabs as a new model provider to Amazon Bedrock, delivering what could be the most significant breakthrough in enterprise ...

16/07/2025

Netflix Boosts Streaming to 46% of TV Viewing in June

NEW YORK Streaming platforms continued to dominate TV viewing patterns in June, with data from Nielsen's 50th monthly report of The Gauge showing that strea...

16/07/2025

Ikegami Introduces Wireless Version of its UHK-X700 Television Camera

MAHWAH, NJ Ikegami has introduced the the UHK-X700RF wireless portable camera, a new addition to its UNICAM-XE product range with the same feature set, operatio...

16/07/2025

IBC2025 Attendees to Get Ticketless Access to Public Transport

LONDON IBC2025 organizers said global attendees will have access to a new feature that will integrate Amsterdam's GVB public transport pass into the officia...

16/07/2025

Tribeca Films to Release the Acclaimed LARPing Doc WE CAN BE HEROES by Carina Mia Wong and Alex Simmons, Alongside a Slate of Festival-Favorite Titles

July 16th, 2025 Tribeca Films to Release the Acclaimed LARPing Doc WE CAN BE HEROES by Carina Mia Wong and Alex Simmons, Alongside a Slate of Festival-Favori...

16/07/2025

Deltatre Acquires Endeavor Streaming in Major Streaming Industry Move

Deltatre Acquires Endeavor Streaming in Major Streaming Industry Move The transaction is expected to close in the third quarter of 2025 By Brandon Costa, Direc...

16/07/2025

SVG Regional Sports Production Summit 2025: All Sessions Now Available to Watch on SVG PLAY

SVG Regional Sports Production Summit 2025: All Sessions Now Available to Watch ...

16/07/2025

Sky and ITV extend multi-year content and platform partnership

Wednesday 16 July 2025 Sky and ITV today announced an extension of their long-standing partnership, which will see ITV's content and services remain seamle...

16/07/2025

Living with the Lions: What really happens on tour

Sky Sports lifts the lid on Lions life with hilarious behind-the-scenes mockumentaryWednesday 16 July 2025 To view this content, please enable our use of cooki...

16/07/2025

2025-07-16

CULVER CITY, CALIFORNIA Apple TV+ today earned a record-breaking 81 Emmy Award nominations across 14 hit Apple Original titles for this year's 77th Emmy Awa...

15/07/2025

Open Call: Resilience Incubator for Independent Media in Brazil and Colombia

Independent media outlets in Brazil and Colombia are invited to apply for a new programme aimed at strengthening the long-term resilience of journalism in the f...

15/07/2025

Give Me the Backstory: Get to Know Sam Feder, the Filmmaker Behind Heightened Scrutiny

By Lucy Spicer One of the most exciting things about the Sundance Film Festival...

15/07/2025

A travs de una nueva colaboracin, usuarios de DiDi en nueve pases de Latinoamrica podrn acceder a Spotify Premium

Spotify une fuerzas con DiDi, la app l der en servicios de movilidad, delivery y...

15/07/2025

Invitation for South African filmmakers to submit films for the 98th Annual Academy Awards (Oscars) International Feature Film category

The National Film and Video Foundation (NFVF), an agency of the Department of Sp...

15/07/2025

L3Harris Showcases Counter-Drone Capability to British Soldiers at VANAHEIM

L3Harris put its CORVUS-RAVEN counter-small UAS capability into the hands of soldiers at VANAHEIM, showcasing its ability to provide passive signal detect, enha...

15/07/2025

Netflix Leads Streaming Growth in June on the Strength of Multiple Big Titles in Nielsen's 50th Report of The Gauge

Netflix Viewing Up 13.5% vs. May, Represents 42% of Monthly Gain for Streaming ...

15/07/2025

Amagi Names Sangeeta Chakraborty as Chief Revenue Officer

SAN FRANCISCO and BENGALARU, India Sangeeta Chakraborty has been named chief revenue officer at Amagi, a cloud-based software-as-a-service (SaaS) technology pro...

15/07/2025

ATSCs New VP of Standards Development Touts 3.0s Global Potential

As part of its mandate, the Advanced Television Systems Committee the U.S. organization tasked with developing advanced broadcast TV standards promotes ATSC 3.0...

15/07/2025

QuickLink Launches StudioPro Proton and StudioPro Fusion...

QuickLink, the leading global provider of multi-camera video productions and remote contributions, announces the launch of two innovative control panels Studi...

15/07/2025

DPA Microphones Welcomes Chris Kontopanos

Chris Kontopanos has joined leading high-quality microphone solutions manufacturer, DPA Microphones, as the company's new Regional Sales Manager for the Mid...

15/07/2025

Magnifi Brings Seamless AI-Powered Solutions to Revolutio...

Magnifi by VideoVerse, a global leader in AI-powered video automation, has launched a major platform upgrade built to simplify live and archival video editing f...

15/07/2025

Telestream IBC 2025 Showcase Powers Friction Free Scalabl...

Telestream, a global leader in media workflow technologies, will preview its latest innovations at IBC2025, Stand 7.B21. This year's showcase highlights how...

15/07/2025

IBC2025 elevates attendee experience with seamless GVB tr...

IBC2025 is set to transform the onsite experience for its global attendees with the launch of a pioneering new feature: full integration of Amsterdam's GVB ...

15/07/2025

ASB GlassFloor Ignites Beats N Buckets with Game-Changing...

Music, sport and technology collided in spectacular fashion as ASB GlassFloor powered the debut of Beats N Buckets, Germany's first Basketball meets Hip-Ho...

15/07/2025

The Collectv Names Industry Leader Peter Russell as new M...

The Collectv, the Emmy award-winning broadcast solutions and workflows consultancy - and winner of Broadcast Tech's Team of the Year is delighted to announc...

15/07/2025

SipRadius builds in security and performance for IP conne...

SipRadius, the expert in secure, low latency media transport, will showcase how broadcasters and media companies can take control of fragmented IP workflows at ...

15/07/2025

beIN ASIA PACIFIC Enhances Broadcast Distribution and Int...

Appear, the global leader in live production technology, today announced its strategic partnership with beIN ASIA PACIFIC, a leading multi-platform sports media...

15/07/2025

Broadpeak to Reveal Pioneering Solutions to Scale Monetiz...

Company marks 15 years of innovation at the show Broadpeak, a leader in streaming and monetization at scale, will return to IBC (Hall 1, Stand F83. RAI, Amster...

15/07/2025

Hitomi Broadcast Demonstrates MatchBox Everywhere at IBC2...

New software solutions extend trusted timing measurement across broadcast, venue, and professional AV applications Hitomi Broadcast, the market leader in audi...

15/07/2025

Telycam Unveils Elgato Stream Deck Plug-in for PTZ Camera Portfolio

SHENZHEN, China Telycam has added control options for its portfolio of pan-tilt-zoom (PTZ) cameras with a new plug-in for Elgato's Stream Deck family of con...

15/07/2025

Chyron LIVE Completes AWS Foundational Technical Review

MELVILLE, N.Y. Chyron said its Chyron LIVE cloud-native live production solution has successfully completed the Amazon Web Services (AWS) Foundational Technical...

15/07/2025

Xumo Expands Smart TV Portfolio With Westinghouse Launch

PHILADELPHIA The Xumo streaming platform joint venture between Comcast and Charter has announced the nationwide launch of a new line of Xumo TVs from Westinghou...

15/07/2025

Chris Kontopanos Joins DPA as Regional Sales Manager

LONGMONT, Colo. DPA Microphones said Chris Kontopanos has joined the microphone solutions provider as its new regional sales manager, Mid-Atlantic....

15/07/2025

SES Successfully Prices 1 Billion Dual-Tranche Bond Offering with Strong 5.5x Oversubscription

NOT FOR DISTRIBUTION IN OR INTO OR TO ANY PERSON LOCATED OR RESIDENT IN THE UNIT...

15/07/2025

Netflix and YRF Entertainment's Mandala Murders' Builds a Gripping World of Mythological Crime-Thriller

Back to All News Netflix and YRF Entertainment's Mandala Murders' Buil...

15/07/2025

Affordable Live Streaming Solutions for Every Level of Government

From Small Town Councils to State Chambers-There's a Broadcast Pix Bundle for You In today's digital-first world, every level of government-whether a ru...

15/07/2025

Winning the Streaming Shift: Adapting to Changing Viewer and Advertiser Behavior

Audiences and ad dollars are moving to streaming - media organizations are evolving to meet them there....