Sony Pixel Power calrec Sony

NVIDIA TensorRT Boosts Stable Diffusion 3.5 Performance on NVIDIA GeForce RTX and RTX PRO GPUs

12/06/2025

Generative AI has reshaped how people create, imagine and interact with digital content.

As AI models continue to grow in capability and complexity, they require more VRAM, or video random access memory. The base Stable Diffusion 3.5 Large model, for example, uses over 18GB of VRAM - limiting the number of systems that can run it well.

By applying quantization to the model, noncritical layers can be removed or run with lower precision. NVIDIA GeForce RTX 40 Series and the Ada Lovelace generation of NVIDIA RTX PRO GPUs support FP8 quantization to help run these quantized models, and the latest-generation NVIDIA Blackwell GPUs also add support for FP4.

NVIDIA collaborated with Stability AI to quantize its latest model, Stable Diffusion (SD) 3.5 Large, to FP8 - reducing VRAM consumption by 40%. Further optimizations to SD3.5 Large and Medium with the NVIDIA TensorRT software development kit (SDK) double performance.

In addition, TensorRT has been reimagined for RTX AI PCs, combining its industry-leading performance with just-in-time (JIT), on-device engine building and an 8x smaller package size for seamless AI deployment to more than 100 million RTX AI PCs. TensorRT for RTX is now available as a standalone SDK for developers.

RTX-Accelerated AI NVIDIA and Stability AI are boosting the performance and reducing the VRAM requirements of Stable Diffusion 3.5, one of the world's most popular AI image models. With NVIDIA TensorRT acceleration and quantization, users can now generate and edit images faster and more efficiently on NVIDIA RTX GPUs.

Stable Diffusion 3.5 quantized FP8 (right) generates images in half the time with similar quality as FP16 (left). Prompt: A serene mountain lake at sunrise, crystal clear water reflecting snow-capped peaks, lush pine trees along the shore, soft morning mist, photorealistic, vibrant colors, high resolution. To address the VRAM limitations of SD3.5 Large, the model was quantized with TensorRT to FP8, reducing the VRAM requirement by 40% to 11GB. This means five GeForce RTX 50 Series GPUs can run the model from memory instead of just one.

SD3.5 Large and Medium models were also optimized with TensorRT, an AI backend for taking full advantage of Tensor Cores. TensorRT optimizes a model's weights and graph - the instructions on how to run a model - specifically for RTX GPUs.

FP8 TensorRT boosts SD3.5 Large performance by 2.3x vs. BF16 PyTorch, with 40% less memory use. For SD3.5 Medium, BF16 TensorRT delivers a 1.7x speedup. Combined, FP8 TensorRT delivers a 2.3x performance boost on SD3.5 Large compared with running the original models in BF16 PyTorch, while using 40% less memory. And in SD3.5 Medium, BF16 TensorRT provides a 1.7x performance increase compared with BF16 PyTorch.

The optimized models are now available on Stability AI's Hugging Face page.

NVIDIA and Stability AI are also collaborating to release SD3.5 as an NVIDIA NIM microservice, making it easier for creators and developers to access and deploy the model for a wide range of applications. The NIM microservice is expected to be released in July.

TensorRT for RTX SDK Released Announced at Microsoft Build - and already available as part of the new Windows ML framework in preview - TensorRT for RTX is now available as a standalone SDK for developers.

Previously, developers needed to pre-generate and package TensorRT engines for each class of GPU - a process that would yield GPU-specific optimizations but required significant time.

With the new version of TensorRT, developers can create a generic TensorRT engine that's optimized on device in seconds. This JIT compilation approach can be done in the background during installation or when they first use the feature.

The easy-to-integrate SDK is now 8x smaller and can be invoked through Windows ML - Microsoft's new AI inference backend in Windows. Developers can download the new standalone SDK from the NVIDIA Developer page or test it in the Windows ML preview.

For more details, read this NVIDIA technical blog and this Microsoft Build recap.

Join NVIDIA at GTC Paris At NVIDIA GTC Paris at VivaTech - Europe's biggest startup and tech event - NVIDIA founder and CEO Jensen Huang yesterday delivered a keynote address on the latest breakthroughs in cloud AI infrastructure, agentic AI and physical AI. Watch a replay.

GTC Paris runs through Thursday, June 12, with hands-on demos and sessions led by industry leaders. Whether attending in person or joining online, there's still plenty to explore at the event.

Each week, the RTX AI Garage blog series features community-driven AI innovations and content for those looking to learn more about NVIDIA NIM microservices and AI Blueprints, as well as building AI agents, creative workflows, digital humans, productivity apps and more on AI PCs and workstations.

Plug in to NVIDIA AI PC on Facebook, Instagram, TikTok and X - and stay informed by subscribing to the RTX AI PC newsletter.

Follow NVIDIA Workstation on LinkedIn and X.

See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/rtx-ai-garage-gtc-paris-tensorrt-rtx-nim...
See more stories from nvidia

North America Stories

15/07/2025

Amagi Names Sangeeta Chakraborty as Chief Revenue Officer

SAN FRANCISCO and BENGALARU, India Sangeeta Chakraborty has been named chief revenue officer at Amagi, a cloud-based software-as-a-service (SaaS) technology pro...

15/07/2025

ATSCs New VP of Standards Development Touts 3.0s Global Potential

As part of its mandate, the Advanced Television Systems Committee the U.S. organization tasked with developing advanced broadcast TV standards promotes ATSC 3.0...

15/07/2025

QuickLink Launches StudioPro Proton and StudioPro Fusion...

QuickLink, the leading global provider of multi-camera video productions and remote contributions, announces the launch of two innovative control panels Studi...

15/07/2025

DPA Microphones Welcomes Chris Kontopanos

Chris Kontopanos has joined leading high-quality microphone solutions manufacturer, DPA Microphones, as the company's new Regional Sales Manager for the Mid...

15/07/2025

Magnifi Brings Seamless AI-Powered Solutions to Revolutio...

Magnifi by VideoVerse, a global leader in AI-powered video automation, has launched a major platform upgrade built to simplify live and archival video editing f...

15/07/2025

Telestream IBC 2025 Showcase Powers Friction Free Scalabl...

Telestream, a global leader in media workflow technologies, will preview its latest innovations at IBC2025, Stand 7.B21. This year's showcase highlights how...

15/07/2025

IBC2025 elevates attendee experience with seamless GVB tr...

IBC2025 is set to transform the onsite experience for its global attendees with the launch of a pioneering new feature: full integration of Amsterdam's GVB ...

15/07/2025

ASB GlassFloor Ignites Beats N Buckets with Game-Changing...

Music, sport and technology collided in spectacular fashion as ASB GlassFloor powered the debut of Beats N Buckets, Germany's first Basketball meets Hip-Ho...

15/07/2025

The Collectv Names Industry Leader Peter Russell as new M...

The Collectv, the Emmy award-winning broadcast solutions and workflows consultancy - and winner of Broadcast Tech's Team of the Year is delighted to announc...

15/07/2025

SipRadius builds in security and performance for IP conne...

SipRadius, the expert in secure, low latency media transport, will showcase how broadcasters and media companies can take control of fragmented IP workflows at ...

15/07/2025

beIN ASIA PACIFIC Enhances Broadcast Distribution and Int...

Appear, the global leader in live production technology, today announced its strategic partnership with beIN ASIA PACIFIC, a leading multi-platform sports media...

15/07/2025

Broadpeak to Reveal Pioneering Solutions to Scale Monetiz...

Company marks 15 years of innovation at the show Broadpeak, a leader in streaming and monetization at scale, will return to IBC (Hall 1, Stand F83. RAI, Amster...

15/07/2025

Hitomi Broadcast Demonstrates MatchBox Everywhere at IBC2...

New software solutions extend trusted timing measurement across broadcast, venue, and professional AV applications Hitomi Broadcast, the market leader in audi...

15/07/2025

Telycam Unveils Elgato Stream Deck Plug-in for PTZ Camera Portfolio

SHENZHEN, China Telycam has added control options for its portfolio of pan-tilt-zoom (PTZ) cameras with a new plug-in for Elgato's Stream Deck family of con...

15/07/2025

Chyron LIVE Completes AWS Foundational Technical Review

MELVILLE, N.Y. Chyron said its Chyron LIVE cloud-native live production solution has successfully completed the Amazon Web Services (AWS) Foundational Technical...

15/07/2025

Xumo Expands Smart TV Portfolio With Westinghouse Launch

PHILADELPHIA The Xumo streaming platform joint venture between Comcast and Charter has announced the nationwide launch of a new line of Xumo TVs from Westinghou...

15/07/2025

Chris Kontopanos Joins DPA as Regional Sales Manager

LONGMONT, Colo. DPA Microphones said Chris Kontopanos has joined the microphone solutions provider as its new regional sales manager, Mid-Atlantic....

15/07/2025

Deadline Extended - Create a Project G-Assist Plug-In for a Chance to Win an NVIDIA GeForce RTX GPU and Laptop

Submissions for NVIDIA's Plug and Play: Project G-Assist Plug-In Hackathon a...

14/07/2025

L3Harris Secures Contract for Royal Moroccan Air Force C-130 Fleet Upgrade

The Royal Moroccan Air Force hosted a C-130 award ceremony with L3Harris Technologies President of Intelligence, Surveillance and Reconnaissance Jason Lambert a...

14/07/2025

Nielsen launches CTV Ad Spend Data into its UK Ad Intel product

UK launch set for September, following on from recent US and Germany releases London - UK, July 14, 2025 - Nielsen, a global leader in audience measurement, da...

14/07/2025

Telycam Unveils Elgato Stram Deck Plug-in for PTZ Camera Portfolio

SHENZHEN, China Telycam has added control options for its portfolio of pan-tilt-zoom (PTZ) cameras with a new plug-in for Elgato's Stream Deck family of con...

14/07/2025

Nominations Now Open for Best of Show Awards at IBC2025

The awards are open to IBC2025 show exhibitors and offer a valuable platform for companies to raise awareness for the new products and solutions they will be la...

14/07/2025

Clear-Com Expands Arcadia Central Station with Updates In...

Clear-Com is excited to announce a significant expansion of the award-winning Arcadia Central Station with new software updates. Together, these releases dram...

14/07/2025

Daktronics and Grass Valley Announce Strategic Partnershi...

Grass Valley and Daktronics today announced a strategic technology partnership that unites Grass Valley's live production expertise with Daktronics' lea...

14/07/2025

Veset Partners with swXtch io to Deliver Cloud Multicasti...

Cloud playout solutions provider, Veset, and swXtch.io, provider of cloud-based multicast and reliable networking solutions, are partnering to deliver multicast...

14/07/2025

Tintri Supercharges Global Partner Programme with Powerfu...

Tintri , a DDN subsidiary and leader in AI-powered data management solutions, today announces a game-changing and award-winning refresh to its global Partner P...

14/07/2025

HighField AI Now Commercially Available Advancing Broadca...

HighField AI, the broadcast industry's first agentic and multimodal AI platform for automating graphics production, is now commercially available. Developed...

14/07/2025

Encompass powers DAZN global broadcast of FIFA Club World...

Encompass Digital Media has today announced that it has been delivering a complex global services project for DAZN, the world s leading sports entertainment pla...

14/07/2025

MASV Showcases the Future of Mass File Movement and Monet...

MASV (massive.io), the fastest, most reliable large file transfer platform trusted by over 10,000 global organizations, announced today it will showcase its nex...

14/07/2025

Calrec to unveil new Argo Software Updates at IBC 2025 an...

Calrec is introducing a series of usability, customisation and system enhancements across the entire range of Argo consoles at IBC 2025, on Stand 8.C47. Look ou...

14/07/2025

ITV selects Fincons Group as strategic partner for their...

Fincons Group, IT business consultancy and systems integrator company with more than 40 years of experience in the market, is proud to announce a new engagement...

14/07/2025

Intinor Showcases Enhanced User Experience and Workflow F...

Intinor returns to IBC to present the latest updates to its Direkt series. A key development on show is Intinor Direkt Management (IDM), a web-based interface f...

14/07/2025

NVIDIA CEO Jensen Huang Promotes AI in Washington, DC and China

This month, NVIDIA founder and CEO Jensen Huang promoted AI in both Washington, D.C. and Beijing - emphasizing the benefits that AI will bring to business and s...

14/07/2025

Sony's New Coach's Headset Was Designed in Collaboration With the NFL

Sony's New Coach's Headset Was Designed in Collaboration With the NFL Featuring a custom dynamic boom mic, the headset will debut in the fall By Dan Da...

14/07/2025

Live From MLB All-Star 2025: MLB Network's Marc Caiafa and Tom Guidice on the Weeklong Efforts in Atlanta

Live From MLB All-Star 2025: MLB Network's Marc Caiafa and Tom Guidice on th...

14/07/2025

Netflix ISP Speed Index for June 2025

Back to All News Netflix ISP Speed Index for June 2025 Product 14 July 2025 Global Link copied to clipboard One percent of Internet Service Providers (ISP...

14/07/2025

Netflix Begins Filming 'The Map of Longing' With Alcia Falc, Pablo lvarez, and Georgina Amors

Back to All News Netflix Begins Filming The Map of Longing With Al cia Falc , P...

14/07/2025

100 Days In: Building Forward at Grass Valley

This July marks 100 days since I officially stepped into the role of CEO at Grass Valley, a milestone not of abrupt change, but of confident continuity. Unlike...

13/07/2025

Netflix Expands B&B Universe' with Yu Jae Seok's B&B (WT) and Kian's Bizarre B&B Season 2

Back to All News Netflix Expands B&B Universe' with Yu Jae Seok's B&B ...

13/07/2025

Live From MLB All-Star 2025: MLB Network Marks Five Straight Years of Onsite MLB Draft Coverage

Live From MLB All-Star 2025: MLB Network Marks Five Straight Years of Onsite MLB...

12/07/2025

Retracing Ryan Coogler's Sundance Institute Journey

Ryan Coogler accepting the 2013 Vanguard Award. Photo by Alberto E. Rodriguez. Editor's Note: In honor of Fruitvale Station s 12th anniversary, we're d...

12/07/2025

Key Code Education Launches New Adobe Premiere Pro Engineer Certification

Key Code Education, the professional training division of Key Code Media, proudly announces a major update to its Adobe Premiere Pro Engineering and Advanced Op...

12/07/2025

Key Code Education Launches New Hands-On Training Programs for Ross Video, Vizrt, EditShare, and SNS

Key Code Education, a leader in instructor-led post production training for over...

12/07/2025

TV Station Groups Launch Texas Flood Relief Efforts

As the death toll continues to mount, with at least 120 killed and more than 170 people still missing on July 10 from devastating Texas floods, a number of broa...

12/07/2025

DirecTV Adds ViX Premium With Ads to MiEspaol Genre Pack

EL SEGUNDO, Calif., and MIAMI -DirecTV and TelevisaUnivision have signed a deal that will make the ad-supported premium subscription tier of ViX, ViX Premium wi...

11/07/2025

2025 Sundance Institute Producers Lab Fellows Announced

PARK CITY, UTAH, July 11, 2025 - The nonprofit Sundance Institute announced today the 11 producers chosen for its annual Producers Labs, returning to Ucross Fou...

11/07/2025

L3Harris Delivers First P-8A Poseidon Aircraft to US Navy

L3Harris Technologies President of Intelligence, Surveillance and Reconnaissance Jason Lambert and General Manager of L3Harris Waco facility Sean Ling held a ce...

11/07/2025

WETA Launches WETA+ Free Streaming Service

ARLINGTON, Va. WETA, the flagship public media station in the national capital area, has launched WETA+, a new streaming service tailored for the local Washingt...

11/07/2025

TV Tech's Top Regulatory Stories of 2025

The Federal Communications Commission has emerged as one of the central players in the broadcast TV landscape in 2025, with its deregulatory policies sparking h...

11/07/2025

Calrec to Feature Suite of Interconnected Audio Solutions at IBC2025

Calrec will introduce usability, customization and system enhancements across its entire range of Argo consoles during IBC2025, Sept. 12-15, at the RAI Amsterda...