Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, software, tools and accelerations for RTX PC users.
The era of the AI PC is here, and it's powered by NVIDIA RTX and GeForce RTX technologies. With it comes a new way to evaluate performance for AI-accelerated tasks, and a new language that can be daunting to decipher when choosing between the desktops and laptops available.
While PC gamers understand frames per second (FPS) and similar stats, measuring AI performance requires new metrics.
Coming Out on TOPS The first baseline is TOPS, or trillions of operations per second. Trillions is the important word here - the processing numbers behind generative AI tasks are absolutely massive. Think of TOPS as a raw performance metric, similar to an engine's horsepower rating. More is better.
Compare, for example, the recently announced Copilot+ PC lineup by Microsoft, which includes neural processing units (NPUs) able to perform upwards of 40 TOPS. Performing 40 TOPS is sufficient for some light AI-assisted tasks, like asking a local chatbot where yesterday's notes are.
But many generative AI tasks are more demanding. NVIDIA RTX and GeForce RTX GPUs deliver unprecedented performance across all generative tasks - the GeForce RTX 4090 GPU offers more than 1,300 TOPS. This is the kind of horsepower needed to handle AI-assisted digital content creation, AI super resolution in PC gaming, generating images from text or video, querying local large language models (LLMs) and more.
Insert Tokens to Play TOPS is only the beginning of the story. LLM performance is measured in the number of tokens generated by the model.
Tokens are the output of the LLM. A token can be a word in a sentence, or even a smaller fragment like punctuation or whitespace. Performance for AI-accelerated tasks can be measured in tokens per second.
Another important factor is batch size, or the number of inputs processed simultaneously in a single inference pass. As an LLM will sit at the core of many modern AI systems, the ability to handle multiple inputs (e.g. from a single application or across multiple applications) will be a key differentiator. While larger batch sizes improve performance for concurrent inputs, they also require more memory, especially when combined with larger models.
The more you batch, the more (time) you save. RTX GPUs are exceptionally well-suited for LLMs due to their large amounts of dedicated video random access memory (VRAM), Tensor Cores and TensorRT-LLM software.
GeForce RTX GPUs offer up to 24GB of high-speed VRAM, and NVIDIA RTX GPUs up to 48GB, which can handle larger models and enable higher batch sizes. RTX GPUs also take advantage of Tensor Cores - dedicated AI accelerators that dramatically speed up the computationally intensive operations required for deep learning and generative AI models. That maximum performance is easily accessed when an application uses the NVIDIA TensorRT software development kit (SDK), which unlocks the highest-performance generative AI on the more than 100 million Windows PCs and workstations powered by RTX GPUs.
The combination of memory, dedicated AI accelerators and optimized software gives RTX GPUs massive throughput gains, especially as batch sizes increase.
Text-to-Image, Faster Than Ever Measuring image generation speed is another way to evaluate performance. One of the most straightforward ways uses Stable Diffusion, a popular image-based AI model that allows users to easily convert text descriptions into complex visual representations.
With Stable Diffusion, users can quickly create and refine images from text prompts to achieve their desired output. When using an RTX GPU, these results can be generated faster than processing the AI model on a CPU or NPU.
That performance is even higher when using the TensorRT extension for the popular Automatic1111 interface. RTX users can generate images from prompts up to 2x faster with the SDXL Base checkpoint - significantly streamlining Stable Diffusion workflows.
ComfyUI, another popular Stable Diffusion user interface, added TensorRT acceleration last week. RTX users can now generate images from prompts up to 60% faster, and can even convert these images to videos using Stable Video Diffuson up to 70% faster with TensorRT.
TensorRT acceleration can be put to the test in the new UL Procyon AI Image Generation benchmark, which delivers speedups of 50% on a GeForce RTX 4080 SUPER GPU compared with the fastest non-TensorRT implementation.
TensorRT acceleration will soon be released for Stable Diffusion 3 - Stability AI's new, highly anticipated text-to-image model - boosting performance by 50%. Plus, the new TensorRT-Model Optimizer enables accelerating performance even further. This results in a 70% speedup compared with the non-TensorRT implementation, along with a 50% reduction in memory consumption.
Of course, seeing is believing - the true test is in the real-world use case of iterating on an original prompt. Users can refine image generation by tweaking prompts significantly faster on RTX GPUs, taking seconds per iteration compared with minutes on a Macbook Pro M3 Max. Plus, users get both speed and security with everything remaining private when running locally on an RTX-powered PC or workstation.
The Results Are in and Open Sourced But don't just take our word for it. The team of AI researchers and engineers behind the open-source Jan.ai recently integrated TensorRT-LLM into its local chatbot app, then tested these optimizations for themselves.
Source: Jan.ai The researchers tested its implementation of TensorRT-LLM against the open-source llama.cpp inference engine across a variety of GPUs and CPUs used by the community. They found that TensorRT is 30-70% faster than llam
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
01/06/2026
January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026
Throughout the week, Dolby brings to life the latest innovatio...
01/05/2026
January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...
01/04/2026
January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION
Douyin Users Can Now Create And Share Videos With Stun...
20/01/2026
SVG Sit-Down: BitFire's Jim Akimchuk on the Future of Cloud, Remote, Softwar...
20/01/2026
From architecture to reality: What CBC's Milano Cortina 2026 workflows tell ...
20/01/2026
Milano Cortina 2026: Making the Winter Olympics and Paralympics workflow work fo...
20/01/2026
Countdown to Milano Cortina 2026: SVG Launches SportsTechLive Blog in Lead-up to...
20/01/2026
Grand finale: Rally Saudi Arabia pushes WRC to make the most of cameras and dron...
20/01/2026
State of Change: What's Happening in the Remote-Production Business?NEP Group, Game Creek Video, Dome Productions, Mobile TV Group, Program Productions exec...
20/01/2026
CFP National Championship 2026: A Look Back at SVG's Complete CoverageGo behind the scenes with ESPN, Game Creek Video, Van Wagner, and the champion Indiana...
20/01/2026
The Gotham, Film Independent, and Creators Coalition on AI Join Alliance
by Michelle Satter, Founding Senior Director, Sundance Institute's Artist Programs...
20/01/2026
The National Film and Video Foundation (NFVF), an agency of the Department of Sp...
20/01/2026
December, the first month of winter and the holiday season, has traditionally encouraged families to spend time together in front of the television. During this...
20/01/2026
During December, streaming's share of TV viewing in Mexico settled at 24.3%, an increase of 0.1 share points from the previous month.
Disclaimer: YUMI TV,...
20/01/2026
Christmas Day is Most-Streamed Day Ever with 55 Billion Viewing Minutes, led by...
20/01/2026
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/01/2026
Disguise and ASB GlassFloor today announced a strategic partnership that turns the arena floor into a fully customizable interactive digital surface. This will ...
20/01/2026
Last summer, ITV celebrated the tenth anniversary of its popular reality dating show, Love Island. To ramp up the excitement of series 12 (launched in the UK on...
20/01/2026
Berklee's Mark Ethier Named One of SPIN's Most Influential People in Mus...
20/01/2026
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/01/2026
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/01/2026
Emergent, a provider of AI-powered media production solutions, today announced its presence at ISE 2026 in Barcelona, taking place February 3 6, where it will u...
20/01/2026
Back to All News
Netflix and Warner Bros. Discovery Amend Agreement to All-Cash Transaction
Business
20 January 2026
Global
Link copied to clipboard
All-C...
20/01/2026
Back to All News
Now You Can Play a Role in Our Live Events With Real-Time Voting
Elmar Nubbemeyer
VP, Member Product
Product
20 January 2026
Global
Link...
19/01/2026
CFP National Championship 2026: Indiana Football's Digital Crew Generate Hyp...
19/01/2026
CFP National Championship 2026: Van Wagner Relies on Experience for Indiana-Miam...
19/01/2026
CFP National Championship 2026: Game Creek Video Fields Premiere IP Compound for...
19/01/2026
CFP National Championship 2026: For a Slew of Studio Shows, ESPN Turns Hard Rock...
19/01/2026
CFP National Championship 2026: ESPN's Sweeping Live Game Production Leverag...
19/01/2026
The new L3Harris NOVA binocular system is the every-soldier goggle' - and the beginning of a new era for night-vison capability....
19/01/2026
The Majority of Those Surveyed Prefer to Buy Brands Who Advertise in Content Tha...
19/01/2026
Back to All News
Straight to Hell' Sets April 27 Premiere: Teaser Trailer,...
19/01/2026
New Workflow Enhancements for All Broadcast Pix Systems, Plus 4-Channel ISO Recording for Hybrid and Roadie Tyngsboro, Mass. - January 2025 - Broadcast Pix ann...
19/01/2026
Monday 19 January 2026
Sky News reinvents News at Ten for the modern audience with The Wrap
Sky News today announces The Wrap, a bold evolution of its 10pm ou...
19/01/2026
Back to All News
Netflix Announces the Series Nights & Days, Directed by Kamila...
19/01/2026
DPA Microphones is striking a new chord in piano miking with the debut of its DPK2015 Piano Stereo Kit, an out-of-the-box, plug-and-play miking solution that ta...
19/01/2026
DPA Microphones will present its 4099 CORE Instrument Microphone and redesigned...
19/01/2026
Trianel relies on Arvato Systems for future-proof data center operations
Moving to the cloud: Arvato Systems implements extensive IT migration project
Migrat...
19/01/2026
Celebrating 21 Years of the RT Choice Music Prize
RT Choice Music Prize
In association with IMRO and IRMA
Irish Album of the Year 2025 - Shortlist Announce...
18/01/2026
January 18 2026, 21:30 (PST) Dolby Cinema Arrives in Bengaluru: Dolby Laborator...
17/01/2026
Lightware, an industry leader in signal management, is helping to elevate the Google Meet experience with the introduction of an integration with Taurus UCX. Th...
17/01/2026
Clear-Com announced a strategic partnership with NETGEAR AV to offer four high-performance networking switches as original equipment manufacturer (OEM) solutio...
17/01/2026
QuickLink, a leading provider of award-winning video production and remote guest contribution solutions, today announced a new U.S. distribution partnership wit...
17/01/2026
NUGEN Audio announces the latest evolution of MasterCheck, its industry-trusted optimization plug-in for cross-platform mastering and loudness verification. Des...
17/01/2026
BHV has announced that it will highlight its new SportsBox remote vision mixer and picture processor at ISE in Booth #4H900 marking its official introduction in...
17/01/2026
Clear-Com will return to ISE 2026 to demonstrate its latest intercom solutions for broadcast, live production, and AV professionals. Exhibiting at Stand #4P700...
17/01/2026
Grass Valley, a leading provider in live production solutions, will exhibit at ISE 2026 in Barcelona from February 3 6, presenting its vision for how modern Bro...
17/01/2026
Utelogy Corporation, the global leader in connected workplace management, monitoring, control, automation and analytics closes out a record-breaking 2025. The c...
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, software, tools and accelerations for RTX PC users.
