Modern AI applications increasingly rely on models that combine huge parameter counts with multi-million-token context windows. Whether it is AI agents following months of conversation, legal assistants reasoning through gigabytes of case law as big as an entire encyclopedia set, or coding copilots navigating sprawling repositories, preserving long-range context is essential for relevance and coherence. On top of that, users expect fast, interactive responses.
The growing demand to decode such massive amounts of data-and let multiple GPUs quickly scale and communicate with each other-underscores the importance of FP4 compute and the high-bandwidth large NVLink domain provided by NVIDIA Blackwell systems. Helix Parallelism, introduced in this blog, is co-designed with Blackwell. It enables up to a 32x increase in the number of concurrent users at a given latency, compared to the best known prior parallelism methods for real-time decoding with ultra-long context.
In other words, it lets AI agents and virtual assistants serve more people, faster than ever before.
(Note: Context in this blog refers to the sequence of previously generated tokens, whose intermediate key and value representations are stored as KV cache and accessed at every decoding step.)
Decoding bottlenecks: KV cache and FFN weight reads To support real-time decoding at scale, a system must overcome two major bottlenecks during the decoding (aka generation) phase:
Key-Value (KV) cache streaming: When handling multi-million-token contexts, each GPU must read a massive history of past tokens (KV cache) from DRAM per sample. This constant streaming can, in turn, saturate DRAM bandwidth, increase token-to-token latency (TTL), and quickly become a major bottleneck as context length grows.
Feed-Forward Network (FFN) weight loading: During autoregressive decoding, generating every new token requires loading large Feed-Forward Network (FFN) weights from DRAM. In low latency scenarios with small batch sizes, this memory access cost is not well amortized, making FFN weight reads a dominant source of latency.
These two bottlenecks, KV cache streaming and FFN weight loading, are difficult to optimize simultaneously using traditional parallelism strategies.
Let's take Tensor Parallelism (TP) as an example: Increasing TP can help reduce FFN stalls by distributing weight loading across multiple GPUs and improving TTL, but only up to a point. In attention schemes like Grouped Query Attention (GQA)-used in Llama models-or Multi-Latent Attention (MLA)-found in DeepSeek models-multiple query heads share a limited number of KV heads. As illustrated in Figure 2(c), when TP exceeds the number of KV heads, the system ends up duplicating the multi-million-token KV cache per sample across GPUs for self-attention. As a result, KV read volume stays high even with increased TP, once again saturating DRAM bandwidth and limiting scalability. In the case of MLA, the upper limit for TP is just one to avoid duplication of KV cache.
So how can developers scale both model size and context length without sacrificing real-time interactivity? Helix Parallelism offers a path forward.
Helix execution flow Helix is a hybrid sharding strategy that disaggregates the parallelism strategies of attention and FFNs in a temporal pipeline, effectively addressing both KV cache and FFN weight-read bottlenecks during multi-million-token decoding.
Figure 1 (below) shows how Helix orchestrates the execution of attention and FFN within a single transformer layer. Inspired by the structure of a DNA helix, Helix interweaves multiple dimensions of parallelism-KV, tensor, and expert-into a unified execution loop. By decoupling the parallelism strategy used for attention and FFN, Helix allows each stage to operate in a configuration tuned to its own bottleneck, all while reusing the same pool of GPUs. Helix reuse approach keeps GPUs efficiently utilized across stages, eliminating idle time as computation flows through the model.
data-src=https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-png.webp alt=A diagram showing the execution flow of Helix Parallelism. Helix reuses the same pool of N GPUs per layer by switching between N=KVPxTPA during attention and N=TPFxEP during FFN. class=lazyload wp-image-102939 data-srcset=https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-png.webp 1522w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-282x300-png.webp 282w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-625x665-png.webp 625w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-108x115-png.webp 108w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-768x817-png.webp 768w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-1443x1536-png.webp 1443w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-645x687-png.webp 645w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-85x90-png.webp 85w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-362x385-png.webp 362w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-103x110-png.webp 103w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-1024x1090-png.webp 1024w, https://developer-blogs.nvidia.com/wp-content/uploads/2025/07/image2-2-507x540-png.webp 507w data-sizes=(max-width: 1522px) 100vw, 1522px />Figure 1. Execution flow of Helix Parallelism. Helix reuses the same pool of N GPUs per layer by switching between N=KVPxTPA during attention and N=TPFxEP during FFN.
Attention phase Helix applies KV Parallelism (KVP) by sharding the multi-million-token KV cache along the sequence dimension across KVP GPUs, while applying Tensor Parallelism across attention heads (TPA), where TP
Most recent headlines
11/12/2025
Dalet, a leading provider of cloud-native, end-to-end media workflow solutions, ...
28/11/2025
Nadia Fall attends the 2025 Sundance Film Festival premiere of Brides at the Egyptian Theatre on January 24, 2025, in Park City, Utah. (Photo by Donyale West/...
28/11/2025
It's easy to ignore those little red update available badges. But when it ...
28/11/2025
WASHINGTON Federal Communications Commission has released a tentative agenda for the December Open Commission Meeting scheduled for Thursday, December 18, 2025 ...
28/11/2025
The Professional Fighters League is looking to super-serve fans of mixed martial...
28/11/2025
Fubo has released in beta on select Roku devices a new feature that lets users display up to four simultaneous streams at once....
28/11/2025
The WNBA playoffs and Week 4 of the NFL regular season highlight the list of live sports events airing on television this weekend....
28/11/2025
The 32nd class of honorees to the B+C Hall of Fame took to the stage at New York's Ziegfeld Ballroom on September 26 for a gala induction event. Click below...
28/11/2025
We hold in our hands the very last Next Text for Next TV, the weekly back-and-fo...
28/11/2025
DirecTV said it made a deal with EchoStar to buy EchoStar's video businesses, including satellite-TV provider Dish TV and virtual MVPD Sling TV, for $1 plus...
28/11/2025
The Broadcasting+Cable Hall of Fame, the premier industry event paying tribute to the influencers, innovators and shining lights of broadcast, cable and streami...
28/11/2025
Friday 28 November 2025
Sky Sports x Slawn drop limited-edition football jersey...
28/11/2025
Rohde & Schwarz shows resilience in a challenging environment, revenue exceeds t...
28/11/2025
Unwrapped: The Toy Show Appeal - airing this Sunday on RT One and RT Player- s...
27/11/2025
LONDON Vizrt has added several AI-driven advanced features offering improved speed, intelligence and accuracy in the newest version of its media asset managemen...
27/11/2025
Prime Video has launched AI-powered video season recaps in a beta version for select English-language Prime Original series in the U.S., a move Amazon is callin...
27/11/2025
Back to All News
Netflix's Raat Akeli Hai: The Bansal Murders Marks a Grand...
27/11/2025
27 Nov 2025
GSMA brings M360 Eurasia 2026 to Samarkand in partnership with VEON...
27/11/2025
Tahar Rahim and Izuka Hoyle star in the gripping six-part Sky Original from Acad...
27/11/2025
Thursday 27 November 2025
Sky Arts Reveals the Nation's Greatest Basslines - and Queen Reign Supreme
The UK's most iconic basslines have been revealed...
27/11/2025
Back to All News
Stranger Things 5': Prepare for One Last Adventure With O...
27/11/2025
The media industry has a paradox at its core. It's an industry built on light, color and imagination, yet behind the scenes, it's powered by one of the ...
27/11/2025
Rating reflects rating progress across areas including policies, diversity & inclusion, health & safety and Net Zero leadership
Winchester, UK, 27 November 202...
27/11/2025
What are the industry standards for Retail Media? Kathryn explains that certification is based on the IAB Europe Retail Media Measurement Standards and the IAB ...
27/11/2025
World champion boxer and Irish sporting icon Katie Taylor will be in studio this...
27/11/2025
Roblox, one of the world's most popular online gaming platforms for primary ...
27/11/2025
Black Friday is leveling up. Get ready to score one of the biggest deals of the season - 50% off the first three months of a new GeForce NOW Ultimate membership...
26/11/2025
SVG Sit-Down: Prime Video EP Mike Muriano Previews Massive Black Friday Slate Fe...
26/11/2025
A cinematic snow sculpture at the 1995 Sundance Film Festival. Photo by Randall Michelson...
26/11/2025
Book podcasts are booming. On Spotify, you'll find everything from celebrity book clubs to deep dives with bestselling authors. And in markets where audiobo...
26/11/2025
Mumbai, November 24, 2025: In a first-of-its-kind initiative, JioStar, in collab...
26/11/2025
LONDON Factual content producer ITN Productions has launched a new low-latency IP gallery for news bulletins....
26/11/2025
MIAMI TelevisaUnivision said it struck a new multiyear distribution agreement with YouTube TV that includes distribution of TelevisaUnivision's U.S. network...
26/11/2025
OpenDrives, Inc., a leader in software-defined data storage and data services, today announced the launch of the Atlas Corporate Creative Solution. This new Atl...
26/11/2025
Disguise, the industry-leading company powering the world's biggest live performances, is partnering with pioneering LED wall manufacturer DVS to give atten...
26/11/2025
HighField AI, the pioneer in agentic and multimodal automation for broadcast and media production, today announced the expansion of its global channel partner n...
26/11/2025
As high-stakes Premier League fixtures approach and additional premium content launches, with MONO positioning themselves to dominate Thailand's sports stre...
26/11/2025
Hosting a wide variety of events from high-intensity NHL games to complex live music concerts and major entertainment productions, Montreal's 21,000 capacit...
26/11/2025
Vizrt, the leader in live production technology revolutionizing viewer engagement and experience, releases AI-driven advances focusing on speed, intelligence, a...
26/11/2025
ITN Productions, an award-winning factual content producer, today launched a new low-latency IP gallery for news bulletins. Responsible for delivering a leading...
26/11/2025
Ikegami reports ongoing advances throughout 2025 in developing and delivering coordinated television production solutions that maximize quality, versatility and...
26/11/2025
Following the Nov. 21 blackout of NBCUniversal channels on Fubo, the two sides have traded barbs about their inability to reach a new carriage deal....
26/11/2025
LONDON As TV sports rights become increasingly important for both broadcasters and streamers, Ampere Analysis predicts global investment in the genre will surpa...
26/11/2025
LOS ANGELES Vubiquity said it has achieved the Amazon Web Services (AWS) Media & Entertainment Competency as part of the AWS Partner Network (APN). This designa...
26/11/2025
WASHINGTON The Federal Communications Commission's Enforcement Bureau said it has entered into a consent decree with Comcast calling for the cable company t...
26/11/2025
Berklee Named to the Hollywood Reporters Top Music Schools List The publication highlights the college's screen scoring program, industry partnerships, and ...
26/11/2025
Back to All News
Animated Series Love Through a Prism' Casts New Light on ...
26/11/2025
Back to All News
NALIP Unveils Fifth Cohort of Director Incubator
Social Impact
26 November 2025
United States
Link copied to clipboard
The National Assoc...
26/11/2025
YouView Achieves Greenly Gold Certification for SustainabilityNov 26, 2025
YouView is proud to announce a Gold Certification award from Greenly for our perform...
Modern AI applications increasingly rely on models that combine huge parameter counts with multi-million-token context windows. Whether it is AI agents following months of conversation, legal assistants reasoning through gigabytes of case law as big as an entire encyclopedia set, or coding copilots navigating sprawling repositories, preserving long-range context is essential for relevance and coherence. On top of that, users expect fast, interactive responses. 
