Inference has emerged as the new frontier of complexity in AI. Modern models are evolving into agentic systems capable of multi-step reasoning, persistent memory, and long-horizon context-enabling them to tackle complex tasks across domains such as software development, video generation, and deep research. These workloads place unprecedented demands on infrastructure, introducing new challenges in compute, memory, and networking that require a fundamental rethinking of how inference is scaled and optimized.
Among these challenges, processing massive context for a specific class of workloads has become increasingly critical. In software development, for example, AI systems must reason over entire codebases, maintain cross-file dependencies, and understand repository-level structure-transforming coding assistants from autocomplete tools into intelligent collaborators. Similarly, long-form video and research applications demand sustained coherence and memory across millions of tokens. These requirements are pushing the boundaries of what current infrastructure can support.
To address this shift, the NVIDIA SMART framework provides a path forward-optimizing inference across scale, multidimensional performance, architecture, ROI, and the broader technology ecosystem. It emphasizes a full-stack disaggregated infrastructure that enables efficient allocation of compute and memory resources. Platforms like NVIDIA Blackwell and NVIDIA GB200 NVL72, combined with NVFP4 for low-precision inference and open source software such as NVIDIA TensorRT-LLM and NVIDIA Dynamo, are redefining inference performance across the AI landscape.
This blog explores the next evolution in disaggregated inference infrastructure and introduces NVIDIA Rubin CPX-a purpose-built GPU designed to meet the demands of long-context AI workloads with greater efficiency and ROI.
Disaggregated inference: a scalable approach to AI complexity Inference consists of two distinct phases: the context phase and the generation phase, each placing fundamentally different demands on infrastructure. The context phase is compute-bound, requiring high-throughput processing to ingest and analyze large volumes of input data to produce the first token output result. In contrast, the generation phase is memory bandwidth-bound, relying on fast memory transfers and high-speed interconnects, such as NVLink, to sustain token-by-token output performance.
Disaggregated inference enables these phases to be processed independently, enabling targeted optimization of compute and memory resources. This architectural shift improves throughput, reduces latency, and enhances overall resource utilization (Figure 1).
data-src=https://developer-blogs.nvidia.com/wp-content/uploads/2025/09/Disaggregated-inference.gif alt=Diagram of a disaggregated inference pipeline. Documents/databases/videos feed a context processor (shown as GPU B with a swap to GPU A); its output goes to a key-value cache read by a GPU B generation node to produce results. Labels note GPU A is optimized for long-context processing, while GPU B delivers strong TCO for both context and generation. class=lazyload wp-image-105631/>Figure 1. Optimizing inference by aligning GPU capabilities with context and generation workloads
However, disaggregation introduces new layers of complexity, requiring precise coordination across low-latency KV cache transfers, LLM-aware routing, and efficient memory management. NVIDIA Dynamo serves as the orchestration layer for these components, and its capabilities played a pivotal role in the latest MLPerf Inference results. Learn how disaggregation with Dynamo on GB200 NVL72 set new performance records.
To capitalize on the benefits of disaggregated inference-particularly in the compute-intensive context phase-specialized acceleration is essential. Addressing this need, NVIDIA is introducing Rubin CPX GPU-a purpose-built solution designed to deliver high-throughput performance for high-value long-context inference workloads while seamlessly integrating into disaggregated infrastructure.
Rubin CPX: built to accelerate long-context processing The Rubin CPX GPU is designed to enhance long-context performance, complementing existing infrastructure while delivering scalable efficiency and maximizing ROI in context-aware inference deployments. Rubin CPX, built with the Rubin architecture, delivers breakthrough performance for the compute-intensive context phase of inference. It features 30 petaFLOPs of NVFP4 compute, 128 GB of GDDR7 memory, hardware support for video decoding and encoding, and 3x attention acceleration (compared to NVIDIA GB300 NVL72).
Optimized for efficiently processing long sequences, Rubin CPX is critical for high-value inference use cases like software application development and HD video generation. Designed to complement existing disaggregated inference architectures, it enhances throughput and responsiveness while maximizing ROI for large-scale generative AI workloads.
Rubin CPX works in tandem with NVIDIA Vera CPUs and Rubin GPUs for generation-phase processing, forming a complete, high-performance disaggregated serving solution for long-context use cases. The NVIDIA Vera Rubin NVL144 CPX rack integrates 144 Rubin CPX GPUs, 144 Rubin GPUs, and 36 Vera CPUs to deliver 8 exaFLOPs of NVFP4 compute-7.5 more than the GB300 NVL72-alongside 100 TB of high-speed memory and 1.7 PB/s of memory bandwidth, all within a single rack.
Using NVIDIA Quantum-X800 InfiniBand or Spectrum-X Ethernet, paired with NVIDIA ConnectX-9 SuperNICs and orchestrated by the Dynamo platform, the Vera Rubin NVL144 CPX is built to power the next wave of million-token context AI inference workloads-cutting inference costs and unlocking advanced capabilities for developers and creators worldwide.
At scale, the platform can deliver 30x to 50x return on investment, transl
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
15/06/2026
Detach from Direct-Attached: How Remote Editing with EVO Keeps Creative Teams Mo...
14/06/2026
Library captures 1960s R&B/pop drum sound
Following on from their recent wave of plug-in effects, Iconic Instruments have just launched an all-new virtual d...
14/06/2026
HBO Comedy Rooster Shot with URSA Cine 17K 65
Brie Clayton June 14, 2026
0 Comments
Large format brings viewers intimately close to characters.
Black...
13/06/2026
Latest expansion pack includes 252 presets
Devious Machines have recently introduced another expansion for their powerful multi-effects plug-in, Infiltrator...
13/06/2026
Create custom DAW/plug-in controllers using prompts
MetaGrid have recently introduced an all-new AI Builder function to their touchscreen-based control surf...
13/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
13/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
YES Network and The Gotham Sports App will air seven Athletes Unlimited Softball...
12/06/2026
The United Football League will host its FAST Innovation Suite at the 2026 United Bowl presented by Credit One Bank on Saturday, June 13 at 3:00 p.m. ET at Audi...
12/06/2026
PTZOptics and LayerJot will present live demonstrations at InfoComm 2026 showing how natural-language AI prompting, robotic camera control, and on-device comput...
12/06/2026
MultiDyne Video and Fiber Optic Systems will exhibit at InfoComm 2026, featuring...
12/06/2026
Ateme has announced that Eurovision Services is using Ateme's software-based frame-rate conversion technology for international live event workflows. The de...
12/06/2026
Bitmovin and Simplestream have announced a partnership with Xperi to simplify the launch of OTT streaming services on TiVo OS smart TVs and devices. The collabo...
12/06/2026
Net Insight has announced that a multinational technology company is deploying a...
12/06/2026
MLB Players Inc., the business arm of the MLB Players Association, has announced a partnership with Athletes First to develop and sell brand partnerships across...
12/06/2026
Guntermann and Drunck (G&D) and VuWall have announced the CommandKeyboard-Advanc...
12/06/2026
Comcast Smart Solutions announces a new smart technology deployment with Major L...
12/06/2026
Elevation Worship completed the initial leg of its Elevation Nights 2026 tour ...
12/06/2026
AJA Video Systems has announced KONA IP25 support for Colorfront Transkoder and ...
12/06/2026
Audinate Group Limited (ASX: AD8) will exhibit at InfoComm 2026 (Booth C7321, Ce...
12/06/2026
Pac-12 Commissioner Teresa Gould has announced the appointment of Scott Adametz as Chief Technology Officer. The Pac-12 describes the hire as the first CTO appo...
12/06/2026
Grass Valley has announced AMPP Edge Live, a production system combining Grass Valley hardware, NVIDIA Blackwell GPU acceleration, and AMPP OS in a single platf...
12/06/2026
At one time a trailblazer with the launch of the Longhorn Network, the Universit...
12/06/2026
Ratings Roundup is a rundown of recent rating news and is derived from press rel...
12/06/2026
Chyron has announced PAINT 10.4, an update to its illustrated replay and sports ...
12/06/2026
SVP, Production, Mark Gross: With the new schedule, with not having every Sunday night, it has given us an opportunity to take a step back and reimagine what o...
12/06/2026
For Televisa Technical Engineering Manager Roberto N nez Ibarra and the small team of 12 technicians and two production personnel at the IBC things are already ...
12/06/2026
Simple Steps to Better Acoustics - Taming The Small Room
Most of us mix in spare rooms and small spaces, where the acoustics fight us at every turn. At Gear...
12/06/2026
Latest addition expands vintage-inspired effects palette
Meris' Ottobit pedal range draws its inspiration from vintage gaming consoles, and the latest a...
12/06/2026
Soundbox-based chamber strings series expanded
Sonora Cinematic have just announced the launch of the second instalment in their Soundbox-based chamber stri...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
AJA Announces KONA IP25 Integration with Colorfront Software
Brie Clayton June 12, 2026
0 Comments
Collaboration enables uncompressed SMPTE ST 2110 I/O ...
12/06/2026
URSA Cine 12K LF Used to Create Visuals for STUTS' K-Arena Concert
Brie Clayton June 12, 2026
0 Comments
Organic visuals projected on a giant scre...
12/06/2026
MTI FILM Acquires Mango New Edit, Expanding its Global Post-Production Services ...
12/06/2026
AI Point Tracking Speeds Up Complex VFX Tracks in Mocha Pro
Jessie Electa Petrov June 12, 2026
0 Comments
The 2026.5 release adds automatic point trac...
12/06/2026
Bitmovin, a provider of video streaming solutions, has partnered with Simplestream, a provider of OTT and broadcast solutions, and technology provider Xperi, to...
12/06/2026
Leostream Corporation, creator of the world-leading Leostream Remote Desktop Access Platform, today announced Jigsaw24, a leading B2B IT solutions provider wit...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
How Aussie indie games and screen are levelling up with IP 11 June 2026
Ari Harrison, Pro Jank Footy
Head of Games Joey Egger and Ari Harrison of Umbrella sha...
12/06/2026
AgentPerf from Artificial Analysis, the industry's first agentic AI benchmark, gives developers, enterprises and infrastructure providers a clear way to com...
Inference has emerged as the new frontier of complexity in AI. Modern models are evolving into agentic systems capable of multi-step reasoning, persistent memory, and long-horizon context-enabling them to tackle complex tasks across domains such as software development, video generation, and deep research. These workloads place unprecedented demands on infrastructure, introducing new challenges in compute, memory, and networking that require a fundamental rethinking of how inference is scaled and optimized.
