AI Infra Summit: NVIDIA Vera Rubin and DSX Platform Advancements Showcase Energy Efficiencies of Optimizing Tokens Per Watt for AI Factories

Ian Buck, vice president of hyperscale and high-performance computing at NVIDIA, Tuesday spoke on AI factory efficiency at the AI Infra Summit, the Santa Clara Convention Center event that has morphed into a Coachella of infrastructure tech.

Before a packed audience - with more than 8,000 attendees this year, up from 3,500 last year - Buck discussed new collaborations across NVIDIA platforms and more.

Amazon's Annapurna Labs is working with NVIDIA on the NVHBM custom high-bandwidth memory technology.

d-Matrix is integrating with NVLink Fusion to combine NVIDIA Vera CPUs with d-Matrix Raptor XPUs to deliver ultra low-latency inference at scale.

NVIDIA and partners unveiled new results as well:

Emerald AI and NVIDIA demonstrated a commercial AI factory flexible-load program, working with Silicon Valley Power.

Lambda improved performance per watt by 23% with NVIDIA DSX MaxLPS.

Pinterest is using the NVIDIA Blackwell platform and NVIDIA Dynamo inference software to bring conversational AI to visual discovery.

The news comes as agentic AI is driving a new class of workloads that demand more performance, efficiency and scale from AI infrastructure.

NVIDIA addresses that challenge with a full-stack AI factory platform spanning Vera Rubin systems, Dynamo inference software, NeMo libraries and NVIDIA networking - including NVIDIA NVLink for scale-up computing, Spectrum-X Ethernet and ConnectX SuperNICs for connecting thousands of nodes, BlueField-powered context-memory storage and BlueField DPUs for infrastructure security.

Infrastructure that's fungible, that's reliable, that's going to last 10 years and really becomes an asset for computing the world's computing problems in the world's industries - they can build that with Vera Rubin, with DSX and all the innovations that we have here, said Buck.

The metric for AI infrastructure is fast shifting from peak performance to validated agentic tokens per megawatt. AI factories must now be codesigned from silicon to grid. NVIDIA DSX MaxLPS can deliver up to 1.4x more tokens per megawatt through factory-wide power optimization, while NVLink helps unite large-scale accelerated computing into a single high-performance system.

The result is AI infrastructure designed to generate more tokens, improve efficiency and help customers get more value from every megawatt of power.

Wednesday, Sept. 16, 8:00 a.m. PT

MLPerf Inference v6.1: NVIDIA Vera Rubin NVL72 Sets New Benchmark for AI Throughput AI inference economics increasingly come down to three factors: system performance, infrastructure scaling and software optimization. Together, they determine how many tokens can be generated, how efficiently AI services scale and how much value organizations extract from their infrastructure investments.

The NVIDIA platform is designed to optimize across all three, while giving enterprises the flexibility to run any model and workload on the same infrastructure, from training and inference to recommender systems, reasoning models and generative AI.

New results from MLPerf Inference v6.1 - the MLCommons consortium's long-standing industry benchmark, spanning a broad range of models with every result peer-reviewed before publication - highlight that advantage. In its first MLPerf Inference preview submission, the NVIDIA Vera Rubin NVL72 system delivered up to 3.7x higher throughput than NVIDIA GB300 NVL72, demonstrating the performance gains possible with NVIDIA's next-generation AI infrastructure.

NVIDIA GB300 NVL72 also showcased industry-leading scalability. A 288-GPU submission spanning four GB300 NVL72 racks achieved 99% scaling efficiency, with throughput growing nearly linearly from a single-rack baseline.

The results also underscore the impact of software innovation. NVIDIA's MLPerf Inference v6.1 submissions delivered up to 1.6x higher performance than v6.0 through software optimizations alone, with additional gains achieved after the benchmark submission period.

For organizations evaluating AI infrastructure, performance, scaling efficiency and software velocity remain critical drivers of long-term inference economics.

Learn more about the NVIDIA Vera Rubin NVL72 benchmark results.

Tuesday, Sept. 15, 9:55 a.m. PT

AI Factory Flexible-Load Program Unlocks More Grid Power Silicon Valley Power operates a flexible-load interconnection program that enables AI factories to support grid flexibility.

Through this program, Emerald AI worked with NVIDIA to demonstrate automated load reduction at Silicon Valley Power. The system successfully responded to hundreds of demand signals from Silicon Valley Power while protecting AI workload performance.

NVIDIA partner Emerald AI is planning to use NVIDIA DSX Flex for its Conductor grid-responsive power management software that dynamically adjusts an AI factory's energy consumption based on real-time electricity grid signals and hybrid energy sources.

Using DSX Flex to autonomously load balance, Emerald AI can demonstrate its ability to automatically throttle back power on low-priority AI jobs and then return to normal.

DSX Flex can receive grid signals - load-shedding requests, demand-response events and pricing signals - and automatically act within a predefined workload hierarchy. The most critical jobs keep going, while everything else pauses temporarily and then resumes. In this way, facilities can participate in energy efficiency for the grid.

This capability enables AI factories to operate as flexible grid resources, so they can reduce demand when the grid needs relief, protect priority AI workloads and prove that controllable AI load can help unlock more grid capacity for growth.

Learn more about the Emerald AI partnership with Silicon Valley Power.

Tuesday, Sept. 15, 9:55 a.m. PT

Lambda Maximizes Performance per Watt With NVIDIA DSX

LINK:	https://blogs.nvidia.com/blog/ai-infra-summit-vera-rubin-dsx-energy-ef...
	See more stories from nvidia

Most recent headlines