From Megawatts to Tokens: How NVIDIA Maximizes AI Factory Production

On a sweltering August evening in Silicon Valley, as the sun dropped and air conditioning loads spiked, Silicon Valley Power sent a signal to an AI factory to adjust its power consumption.

Varun Sivaram was watching on Zoom with about forty others - his team at Emerald AI in their San Francisco conference room, engineers at the data center and people from the utility itself. Nobody touched anything.

Emerald AI's Conductor platform - a grid-orchestration platform from NVIDIA partner Emerald AI, and an early example of the kind of flexibility NVIDIA DSX Flex is built to deliver - receives signals about grid conditions and adjusts the data center's flexible computing workloads. Work that can wait is slowed or rescheduled, while higher-priority services continue operating.

The goal is to reduce electricity demand when the grid is constrained without interrupting critical AI workloads- exactly what Silicon Valley Power needed,

When the reduction showed on screen, everyone cheered.

We were watching with bated breath, Sivaram said. It was our first time deploying across thousands of NVIDIA GPUs. His head of product, Mansi Shah, was emotional. This feels kind of like a SpaceX rocket launch, she said.

Silicon Valley Power has since sent more than 200 demand signals to that AI factory. It worked every single time.

The Emerald AI team in San Francisco watches as Silicon Valley Power's demand signal hits the factory floor - power dropping from four megawatts to three, automatically, while every high-priority job keeps running. This is grid flexibility in production. And it points at something much bigger than one facility in Santa Clara: a path to unlocking the power America's AI factories need, without waiting a decade to build new transmission lines.

At the AI Infra Summit on Tuesday, Ian Buck, NVIDIA's vice president of hyperscale and high-performance computing, made AI factory efficiency the centerpiece of his infrastructure keynote.

Results from cloud provider Lambda's first validation in a deployment environment, released the same day, put numbers to it: a fixed power budget can support 24% more token throughput when managed intelligently.

With our proof of concept, we believe we've moved beyond the limitation of fixed power budgets, said Dave Ward, president of cloud services at Lambda. NVIDIA DSX MaxLPS paves the way to reclaiming stranded capacity and converting it into real-world usage, with significantly more compute density in the same footprint.

That August evening, when SVP called, Conductor executed against a predefined workload hierarchy: lowest-priority jobs yielded, high-priority inference kept running, and power fell from four megawatts to three. Automated. No operator required.

DSX at a glance

NVIDIA DSX MaxLPS

A suite of technologies to optimize AI factory throughput per megawatt, including dynamic power allocation software that monitors GPU and rack-level consumption in real time, recovering stranded capacity to maximize token throughput within a fixed power budget.

NVIDIA DSX Flex

Receives grid signals (load-shedding, demand-response, pricing events) and adapts AI workload priorities in response, protecting high-priority jobs while reducing overall power draw.

NVIDIA DSX OS

Open-source, modular software for AI factory lifecycle management, runtime consistency, health automation and resiliency.

NVIDIA DSX Sim

Simulation tools that let operators model and validate factory designs before physical deployment, identifying bottlenecks before capital is fixed.

NVIDIA DSX Reference Designs

Generation-specific, validated architectures spanning compute, networking, storage and facilities, co-designed with NVIDIA's ecosystem partners.

In the AI factory economy, power is the constraint. Work per gigawatt is the metric. Data center operators are meticulous about efficiency - every watt put to work is a watt delivering productive compute, and the industry has driven remarkable gains at every layer of the stack, from facility design to rack-level power conversion.

DSX extends that discipline into the AI workload itself. Smarter rack provisioning puts power where workloads actually need it. Operational intelligence - tighter scheduling, faster restarts, leaner checkpointing - keeps GPUs running rather than waiting. The goal is the same one operators have always pursued: more work from the power you have.

A one-gigawatt factory will never become a two-gigawatt factory, NVIDIA founder and CEO Jensen Huang has said.

The answer engineers reach when systems hit physical limits is always the same: stop optimizing the parts and start designing the whole.

Introduced at GTC Taipei in May, NVIDIA DSX is that answer for the AI factory - and the early deployments are already proving it out.

The full platform spans networking, cooling, water efficiency and facility design; the sections below focus on some of the results so far in power management and grid participation.

More Compute, Same Budget: DSX MaxLPS Lambda's results, released at the AI Infra Summit, are the first validation of DSX MaxLPS on NVIDIA HGX B200 GPU Servers.

DSX MaxLPS monitors GPU and rack-level power consumption and reallocates headroom across nodes based on workload type, recovering capacity that static provisioning would leave stranded. Training and inference draw power differently; MaxLPS optimizes allocation in AI factories running both.

Lambda, a GPU cloud provider serving more than 10,000 customers from AI-native startups to hyperscalers, ran the software on a five-rack, 19-node cluster.

What they found: by running 19 nodes within the same power budget as 16 nodes at full power, Lambda achieved 24% more cluster-wide token throughput - from roughly 4 million tokens per second to 5 million. Performance per watt improved by 23%.

Based on NVIDI

LINK:	https://blogs.nvidia.com/blog/from-megawatts-to-tokens-how-nvidia-maxi...
	See more stories from nvidia

Most recent headlines