As AI Increases Demands on Memory, Storage Steps Up

Surging AI demands are driving the need for massive datasets and context windows that burst past the confines of system memory.

But rising needs aren't met by simply adding more storage capacity. What's needed is useful, grounded insights from AI factories and efficient, secure storage architectures that enable those insights.

At this week's Future of Memory and Storage (FMS) conference, NVIDIA is unveiling new storage advancements and showcasing how the next leap in AI depends as much on the storage infrastructure feeding accelerated computing as on the computing power itself.

The pressure on that infrastructure is intensifying as AI agents consume massive amounts of data - and GPUs can now initiate storage requests directly, generating thousands of concurrent operations.

To serve those requests, storage systems must continuously encrypt, compress, verify and reconstruct data. These critical data services can become bottlenecks when thousands of agents access storage simultaneously.

Benchmarks highlighted in this NVIDIA technical blog show that the NVIDIA Vera CPU, part of NVIDIA Vera BlueField-4 STX, delivers up to 3.21x higher throughput than an x86 CPU in a two-stage compression and encryption pipeline. This means that with Vera, storage platforms can absorb the flood of AI data more efficiently - delivering greater throughput with significantly less compute infrastructure.

With accelerated computing, storage stops being a passive place to keep data and becomes an active part of the data path.

This upends the old economics of determining when data belongs in memory (where applications can fetch it faster) versus on a storage drive (where it can be held in cheap and plentiful space). The tradeoff was first framed 40 years ago, when the answer was measured in accessing that data in minutes. On today's GPUs, paired with AI storage solutions from NVIDIA and partners, the same tradeoff now plays out in microseconds.

Closing the gap between AI's needs and memory shortage depends on extreme codesign across the whole ecosystem, from memory and storage manufacturers to the software built on them.

Open Source NVIDIA cuFile APIs Enable Interoperability for Storage Solutions At FMS, NVIDIA announced it is open sourcing its cuFile application programming interfaces (APIs) - and the vertical storage software stack underneath them - which let GPUs, not just CPUs, read from and write to storage directly. cuFile is an open source component of NVIDIA GPUDirect Storage.

Using hundreds of thousands of GPU threads, fast high-bandwidth memory and other methodologies, cuFile enables securely accessing data from storage in just microseconds.

This represents how the industry is unifying a security-first storage stack based on Linux best practices, providing interoperability between GPUs and data.

In addition, fast, secure access to data and storage is a foundational element to powering preventive and detective cybersecurity measures. Making cuFile openly available will help make security context, data and storage accessible at the speed AI-powered defenses need. Such open technologies support initiatives such as the new Open Secure AI Alliance.

This site is the new home for APIs that are open to contributions - with Google, Intel, NVIDIA and Meta as inaugural maintainers - and can be optimized for use across various software and hardware platforms, driving innovation and efficiency for developers and enterprises.

NVIDIA and Industry Leaders Advance New Frontier of AI Storage In addition, NVIDIA and storage industry leaders are optimizing memory and storage solutions through an initiative called Storage-Next. The NVIDIA-driven initiative brings together storage makers, controller vendors, thermal design, cooling and orchestration operators, and standards bodies to align on how GPU-driven storage should behave - then turn these advancements into interoperable, open industry standards.

Storage-Next includes over 40 leading storage and flash vendors - including DDN, KIOXIA and Micron - each contributing to the next generation of AI storage technologies with NVIDIA.

The initiative is grounded in accelerated data access for large AI datasets. To support this, NVIDIA offers SCADA - short for scaled, accelerated data access - a framework that lets massively parallel GPUs pull only the data necessary for the application directly from storage into their own high-speed memory.

For example, DDN is integrating SCADA with Infinia, its software-defined, AI-native data intelligence platform built to eliminate storage bottlenecks at scale.

AI success will be defined not by how much infrastructure organizations own, but by how productively they use it, said Sven Oehme, chief technology officer at DDN. Our collaboration with NVIDIA is helping create a more direct, efficient connection between GPUs and data - keeping accelerated computing resources productive, speeding time to insight and enabling customers to achieve stronger business and financial returns from their AI investments.

Storage-Next and SCADA extend NVIDIA's longstanding work on AI storage infrastructure, including on NVIDIA Vera BlueField-4 STX - a modular, rack-scale foundation powered by the NVIDIA Vera Rubin platform, NVIDIA Vera BlueField-4 storage processors and NVIDIA Spectrum-X Ethernet networking.

NVIDIA Vera BlueField-4 STX storage processor. Defining a new class of AI-native data platforms, NVIDIA STX uses the unified NVIDIA DOCA security stack to let enterprises enable continuous policy enforcement in the AI data path.

Plus, NVIDIA CMX Context Memory Storage provides an AI native context tier for long context, multi turn, agentic AI inference, built on NVIDIA STX.

SCADA Enables Fast AI Storage That Stays Secure Speed at the storage layer comes with a catch. Letting an application talk straight t

LINK:	https://blogs.nvidia.com/blog/ai-storage-fms/...
	See more stories from nvidia

More from Nvidia