3 Ways NVIDIA Is Powering the Industrial Revolution

The NVIDIA accelerated computing platform is leading supercomputing benchmarks once dominated by CPUs, enabling AI, science, business and computing efficiency worldwide.

Moore's Law has run its course, and parallel processing is the way forward. With this evolution, NVIDIA GPU platforms are now uniquely positioned to deliver on the three scaling laws - pretraining, post-training and test-time compute - for everything from next-generation recommender systems and large language models (LLMs) to AI agents and beyond.

How NVIDIA has transformed the foundation of computing

AI pretraining, post-training and inference are driving the frontier

How hyperscalers are using AI to transform search and recommender systems

The CPU-to-GPU Transition: A Historic Shift in Computing At SC25, NVIDIA founder and CEO Jensen Huang highlighted the shifting landscape. Within the TOP100, a subset of the TOP500 list of supercomputers, over 85% of systems use GPUs. This flip represents a historic transition from the serial processing paradigm of CPUs to massively parallel accelerated architectures.

Before 2012, machine learning was based on programmed logic. Statistical models were used and ran efficiently on CPUs as a corpus of hard-coded rules. But this all changed when AlexNet running on gaming GPUs demonstrated image classification could be learned by examples. Its implications were enormous for the future of AI, with parallel processing on increasing sums of data on GPUs driving a new wave of computing.

This flip isn't just about hardware. It's about platforms unlocking new science. GPUs deliver far more operations per watt, making exascale practical without untenable energy demands.

Recent results from the Green500, a ranking of the world's most energy-efficient supercomputers, underscore the contrast between GPUs versus CPUs. The top five performers in this industry standard benchmark were all NVIDIA GPUs, delivering an average of 70.1 gigaflops per watt. Meanwhile, the top CPU-only systems provided 15.5 flops per watt on average. This 4.5x differential between GPUs versus CPUs on energy efficiency highlights the massive TCO (total cost of ownership) advantage of moving these systems to GPUs.

Another measure of the CPU-versus-GPU energy-efficiency and performance differential arrived with NVIDIA's results on the Graph500. NVIDIA delivered a record-breaking result of 410 trillion traversed edges per second, placing first on the Graph500 breadth-first search list.

The winning run more than doubled the next highest score and utilized 8,192 NVIDIA H100 GPUs to process a graph with 2.2 trillion vertices and 35 trillion edges. That compares with the next best result on the list, which required roughly 150,000 CPUs for this workload. Hardware footprint reductions of this scale save time, money and energy.

Yet NVIDIA showcased at SC25 that its AI supercomputing platform is far more than GPUs. Networking, CUDA libraries, memory, storage and orchestration are co-designed to deliver a full-stack platform.

Enabled by CUDA, NVIDIA is a full-stack platform. Open-source libraries and frameworks such as those in the CUDA-X ecosystem are where big speedups occur. Snowflake recently announced an integration of NVIDIA A10 GPUs to supercharge data science workflows. Snowflake ML now comes preinstalled with NVIDIA cuML and cuDF libraries to accelerate popular ML algorithms with these GPUs.

With this native integration, Snowflake's users can easily accelerate model development cycles with no code changes required. NVIDIA's benchmark runs show 5x less time required for Random Forest and up to 200x for HDBSCAN on NVIDIA A10 GPUs compared with CPUs.

The flip was the turning point. The scaling laws are the trajectory forward. And at every stage, GPUs are the engine driving AI into its next chapter.

But CUDA-X and many open-source software libraries and frameworks are where much of the magic happens. CUDA-X libraries accelerate workloads across every industry and application - engineering, finance, data analytics, genomics, biology, chemistry, telecommunications, robotics and much more.

The world has a massive investment in non-AI software. From data processing to science and engineering simulations, representing hundreds of billions of dollars in compute cloud computing spend each year, Huang said on NVIDIA's recent earning call.

Many applications that once ran exclusively on CPUs are now rapidly shifting to CUDA GPUs. Accelerated computing has reached a tipping point. AI has also reached a tipping point and is transforming existing applications while enabling entirely new ones, he said.

What began as an energy efficiency imperative has matured into a scientific platform: simulation and AI fused at scale. The leadership of NVIDIA GPUs in the TOP100 is both proof of this trajectory and a signal of what comes next - breakthroughs across every discipline.

As a result, researchers can now train trillion parameter models, simulate fusion reactors and accelerate drug discovery at scales CPUs alone could never reach.

The Three Scaling Laws Driving AI's Next Frontier The change from CPUs to GPUs is not just a milestone in supercomputing. It's the foundation for the three scaling laws that represent the roadmap for AI's next workflow: pretraining, post training and test time scaling.

Pre training scaling was the first law to assist the industry. Researchers discovered that as datasets, parameter counts and compute grew, model performance improved predictably. Doubling the data or parameters meant leaps in accuracy and versatility.

On the latest MLPerf Training industry benchmarks, the NVIDIA platform delivered the highest performance on every test and was the only platform to submit on all tests. Without GPUs, the bigger is better era of AI research would have stall

LINK:	https://blogs.nvidia.com/blog/gpu-cuda-scaling-laws-industrial-revolut...
	See more stories from nvidia

Most recent headlines