The Three Computer Solution: Powering the Next Wave of AI Robotics

ChatGPT marked the big bang moment of generative AI. Answers can be generated in response to nearly any query, helping transform digital work such as content creation, customer service, software development and business operations for knowledge workers.

Physical AI, the embodiment of artificial intelligence in humanoids, factories and other devices within industrial systems, has yet to experience its breakthrough moment.

This has held back industries such as transportation and mobility, manufacturing, logistics and robotics. But that's about to change thanks to three computers bringing together advanced training, simulation and inference.

The Rise of Multimodal, Physical AI For 60 years, Software 1.0 - serial code written by human programmers - ran on general-purpose computers powered by CPUs.

Then, in 2012, Alex Krizhevsky, mentored by Ilya Sutskever and Geoffrey Hinton, won the ImageNet computer image recognition competition with AlexNet, a revolutionary deep learning model for image classification.

This marked the industry's first contact with AI. The breakthrough of machine learning - neural networks running on GPUs - jump-started the era of Software 2.0.

Today, software writes software. The world's computing workloads are shifting from general-purpose computing on CPUs to accelerated computing on GPUs, leaving Moore's law far behind.

With generative AI, multimodal transformer and diffusion models have been trained to generate responses.

Large language models are one-dimensional, able to predict the next token, in modes like letters or words. Image- and video-generation models are two-dimensional, able to predict the next pixel.

None of these models can understand or interpret the three-dimensional world. And that's where physical AI comes in.

Physical AI models can perceive, understand, interact with and navigate the physical world with generative AI. With accelerated computing, multimodal physical AI breakthroughs and large-scale physically based simulations are allowing the world to realize the value of physical AI through robots.

A robot is a system that can perceive, reason, plan, act and learn. Robots are often thought of as autonomous mobile robots (AMRs), manipulator arms or humanoids. But there are many more types of robotic embodiments.

In the near future, everything that moves, or that monitors things that move, will be autonomous robotic systems. These systems will be capable of sensing and responding to their environments.

Everything from surgical rooms to data centers, warehouses to factories, even traffic control systems or entire smart cities will transform from static, manually operated systems to autonomous, interactive systems embodied by physical AI.

The Next Frontier: Humanoids Robots Humanoid robots are an ideal general-purpose robotic manifestation because they can operate efficiently in environments built for humans, while requiring minimal adjustments for deployment and operation.

The global market for humanoid robots is expected to reach $38 billion by 2035, a more than sixfold increase from the roughly $6 billion for the period forecast nearly two years ago, according to Goldman Sachs.

Researchers and developers around the world are racing to build this next wave of robots.

Three Computers to Develop Physical AI To develop humanoid robots, three accelerated computer systems are required to handle physical AI and robot training, simulation and runtime. Two computing advancements are accelerating humanoid robot development: multimodal foundation models and scalable, physically based simulations of robots and their worlds.

Breakthroughs in generative AI are bringing 3D perception, control, skill planning and intelligence to robots. Robot simulation at scale lets developers refine, test and optimize robot skills in a virtual world that mimics the laws of physics - helping reduce real-world data acquisition costs and ensuring they can perform in safe, controlled settings.

NVIDIA has built three computers and accelerated development platforms to enable developers to create physical AI.

First, models are trained on a supercomputer. Developers can use NVIDIA NeMo on the NVIDIA DGX platform to train and fine-tune powerful foundation and generative AI models. They can also tap into NVIDIA Project GR00T, an initiative to develop general-purpose foundation models for humanoid robots to enable them to understand natural language and emulate movements by observing human actions.

Second, NVIDIA Omniverse, running on NVIDIA OVX servers, provides the development platform and simulation environment for testing and optimizing physical AI with application programming interfaces and frameworks like NVIDIA Isaac Sim.

Developers can use Isaac Sim to simulate and validate robot models, or generate massive amounts of physically-based synthetic data to bootstrap robot model training. Researchers and developers can also use NVIDIA Isaac Lab, an open-source robot learning framework that powers robot reinforcement learning and imitation learning, to help accelerate robot policy training and refinement.

Lastly, trained AI models are deployed to a runtime computer. NVIDIA Jetson Thor robotics computers are specifically designed for compact, on-board computing needs. An ensemble of models consisting of control policy, vision and language models composes the robot brain and is deployed on a power-efficient, on-board edge computing system.

Depending on their workflows and challenge areas, robot makers and foundation model developers can use as many of the accelerated computing platforms and systems as needed.

Building the Next Wave of Autonomous Facilities Robotic facilities result from a culmination of all of these technologies.

Manufacturers like Foxconn or logistics companies like Amazon Robotics can orchestrate teams of autonomous robots to

LINK:	https://blogs.nvidia.com/blog/three-computers-robotics/...
	See more stories from nvidia

More from Nvidia