It can take a puppy weeks to learn that certain kinds of behaviors will result in a yummy treat, extra cuddles or a belly rub - and that other behaviors won't. With a system of positive reinforcement, a pet pooch will in time anticipate that chasing squirrels is less likely to be rewarded than staying by their human's side.
Deep reinforcement learning, a technique used to train AI models for robotics and complex strategy problems, works off the same principle.
In reinforcement learning, a software agent interacts with a real or virtual environment, relying on feedback from rewards to learn the best way to achieve its goal. Like the brain of a puppy in training, a reinforcement learning model uses information it's observed about the environment and its rewards, and determines which action the agent should take next.
To date, most researchers have relied on a combination of CPUs and GPUs to run reinforcement learning models. This means different parts of the computer tackle different steps of the process - including simulating the environment, calculating rewards, choosing what action to take next, actually taking action, and then learning from the experience.
But switching back and forth between CPU cores and powerful GPUs is by nature inefficient, requiring data to be transferred from one part of the system's memory to another at multiple points during the reinforcement learning training process. It's like a student who has to carry a tall stack of books and notes from classroom to classroom, plus the library, before grasping a new concept.
With Isaac Gym, NVIDIA developers have made it possible to instead run the entire reinforcement learning pipeline on GPUs - enabling significant speedups and reducing the hardware resources needed to develop these models.
Here's what this breakthrough means for the deep reinforcement learning process, and how much acceleration it can bring developers.
Reinforcement Learning on GPUs: Simulation to Action When training a reinforcement learning model for a robotics task - like a humanoid robot that walks up and down stairs - it's much faster, safer and easier to use a simulated environment than the physical world. In a simulation, developers can create a sea of virtual robots that can quickly rack up thousands of hours of experience at a task.
If tested solely in the real world, a robot in training could fall down, bump into or mishandle objects - causing potential damage to its own machinery, the object it's interacting with or its surroundings. Testing in simulation provides the reinforcement learning model a space to practice and work out the kinks, giving it a head start when shifting to the real world.
In a typical system today, the NVIDIA PhysX simulation engine runs this experience-gathering phase of the reinforcement learning process on NVIDIA GPUs. But for other steps of the training application, developers have traditionally still used CPUs.
Traditional deep reinforcement learning uses a combination of CPU and GPU computing resources, requiring significant data transfers back and forth. A key part of reinforcement learning training is conducting what's known as the forward pass: First, the system simulates the environment, records a set of observations about the state of the world and calculates a reward for how well the agent did.
The recorded observations become the input to a deep learning policy network, which chooses an action for the agent to take. Both the observations and the rewards are stored for use later in the training cycle.
Finally, the action is sent back to the simulator so that the rest of the environment can be updated in response.
After several rounds of these forward passes, the reinforcement learning model takes a look back, evaluating whether the actions it chose were effective or not. This information is used to update the policy network, and the cycle begins again with the improved model.
GPU Acceleration with Isaac Gym To eliminate the overhead of transferring data back and forth from CPU to GPU during this reinforcement learning training cycle, NVIDIA researchers have developed an approach to run every step of the process on GPUs. This is Isaac Gym, an end-to-end training environment, which includes the PhysX simulation engine and a PyTorch tensor-based API.
Isaac Gym makes it possible for a developer to run tens of thousands of environments simultaneously on a single GPU. That means experiments that previously required a data center with thousands of CPU cores can in some cases be trained on a single workstation.
NVIDIA Isaac Gym runs entire reinforcement learning pipelines on GPUs, enabling significant speedups. Decreasing the amount of hardware required makes reinforcement learning more accessible to individual researchers who don't have access to large data center resources. It can also make the process a lot faster.
A simple reinforcement learning model tasked with getting a humanoid robot to walk can be trained in just a few minutes with Isaac Gym. But the impact of end-to-end GPU acceleration is most useful for more challenging tasks, like teaching a complex robot hand to manipulate a cube into a specific position.
This problem requires significant dexterity by the robot, and a simulation environment that involves domain randomization, a mechanism that allows the learned policy to more easily transfer to a real-world robot.
Research by OpenAI tackled this task with a cluster of more than 6,000 CPU cores plus multiple NVIDIA Tensor Core GPUs - and required about 30 hours of training for the reinforcement learning model to succeed at the task 20 times in a row using a feed-forward network model.
Using just one NVIDIA A100 GPU with Isaac Gym, NVIDIA developers were able to achieve the same level of success in around 10 hours - a single GPU outperforming
Most recent headlines
04/09/2025
Monumental Sports & Entertainment (MSE), in collaboration with Dalet, has been a...
09/05/2025
CAMBRIDGE, Mass. Studio Technologies said the Harvard University athletics department has integrated Dante-enabled equipment from the vendor into its broadcast ...
09/05/2025
NEW YORK ITN, a provider of a local linear supply side platform, and Magnite, a independent sell-side advertising company, have announced that they working toge...
09/05/2025
LEEDS, U.K. Nugen Audio has launched a new speech intelligibility plug-in, DialogCheck and offered up quotes from technologists working at places like Netflix p...
09/05/2025
AJA I/O Gear: The Heart of Broadcast Solutions' VEGO Mobile Editing Solution
Brie Clayton May 8, 2025
0 Comments
When working remotely on broadcas...
09/05/2025
What do they teach in an Advanced Adobe After Effects Course?
Roland Kahlenberg May 8, 2025
0 Comments
There aren't many advanced After Effects co...
09/05/2025
Larry Jordan Sits with Trevor Morgan of OpenDrives at NAB 2025
Brie Clayton May 8, 2025
0 Comments
Trevor Morgan, COO of OpenDrives, shares how the co...
09/05/2025
Berklee Popular Music Institute Announces UK Festival Debut and Tour Dates For the first time, BPMI will bring Berklee-affiliated artists and students to the ...
08/05/2025
A sinister fairy infiltrates a desperate family in Kenneth Dagatan's In My Mother's Skin, which premiered at the 2023 Sundance Film Festival. Photo co...
08/05/2025
As expected, continued weak demand from key sales markets and declining economic...
08/05/2025
A new three-part series is coming to BBC iPlayer and BBC One
(Image: The Christie Archive Trust)
The BBC has announced Agatha Christie's Endless Night, a...
08/05/2025
For skyward-bound operators, training focuses on the unique aspects of flying ISR missions, including the management of onboard surveillance equipment and the e...
08/05/2025
The cable industry has told the Federal Communications Commission it supports the National Association of Broadcasters' proposal to allow broadcasters to us...
08/05/2025
WASHINGTON The Consumer Technology Association has continued its opposition to mandates requiring that NextGen TV/ATSC 3.0 tuners be included in new TV sets, sa...
08/05/2025
TAG Video Systems, the leader in software-based IP end-to-end workflow monitoring, deep probing, and real time visualization, has named Paul Maroni as Vice Pres...
08/05/2025
This year's UK Pavilion in hall 5, once again managed by Tradefair, will provide visitors with the unique opportunity to discuss and be involved in cutting ...
08/05/2025
Rohde & Schwarz will showcase its latest energy-efficient transmitters and 5G Broadcast technologies, designed to support network operators and content provider...
08/05/2025
IRVING, Texas Nexstar Media Group has tapped Bill Nardi as vice president of station operations, responsible for overseeing the day-to-day broadcast operations ...
08/05/2025
SEATTLE LumaTouch is partnering with CNN Academy to improve mobile storytelling techniques and support training across all of CNN Academy's training simulat...
08/05/2025
WASHINGTON The Society of Broadcast Engineers has filed comments with the Federal Communications Commission that support a proposal by the National Association ...
08/05/2025
Senior adviser to the United States Agency for Global Media Kari Lake has announced that One America News Network (OAN) will provide newsfeed services for fre...
08/05/2025
EdMon Expands as AI-Driven Post Production Workflows Gains Traction in Sweden an...
08/05/2025
Using Luma Mattes in Adobe Premiere Pro
Graham Quince May 7, 2025
0 Comments
This very quick tutorial shows you how to take an RGB clip and apply its ...
08/05/2025
OpenDrives Unveils Free Your Data' Initiative with New Astraeus Cloud-Nativ...
08/05/2025
Student Spotlight: Grigori Balasanyan The Armenian composer, who was named Boston Conservatory at Berklees 2025 student commencement speaker, talks about his ...
08/05/2025
08 May 2025
VEON Shareholders Re-elect Board at 2025 AGM, Founder Augie Fabela ...
08/05/2025
Comedy and entertainment channel U&Dave bring back their #1 ranked programme of ...
08/05/2025
May 8th, 2025 Press Materials Available Here
Tribeca Festival 2025 Unveils New Premieres Spanning Film and Music
Slick Rick's Victory with Idris Elba a...
08/05/2025
May 8th, 2025 Press Materials Available Here
Tribeca Festival 2025 Announces Lineup for Inaugural Storytelling Summit
11-Day Industry Event Launches with Tal...
08/05/2025
SVG Sit-Down: Vizrt's Nicholas Jameson on AI in Workflows, Pushing Boundarie...
08/05/2025
Creating Alternative Brand Experiences: Live Sports in the Age of Fortnite, Meta...
08/05/2025
PGA TOUR's David Piccolo: Advanced Graphics and Virtual Production Tools are...
08/05/2025
Tech Focus: Advancing Immersion in Sports Broadcasting with AR and Virtual Produ...
08/05/2025
Expect even more chaos this July with Harriet Webb leading the returning cast, p...
08/05/2025
Back to All News
Now in Production: Comedy Action Film Husbands in Action'...
08/05/2025
TenneT relies on Arvato Systems for market communication
Energy industry: Impressive market communication know-how and system integration expertise
G tersloh...
08/05/2025
When Taiki Hamamoto, 22, came across a Hanafuda deck at his local game shop, he was intrigued. He had grown up playing the traditional Japanese card game with f...
08/05/2025
The Liveline is now open , said Joe Duffy earlier today, as he previewed this af...
08/05/2025
RT , in association with the BBC, Screen Ireland and Cineflix Rights has reveale...
08/05/2025
Artificial intelligence is helping identify and treat diseases faster with better results for humankind. Natural disasters like wildfires are next.
Fires in th...
08/05/2025
Calling all wiseguys - 2K's acclaimed Mafia franchise is available to stream...
08/05/2025
As AI use cases continue to expand - from document summarization to custom software agents - developers and enthusiasts are seeking faster, more flexible ways t...
07/05/2025
Discovering music should feel effortless and fun. That's why Spotify continu...
07/05/2025
SBS and NITV mark National Reconciliation Week with compelling premieres recogni...
07/05/2025
SBS commences search for a new Western Sydney production hub location
7 May, 2025
Media releases
SBS has today launched a Request for Expressions of Intere...
07/05/2025
Warsaw, Poland - April 28, 2025 - Nielsen, a global leader in audience measurement, data and analytics, has released its latest March All Screens Video Landscap...
07/05/2025
LONDON Movie fans hoping to save money by waiting until their favorite new films appear on streaming services will have to wait a bit longer now, according to a...
07/05/2025
MECCA, Saudi Arabia Saudi Broadcasting Authority (SBA) has selected Grass Valley to provide a major technology upgrade of its broadcast facility here....
07/05/2025
Sony and Nevion provide guidance on IP network architecture options for live pro...
It can take a puppy weeks to learn that certain kinds of behaviors will result in a yummy treat, extra cuddles or a belly rub - and that other behaviors won't. With a system of positive reinforcement, a pet pooch will in time anticipate that chasing squirrels is less likely to be rewarded than staying by their human's side.
