NVIDIA Research's latest AI model is a prodigy among generative adversarial networks. Using a fraction of the study material needed by a typical GAN, it can learn skills as complex as emulating renowned painters and recreating images of cancer tissue.
By applying a breakthrough neural network training technique to the popular NVIDIA StyleGAN2 model, NVIDIA researchers reimagined artwork based on fewer than 1,500 images from the Metropolitan Museum of Art. Using NVIDIA DGX systems to accelerate training, they generated new AI art inspired by the historical portraits.
The technique - called adaptive discriminator augmentation, or ADA - reduces the number of training images by 10-20x while still getting great results. The same method could someday have a significant impact in healthcare, for example by creating cancer histology images to help train other AI models.
These results mean people can use GANs to tackle problems where vast quantities of data are too time-consuming or difficult to obtain, said David Luebke, vice president of graphics research at NVIDIA. I can't wait to see what artists, medical experts and researchers use it for.
The research paper behind this project is being presented this week at the annual Conference on Neural Information Processing Systems, known as NeurIPS. It's one of a record 28 NVIDIA Research papers accepted to the prestigious conference.
This new method is the latest in a legacy of GAN innovation by NVIDIA researchers, who've developed groundbreaking GAN-based models for the AI painting app GauGAN, the game engine mimicker GameGAN, and the pet photo transformer GANimal. All are available on the NVIDIA AI Playground.
The Training Data Dilemma Like most neural networks, GANs have long followed a basic principle: the more training data, the better the model. That's because each GAN consists of two cooperating networks - a generator, which creates synthetic images, and a discriminator, which learns what realistic images should look like based on training data.
The discriminator coaches the generator, giving pixel-by-pixel feedback to help it improve the realism of its synthetic images. But with limited training data to learn from, a discriminator won't be able to help the generator reach its full potential - like a rookie coach who's experienced far fewer games than a seasoned expert.
It typically takes 50,000 to 100,000 training images to train a high-quality GAN. But in many cases, researchers simply don't have tens or hundreds of thousands of sample images at their disposal.
With just a couple thousand images for training, many GANs would falter at producing realistic results. This problem, called overfitting, occurs when the discriminator simply memorizes the training images and fails to provide useful feedback to the generator.
In image classification tasks, researchers get around overfitting with data augmentation, a technique that expands smaller datasets using copies of existing images that are randomly distorted by processes like rotating, cropping or flipping - forcing the model to generalize better.
But previous attempts to apply augmentation to GAN training images resulted in a generator that learned to mimic those distortions, rather than creating believable synthetic images.
A GAN on a Mission NVIDIA Research's ADA method applies data augmentations adaptively, meaning the amount of data augmentation is adjusted at different points in the training process to avoid overfitting. This enables models like StyleGAN2 to achieve equally amazing results using an order of magnitude fewer training images.
As a result, researchers can apply GANs to previously impractical applications where examples are too scarce, too hard to obtain or too time-consuming to gather into a large dataset.
Different editions of StyleGAN have been used by artists to create stunning exhibits and produce a new manga based on the style of legendary illustrator Osamu Tezuka. It's even been adopted by Adobe to power Photoshop's new AI tool, Neural Filters.
With less training data required to get started, StyleGAN2 with ADA could be applied to rare art, such as the work by Paris-based AI art collective Obvious on African Kota masks.
Another promising application lies in healthcare, where medical images of rare diseases can be few and far between because most tests come back normal. Amassing a useful dataset of abnormal pathology slides would require many hours of painstaking labeling by medical experts.
Synthetic images created with a GAN using ADA could fill that gap, generating training data for another AI model that helps pathologists or radiologists spot rare conditions on pathology images or MRI studies. An added bonus: With AI-generated data, there are no patient data or privacy concerns, making it easier for healthcare institutions to share datasets.
NVIDIA Research at NeurIPS The NVIDIA Research team consists of more than 200 scientists around the globe, focusing on areas including AI, computer vision, self-driving cars, robotics and graphics. Over two dozen papers authored by NVIDIA researchers will be highlighted at NeurIPS, the year's largest AI research conference, taking place virtually from Dec. 6-12.
Check out the full lineup of NVIDIA Research papers at NeurIPS.
Main images generated by StyleGAN2 with ADA, trained on a dataset of fewer than 1,500 images from the Metropolitan Museum of Art Collection API.
Most recent headlines
04/09/2025
Monumental Sports & Entertainment (MSE), in collaboration with Dalet, has been a...
09/05/2025
CAMBRIDGE, Mass. Studio Technologies said the Harvard University athletics department has integrated Dante-enabled equipment from the vendor into its broadcast ...
09/05/2025
NEW YORK ITN, a provider of a local linear supply side platform, and Magnite, a independent sell-side advertising company, have announced that they working toge...
09/05/2025
LEEDS, U.K. Nugen Audio has launched a new speech intelligibility plug-in, DialogCheck and offered up quotes from technologists working at places like Netflix p...
09/05/2025
AJA I/O Gear: The Heart of Broadcast Solutions' VEGO Mobile Editing Solution
Brie Clayton May 8, 2025
0 Comments
When working remotely on broadcas...
09/05/2025
What do they teach in an Advanced Adobe After Effects Course?
Roland Kahlenberg May 8, 2025
0 Comments
There aren't many advanced After Effects co...
09/05/2025
Larry Jordan Sits with Trevor Morgan of OpenDrives at NAB 2025
Brie Clayton May 8, 2025
0 Comments
Trevor Morgan, COO of OpenDrives, shares how the co...
09/05/2025
Berklee Popular Music Institute Announces UK Festival Debut and Tour Dates For the first time, BPMI will bring Berklee-affiliated artists and students to the ...
08/05/2025
A sinister fairy infiltrates a desperate family in Kenneth Dagatan's In My Mother's Skin, which premiered at the 2023 Sundance Film Festival. Photo co...
08/05/2025
As expected, continued weak demand from key sales markets and declining economic...
08/05/2025
A new three-part series is coming to BBC iPlayer and BBC One
(Image: The Christie Archive Trust)
The BBC has announced Agatha Christie's Endless Night, a...
08/05/2025
For skyward-bound operators, training focuses on the unique aspects of flying ISR missions, including the management of onboard surveillance equipment and the e...
08/05/2025
The cable industry has told the Federal Communications Commission it supports the National Association of Broadcasters' proposal to allow broadcasters to us...
08/05/2025
WASHINGTON The Consumer Technology Association has continued its opposition to mandates requiring that NextGen TV/ATSC 3.0 tuners be included in new TV sets, sa...
08/05/2025
TAG Video Systems, the leader in software-based IP end-to-end workflow monitoring, deep probing, and real time visualization, has named Paul Maroni as Vice Pres...
08/05/2025
This year's UK Pavilion in hall 5, once again managed by Tradefair, will provide visitors with the unique opportunity to discuss and be involved in cutting ...
08/05/2025
Rohde & Schwarz will showcase its latest energy-efficient transmitters and 5G Broadcast technologies, designed to support network operators and content provider...
08/05/2025
IRVING, Texas Nexstar Media Group has tapped Bill Nardi as vice president of station operations, responsible for overseeing the day-to-day broadcast operations ...
08/05/2025
SEATTLE LumaTouch is partnering with CNN Academy to improve mobile storytelling techniques and support training across all of CNN Academy's training simulat...
08/05/2025
WASHINGTON The Society of Broadcast Engineers has filed comments with the Federal Communications Commission that support a proposal by the National Association ...
08/05/2025
Senior adviser to the United States Agency for Global Media Kari Lake has announced that One America News Network (OAN) will provide newsfeed services for fre...
08/05/2025
EdMon Expands as AI-Driven Post Production Workflows Gains Traction in Sweden an...
08/05/2025
Using Luma Mattes in Adobe Premiere Pro
Graham Quince May 7, 2025
0 Comments
This very quick tutorial shows you how to take an RGB clip and apply its ...
08/05/2025
OpenDrives Unveils Free Your Data' Initiative with New Astraeus Cloud-Nativ...
08/05/2025
Student Spotlight: Grigori Balasanyan The Armenian composer, who was named Boston Conservatory at Berklees 2025 student commencement speaker, talks about his ...
08/05/2025
08 May 2025
VEON Shareholders Re-elect Board at 2025 AGM, Founder Augie Fabela ...
08/05/2025
Comedy and entertainment channel U&Dave bring back their #1 ranked programme of ...
08/05/2025
May 8th, 2025 Press Materials Available Here
Tribeca Festival 2025 Unveils New Premieres Spanning Film and Music
Slick Rick's Victory with Idris Elba a...
08/05/2025
May 8th, 2025 Press Materials Available Here
Tribeca Festival 2025 Announces Lineup for Inaugural Storytelling Summit
11-Day Industry Event Launches with Tal...
08/05/2025
SVG Sit-Down: Vizrt's Nicholas Jameson on AI in Workflows, Pushing Boundarie...
08/05/2025
Creating Alternative Brand Experiences: Live Sports in the Age of Fortnite, Meta...
08/05/2025
PGA TOUR's David Piccolo: Advanced Graphics and Virtual Production Tools are...
08/05/2025
Tech Focus: Advancing Immersion in Sports Broadcasting with AR and Virtual Produ...
08/05/2025
Expect even more chaos this July with Harriet Webb leading the returning cast, p...
08/05/2025
Back to All News
Now in Production: Comedy Action Film Husbands in Action'...
08/05/2025
TenneT relies on Arvato Systems for market communication
Energy industry: Impressive market communication know-how and system integration expertise
G tersloh...
08/05/2025
When Taiki Hamamoto, 22, came across a Hanafuda deck at his local game shop, he was intrigued. He had grown up playing the traditional Japanese card game with f...
08/05/2025
The Liveline is now open , said Joe Duffy earlier today, as he previewed this af...
08/05/2025
RT , in association with the BBC, Screen Ireland and Cineflix Rights has reveale...
08/05/2025
Artificial intelligence is helping identify and treat diseases faster with better results for humankind. Natural disasters like wildfires are next.
Fires in th...
08/05/2025
Calling all wiseguys - 2K's acclaimed Mafia franchise is available to stream...
08/05/2025
As AI use cases continue to expand - from document summarization to custom software agents - developers and enthusiasts are seeking faster, more flexible ways t...
07/05/2025
Discovering music should feel effortless and fun. That's why Spotify continu...
07/05/2025
SBS and NITV mark National Reconciliation Week with compelling premieres recogni...
07/05/2025
SBS commences search for a new Western Sydney production hub location
7 May, 2025
Media releases
SBS has today launched a Request for Expressions of Intere...
07/05/2025
Warsaw, Poland - April 28, 2025 - Nielsen, a global leader in audience measurement, data and analytics, has released its latest March All Screens Video Landscap...
07/05/2025
LONDON Movie fans hoping to save money by waiting until their favorite new films appear on streaming services will have to wait a bit longer now, according to a...
07/05/2025
MECCA, Saudi Arabia Saudi Broadcasting Authority (SBA) has selected Grass Valley to provide a major technology upgrade of its broadcast facility here....
07/05/2025
Sony and Nevion provide guidance on IP network architecture options for live pro...
NVIDIA Research's latest AI model is a prodigy among generative adversarial networks. Using a fraction of the study material needed by a typical GAN, it can learn skills as complex as emulating renowned painters and recreating images of cancer tissue.
