Sony Pixel Power calrec Sony

World-Class: NVIDIA Research Builds AI Model to Populate Virtual Worlds With 3D Objects, Characters

23/09/2022

The massive virtual worlds created by growing numbers of companies and creators could be more easily populated with a diverse array of 3D buildings, vehicles, characters and more - thanks to a new AI model from NVIDIA Research.

Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. These 3D objects are created in the same format used by popular graphics software applications, allowing users to immediately import their shapes into 3D renderers and game engines for further editing.

The generated objects could be used in 3D representations of buildings, outdoor spaces or entire cities, designed for industries including gaming, robotics, architecture and social media.

GET3D can generate a virtually unlimited number of 3D shapes based on the data it's trained on. Like an artist who turns a lump of clay into a detailed sculpture, the model transforms numbers into complex 3D shapes.

With a training dataset of 2D car images, for example, it creates a collection of sedans, trucks, race cars and vans. When trained on animal images, it comes up with creatures such as foxes, rhinos, horses and bears. Given chairs, the model generates assorted swivel chairs, dining chairs and cozy recliners.

GET3D brings us a step closer to democratizing AI-powered 3D content creation, said Sanja Fidler, vice president of AI research at NVIDIA, who leads the Toronto-based AI lab that created the tool. Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and interesting objects.

GET3D is one of more than 20 NVIDIA-authored papers and workshops accepted to the NeurIPS AI conference, taking place in New Orleans and virtually, Nov. 26-Dec. 4.

It Takes AI Kinds to Make a Virtual World The real world is full of variety: streets are lined with unique buildings, with different vehicles whizzing by and diverse crowds passing through. Manually modeling a 3D virtual world that reflects this is incredibly time consuming, making it difficult to fill out a detailed digital environment.

Though quicker than manual methods, prior 3D generative AI models were limited in the level of detail they could produce. Even recent inverse rendering methods can only generate 3D objects based on 2D images taken from various angles, requiring developers to build one 3D shape at a time.

GET3D can instead churn out some 20 shapes a second when running inference on a single NVIDIA GPU - working like a generative adversarial network for 2D images, while generating 3D objects. The larger, more diverse the training dataset it's learned from, the more varied and detailed the output.

NVIDIA researchers trained GET3D on synthetic data consisting of 2D images of 3D shapes captured from different camera angles. It took the team just two days to train the model on around 1 million images using NVIDIA A100 Tensor Core GPUs.

Enabling Creators to Modify Shape, Texture, Material GET3D gets its name from its ability to Generate Explicit Textured 3D meshes - meaning that the shapes it creates are in the form of a triangle mesh, like a papier-m ch model, covered with a textured material. This lets users easily import the objects into game engines, 3D modelers and film renderers - and edit them.

Once creators export GET3D-generated shapes to a graphics application, they can apply realistic lighting effects as the object moves or rotates in a scene. By incorporating another AI tool from NVIDIA Research, StyleGAN-NADA, developers can use text prompts to add a specific style to an image, such as modifying a rendered car to become a burned car or a taxi, or turning a regular house into a haunted one.

The researchers note that a future version of GET3D could use camera pose estimation techniques to allow developers to train the model on real-world data instead of synthetic datasets. It could also be improved to support universal generation - meaning developers could train GET3D on all kinds of 3D shapes at once, rather than needing to train it on one object category at a time.

For the latest news from NVIDIA AI research, watch the replay of NVIDIA founder and CEO Jensen Huang's keynote address at GTC:
LINK: https://blogs.nvidia.com/blog/2022/09/23/3d-generative-ai-research-vir...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

20/03/2026

NAB 2026: Net Insight unveils Market-Leading JPEG XS at Scale for Live IP Media Production

Net Insight will introduce a JPEG XS solution for full IP environments at NAB Sh...

20/03/2026

NAB 2026: LTN and Harmonic Expand Partnership to Support FAST Growth and C-Band Migration

LTN has expanded its technology partnership with Harmonic ahead of the FCC's...

20/03/2026

NAB 2026: Solid State Logic to Preview SSL Live V6.2 with New SolidPitch Effect and Major Workflow Enhancements

Solid State Logic will preview SSL Live V6.2 at NAB Show, booth C6907. The softw...

20/03/2026

NAB 2026: Fujifilm Announces Availability of FUJINON UA22x4.8BERD 4K Broadcast Zoom Lens

FUJIFILM North America Corporation's Optical Devices Division has announced ...

20/03/2026

NAB 2026: Fujifilm Announces Development of FUJINON UA16x4BERD, UA30x7.3BERD, and UA94x8.7BESM 4K Broadcast Zooms

FUJIFILM North America Corporation's Optical Devices Division has announced ...

20/03/2026

TrueVisions NOW Chooses Bitmovin's Observability

TrueVisions NOW, a streaming platform in Thailand and part of the TrueVisions Group, has selected Bitmovin's Observability product for real-time video analy...

20/03/2026

Marquee Sports Network Expands Distribution to Hulu + Live TV, Prime Video

Marquee Sports Network has announced distribution agreements with Hulu + Live TV and Prime Video ahead of the 2026 MLB season. Marquee Sports Network is now av...

20/03/2026

NAB 2026: Software-Defined, AI-Powered Workflow Tells the Story of the New FOR-A America

FOR-A will exhibit software-defined and AI-driven solutions at NAB Show 2026, bo...

20/03/2026

TNA Wrestling and Eurosport India Announce New Multi-Year Exclusive Programming Agreement

TNA Wrestling and Eurosport India have entered into a multi-year exclusive progr...

20/03/2026

GameTime Productions Expands Technical Vision for Athletes Unlimited Basketball in Nashville

When Athletes Unlimited brought its professional women's basketball season t...

20/03/2026

Calrec Craft Interview with Senior Broadcast Audio A1 Engineer and Music Director Rick Bernier

In this craft interview, Rick Bernier reflects on a career that has taken him to...

20/03/2026

NAB 2026: IP Innovation, SoftwareBased Media Infrastructure & Dynamic Media Facility Workflows on Display for Lawo

Lawo will announce a new product ahead of NAB Show 2026 in Las Vegas, where it w...

20/03/2026

Ratings Roundup: 2026 World Baseball Classic is Most Watched WBC Telecast EVER with Over 10 Million Viewers

Ratings Roundup is a rundown of recent rating news and is derived from press rel...

20/03/2026

MLB Names Polymarket Exclusive Prediction Market Exchange Partner and Signs Agreement with CFTC to Establish Integrity Framework

Major League Baseball (MLB) has named Polymarket as its Official Prediction Mark...

20/03/2026

How Big Tech AI Will Lead Broadcast Innovation: Lessons from the Enterprise Market

With AI now the industry-wide priority, Big Tech companies are uniquely position...

20/03/2026

SVG GameDay, Ep. 8: Los Angeles Angels' Davin Maske - SoCal Baseball in Anaheim

In-venue and creative video staffers at the professional and collegiate level ha...

20/03/2026

Fanatics Studios, FOX Sports Focus on Cavalcade of Stars in Fanatics Flag Football Classic

Abundant player mics and RF and other ground-level cameras will be used to captu...

20/03/2026

ESPN Ramps Up Production Levels for NCAA Women's Tournament as Popularity, Viewership Skyrocket

Regional site will also receive big boost in production resources, including on-...

20/03/2026

Give Me the Backstory: Get to Know Addison Heimann, the Writer-Director of Touch Me

By Jessica Herndon One of the most exciting things about the Sundance Film Fest...

20/03/2026

Spotify Marks 5 Years of EQUAL With EQUAL: The Podcast' and Global Events

In 2021, we launched EQUAL, a program designed to address an industry reality that persists: Women artists, songwriters, and producers too often face fewer oppo...

20/03/2026

Spotify's BTS Music Quiz Celebrates ARIRANG' and Puts ARMY Knowledge to the Test

BTS' long-awaited fifth studio album, ARIRANG, is finally here. To celebrate...

20/03/2026

Spotify and Kenia Os Bring K de Karma' From Streaming to Stage With Fan-First Experiences

A new era for Kenia Os has arrived, and Spotify marked the moment by putting fan...

20/03/2026

Spotify y Kenia Os llevan K de Karma' del streaming al escenario con experiencias nicas para Top Fans

Una nueva era para Kenia Os ha llegado, y Spotify marc el momento poniendo a lo...

20/03/2026

Sound Magic launch Supreme Drums Orange

Combines sampling & physical modelling Sound Magic have announced the launch of a comprehensive virtual drum instrument that's been designed to cater to...

20/03/2026

Mix Rescue: Ian Shepherd Video

How much difference should mastering make? In our latest Mix Rescue feature, SOS Editor in Chief Sam Inglis revisits a project from back in 2019, carrying o...

20/03/2026

Feast for cycling fans as SBS extends road cycling broadcasts to include all Grand Tours and Monuments

Feast for cycling fans as SBS extends road cycling broadcasts to include all Gra...

20/03/2026

London Calling - When The Industry Convened to Help Streaming Find its MoJo

In this blog, Laura Rognoni reflects on key discussions from the Connected TV World Summit in London, where NAGRAVISION hosted a panel on content discovery and ...

20/03/2026

The Gauge: Poland | February 2026

After a series of increases, February brought the first slowdown in time spent in front of TV sets in a long time. While traditional television was losing viewi...

20/03/2026

Eric Wolff Joins Chyron as Director of Venues Sales, North America

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

Study: Broadcasters Rebuild Live Operations Around IP

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

Matt Conrad Acquires DSC Labs

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

Glensound celebrates its diamond anniversary at NAB Show

60 years of audio innovation, driven by the people who use it...

20/03/2026

FCC Approves Nexstar's Acquisition of Tegna

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

Icesi names Ronald David Reyes as the 2025 recipient of t...

Colombia's Icesi University and WSDG are proud to announce Ronald David Reyes as the recipient of the 2025 WSDG Excellence Scholarship, awarded to an outsta...

20/03/2026

Celebrating the greatest creators - One Battle After Ano...

Avid today celebrated the filmmakers, editors and sound teams that worked with Avid Media Composer and Pro Tools to create the vast majority of this year'...

20/03/2026

MRMC Names CP Communications Its Official U.S. Rental, Sales Partner

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

FOR-A To Feature Software-Defined, AI-Driven Solutions At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

2026 NAB Show Exhibitor Insight: Riedel Communications

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

DirecTV Files Suit to Block Nexstar/Tegna Deal

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

Fujifilm Announces Four New Broadcast Zoom Lenses

Share Copy link Facebook X Linkedin Bluesky Email...

20/03/2026

NAB 2026: Software-Defined, AI-Powered Workflow Tells the...

Real-time 9:16 AI-Generated Autocropping; Software-Defined Station in a Box; and Software Switcher with Unlimited Layering Are Among Show Highlights For the fi...

20/03/2026

Signiant Showcases New Content Innovations Driving Visibility, Access, and Action at NAB 2026

Signiant Showcases New Content Innovations Driving Visibility, Access, and Actio...

20/03/2026

Caffeine Relies on DaVinci Resolve Studio for End to End Post Workflow

Caffeine Relies on DaVinci Resolve Studio for End to End Post Workflow Brie Clayton March 19, 2026 0 Comments Blackmagic Cloud helps Mexican post faci...

20/03/2026

Sky Extends Saturday Night Live UK Run to Eight Shows

Friday 20 March 2026 Sky Extends Saturday Night Live UK Run to Eight Shows Ahead of Saturday Night Live UK's launch tomorrow (March 21), Sky has confirmed...

20/03/2026

It's time to play The Money List! Baz Ashmawy is back at the helm as the quiz returns for a fourth season

It's time to play The Money List! Baz Ashmawy is back at the helm as the qui...