Sony Pixel Power calrec Sony

World-Class: NVIDIA Research Builds AI Model to Populate Virtual Worlds With 3D Objects, Characters

23/09/2022

The massive virtual worlds created by growing numbers of companies and creators could be more easily populated with a diverse array of 3D buildings, vehicles, characters and more - thanks to a new AI model from NVIDIA Research.

Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. These 3D objects are created in the same format used by popular graphics software applications, allowing users to immediately import their shapes into 3D renderers and game engines for further editing.

The generated objects could be used in 3D representations of buildings, outdoor spaces or entire cities, designed for industries including gaming, robotics, architecture and social media.

GET3D can generate a virtually unlimited number of 3D shapes based on the data it's trained on. Like an artist who turns a lump of clay into a detailed sculpture, the model transforms numbers into complex 3D shapes.

With a training dataset of 2D car images, for example, it creates a collection of sedans, trucks, race cars and vans. When trained on animal images, it comes up with creatures such as foxes, rhinos, horses and bears. Given chairs, the model generates assorted swivel chairs, dining chairs and cozy recliners.

GET3D brings us a step closer to democratizing AI-powered 3D content creation, said Sanja Fidler, vice president of AI research at NVIDIA, who leads the Toronto-based AI lab that created the tool. Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and interesting objects.

GET3D is one of more than 20 NVIDIA-authored papers and workshops accepted to the NeurIPS AI conference, taking place in New Orleans and virtually, Nov. 26-Dec. 4.

It Takes AI Kinds to Make a Virtual World The real world is full of variety: streets are lined with unique buildings, with different vehicles whizzing by and diverse crowds passing through. Manually modeling a 3D virtual world that reflects this is incredibly time consuming, making it difficult to fill out a detailed digital environment.

Though quicker than manual methods, prior 3D generative AI models were limited in the level of detail they could produce. Even recent inverse rendering methods can only generate 3D objects based on 2D images taken from various angles, requiring developers to build one 3D shape at a time.

GET3D can instead churn out some 20 shapes a second when running inference on a single NVIDIA GPU - working like a generative adversarial network for 2D images, while generating 3D objects. The larger, more diverse the training dataset it's learned from, the more varied and detailed the output.

NVIDIA researchers trained GET3D on synthetic data consisting of 2D images of 3D shapes captured from different camera angles. It took the team just two days to train the model on around 1 million images using NVIDIA A100 Tensor Core GPUs.

Enabling Creators to Modify Shape, Texture, Material GET3D gets its name from its ability to Generate Explicit Textured 3D meshes - meaning that the shapes it creates are in the form of a triangle mesh, like a papier-m ch model, covered with a textured material. This lets users easily import the objects into game engines, 3D modelers and film renderers - and edit them.

Once creators export GET3D-generated shapes to a graphics application, they can apply realistic lighting effects as the object moves or rotates in a scene. By incorporating another AI tool from NVIDIA Research, StyleGAN-NADA, developers can use text prompts to add a specific style to an image, such as modifying a rendered car to become a burned car or a taxi, or turning a regular house into a haunted one.

The researchers note that a future version of GET3D could use camera pose estimation techniques to allow developers to train the model on real-world data instead of synthetic datasets. It could also be improved to support universal generation - meaning developers could train GET3D on all kinds of 3D shapes at once, rather than needing to train it on one object category at a time.

For the latest news from NVIDIA AI research, watch the replay of NVIDIA founder and CEO Jensen Huang's keynote address at GTC:
LINK: https://blogs.nvidia.com/blog/2022/09/23/3d-generative-ai-research-vir...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

20/04/2026

Live From NAB 2026: Sonys Hugo Gaggioni Highlights HDR Advances, Software-Defined Workflows

At the 2026 NAB Show, Sony is showcasing a broad slate of innovations across liv...

20/04/2026

Live From NAB 2026: Fujinons Stosh Durbacz on Expanding the 4K Broadcast Lens Lineup With New Portable Zooms, 94x Box Lens

Fujifilm is sharpening its focus on core broadcast production with a new wave of...

20/04/2026

Live From NAB 2026: Rock-It Sports' John Walberg on Powering Logistics, Shipping for the 2026 FIFA Men's World Cup

This upcoming summer in North America is going to be a busy one. The 2026 FIFA M...

20/04/2026

NAB 2026: Glookast outlines product updates including Media Producer UX, connectors and Premiere Pro panel

Glookast (Booth W1661) announced a series of product updates at NAB Show 2026, c...

20/04/2026

NAB 2026: Matrox Video and Amagi collaborate on cloud-based broadcast workflows using ORIGIN framework

Matrox Video and Amagi announced a collaboration to integrate the Matrox ORIGIN ...

20/04/2026

NAB 2026: Riedel SimplyLive supports expanded centralised VAR system for Argentina football league

Riedel Communications (Booth C4908) announced that the Asociaci n del F tbol Arg...

20/04/2026

NAB 2026: Ikegami introduces VFE-P07D OLED viewfinder with integrated LCD monitor

Ikegami (Booth C3819) announced the VFE-P07D monocular OLED viewfinder at NAB Sh...

20/04/2026

NAB 2026: IABM rebrands as IAMT and launches AI discovery platform and global alliance

International Association of MediaTech (IAMT), formerly known as IABM, announced...

20/04/2026

NAB 2026: Harmonic supports DIRECTV DTH platform upgrade with VOS Media Software

Harmonic (Booth W2831) announced that DIRECTV is updating its US direct-to-home (DTH) video platform using Harmonic's VOS Media Software. The deployment is...

20/04/2026

NAB 2026: Wasabi Technologies acquires Seagate Lyve Cloud business

Wasabi Technologies announced that it has acquired the Lyve Cloud business from Seagate Technology. As part of the agreement, Seagate received equity in Wasabi ...

20/04/2026

NAB 2026: EVS introduces Choreon robotics orchestration platform for unified production control

EVS (Booth N1841) has launched Choreon, a robotics controller for media producti...

20/04/2026

SportsTechBuzz at NAB 2026, Day 2: Live Reports From the Show Floor in Vegas

The NAB Show is in full swing, and the SVG and SVG Europe editorial teams are chasing down the hottest stories from all over the Las Vegas Convention Center. He...

20/04/2026

NAB 2026: Skyline Communications launches DataMiner packages on Grass Valley AMPP App Store

Skyline Communications announced the availability of its DataMiner xOps platform...

20/04/2026

NAB 2026: SNS launches Outpost, Trio and AI Suite for connected post-production workflows

Studio Network Solutions (Booth N1129) introduced a set of new products at NAB S...

20/04/2026

NAB 2026: Dell Technologies and NVIDIA present AI data platform for media workflows

Dell Technologies is showcasing its Dell AI Data Platform with NVIDIA at NAB Sho...

20/04/2026

NAB 2026: Blackmagic Design Announces Fairlight Live Software Audio Mixer

Blackmagic Design has announced Fairlight Live, a software-based live audio mixer with SMPTE 2110 support and spatial audio mixing. A public beta is available n...

20/04/2026

Live From NAB 2026: Imagine Comms Jimbo Haneklau Talks Prismon, Hybrid IP/SDI Workflows, and Cloud Playout

At the 2026 NAB Show in Las Vegas, Imagine Communications VP of Sales, Sports an...

20/04/2026

Live From NAB 2026: LiveUs Phillip Broaddus on LU900Q Launch, Nexus Cloud Platform, and REMI Growth

At the 2026 NAB Show in Las Vegas, LiveU Senior Director of Sales, Sports Philli...

20/04/2026

3 New Ways to Dive Deeper Into the Music You Love

A song that perfectly captures a moment is magic. But when you uncover the story behind it, who made it, what inspired it, and the meaning woven into the lyrics...

20/04/2026

Deity Microphones announce the PR-4

Ultra-compact 32-bit recorder set for launch Deity Microphones will soon be launching a new 32-bit six-track recorder that's been designed with producti...

20/04/2026

Lectrosonics preview the S1

Uncoming lightweight shotgun mic announced Production-sound experts Lectrosonics have recently announced the upcoming launch of a new lightweight shotgun mi...

20/04/2026

The story of Focusrite ISA

New 20-minute documentary explores iconic preamp In 2025, Focusrite commissioned a new short-form documentary with filmmaker Chris Mayes-Wright - the direct...

20/04/2026

Sampleson release Boomcha

Turn quick sketches into real drum grooves Sampleson have been experimenting with assitive production tools recently, and their latest creation aims to make...

20/04/2026

Rohde & Schwarz rolls out its full ARDRONIS counter UAS suite in a demonstration van at Counter UAS Technology Europe 2026

Rohde & Schwarz rolls out its full ARDRONIS counter UAS suite in a demonstration...

20/04/2026

Protecting America's Shores: L3Harris Keeps the Coast Guard Mission-Ready

L3Harris delivers integrated communications, navigation and C4ISR capabilities that empower the U.S. Coast Guard to protect Americas maritime interests and resp...

20/04/2026

Google Cloud Embraces the Rise of Agentic Production

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

Creators Go All in on AI, Niche Content

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

NBC Sports' Jon Miller: Broadcast Is Having a Moment'

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

Beyond the Lift and Shift': Cloud Migration's New Mandate

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

Virtual Production Finds Its Footing

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

Corporate Creators: All Companies Are Media Companies Now

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

IABM Rebrands as the International Association of MediaTech

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

CBS Detroit Debuts New AR/VR Technology-Driven Studio

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

Fox Sports Taps Appear X Platform for Remote Production

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

CueScript and Lighting Design Group Expand Customer Oppor...

CueScript and Lighting Design Group Expand Customer Opportunities Through New Partnership Find both companies at 2026 NAB Show in CueScript Booth # C 4720 ...

20/04/2026

Layercake Deepens Bitmovin Integration to Power End-to-En...

[Sydney, NSW, 20 April 2026] - Layercake, the company behind the intelligent media orchestration platform Streamcake, today announced the formalisation of its i...

20/04/2026

FOX Sports selects Appear X Platform for next-generation...

Deployment spans FOX Sports' REMI infrastructure, IP production for a major global soccer event, and its Jewel Events production systems Appear, a global l...

20/04/2026

Pro Sound Effects Launches the Industry's First and Only Native Sound Effects Integration for Avid Media Composer at NAB 2026

Pro Sound Effects Launches the Industry's First and Only Native Sound Effect...

20/04/2026

SBE Elevates Fred Willard to SBE Fellow

Share Copy link Facebook X Linkedin Bluesky Email...

20/04/2026

Blackmagic Design Announces Blackmagic Camera for iOS 3.3 Update

Blackmagic Design Announces Blackmagic Camera for iOS 3.3 Update Brie Clayton April 20, 2026 0 Comments New update adds camera control and monitoring ...

20/04/2026

Maxon Announces Free Tools and Mobile Expansion of ZBrush and Cinema 4D

Maxon Announces Free Tools and Mobile Expansion of ZBrush and Cinema 4D Brie Clayton April 20, 2026 0 Comments Cinema 4D brings professional 3D workfl...

20/04/2026

Vizrt AI Keyer kills the green screen and creates virtual scenes in any environment

Vizrt AI Keyer kills the green screen and creates virtual scenes in any environm...

20/04/2026

Register now - Market & Audience Department Ask Me Anything (AMA) Session

Register now - Market & Audience Department Ask Me Anything (AMA) Session 11 February 2026 Screen Australia Head of Market & Audience Rakel Tansley Talking to...