Sony Pixel Power calrec Sony

World-Class: NVIDIA Research Builds AI Model to Populate Virtual Worlds With 3D Objects, Characters

23/09/2022

The massive virtual worlds created by growing numbers of companies and creators could be more easily populated with a diverse array of 3D buildings, vehicles, characters and more - thanks to a new AI model from NVIDIA Research.

Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. These 3D objects are created in the same format used by popular graphics software applications, allowing users to immediately import their shapes into 3D renderers and game engines for further editing.

The generated objects could be used in 3D representations of buildings, outdoor spaces or entire cities, designed for industries including gaming, robotics, architecture and social media.

GET3D can generate a virtually unlimited number of 3D shapes based on the data it's trained on. Like an artist who turns a lump of clay into a detailed sculpture, the model transforms numbers into complex 3D shapes.

With a training dataset of 2D car images, for example, it creates a collection of sedans, trucks, race cars and vans. When trained on animal images, it comes up with creatures such as foxes, rhinos, horses and bears. Given chairs, the model generates assorted swivel chairs, dining chairs and cozy recliners.

GET3D brings us a step closer to democratizing AI-powered 3D content creation, said Sanja Fidler, vice president of AI research at NVIDIA, who leads the Toronto-based AI lab that created the tool. Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and interesting objects.

GET3D is one of more than 20 NVIDIA-authored papers and workshops accepted to the NeurIPS AI conference, taking place in New Orleans and virtually, Nov. 26-Dec. 4.

It Takes AI Kinds to Make a Virtual World The real world is full of variety: streets are lined with unique buildings, with different vehicles whizzing by and diverse crowds passing through. Manually modeling a 3D virtual world that reflects this is incredibly time consuming, making it difficult to fill out a detailed digital environment.

Though quicker than manual methods, prior 3D generative AI models were limited in the level of detail they could produce. Even recent inverse rendering methods can only generate 3D objects based on 2D images taken from various angles, requiring developers to build one 3D shape at a time.

GET3D can instead churn out some 20 shapes a second when running inference on a single NVIDIA GPU - working like a generative adversarial network for 2D images, while generating 3D objects. The larger, more diverse the training dataset it's learned from, the more varied and detailed the output.

NVIDIA researchers trained GET3D on synthetic data consisting of 2D images of 3D shapes captured from different camera angles. It took the team just two days to train the model on around 1 million images using NVIDIA A100 Tensor Core GPUs.

Enabling Creators to Modify Shape, Texture, Material GET3D gets its name from its ability to Generate Explicit Textured 3D meshes - meaning that the shapes it creates are in the form of a triangle mesh, like a papier-m ch model, covered with a textured material. This lets users easily import the objects into game engines, 3D modelers and film renderers - and edit them.

Once creators export GET3D-generated shapes to a graphics application, they can apply realistic lighting effects as the object moves or rotates in a scene. By incorporating another AI tool from NVIDIA Research, StyleGAN-NADA, developers can use text prompts to add a specific style to an image, such as modifying a rendered car to become a burned car or a taxi, or turning a regular house into a haunted one.

The researchers note that a future version of GET3D could use camera pose estimation techniques to allow developers to train the model on real-world data instead of synthetic datasets. It could also be improved to support universal generation - meaning developers could train GET3D on all kinds of 3D shapes at once, rather than needing to train it on one object category at a time.

For the latest news from NVIDIA AI research, watch the replay of NVIDIA founder and CEO Jensen Huang's keynote address at GTC:
LINK: https://blogs.nvidia.com/blog/2022/09/23/3d-generative-ai-research-vir...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

08/05/2026

Miri V410 Encoder-Decoder Wins 2026 NAB Show Product of t...

For the second straight year, Miri Technologies Inc. has been honored with a pair of coveted awards at the broadcast, media and entertainment industry's pre...

08/05/2026

Nine Alumni Nominated for Tony Awards

Nine Alumni Nominated for Tony Awards More than 20 alumni contributed to this year's Tony-nominated productions. May 7, 2026 By Tori Donahue Boston Co...

08/05/2026

Report: Rise of Video Podcasts Prompts Hybrid Approach

Share Copy link Facebook X Linkedin Bluesky Email...

08/05/2026

U.K. Military Broadcaster Taps Synamedia for Digital Platform

Share Copy link Facebook X Linkedin Bluesky Email...

08/05/2026

Gayle Benson To Receive NAB Leadership Foundation Leadership Award

Share Copy link Facebook X Linkedin Bluesky Email...

08/05/2026

Dems Blast FCC Disney Probe As 'Egregious Abuse of Power'

Share Copy link Facebook X Linkedin Bluesky Email...

08/05/2026

Liberty University on why broadcast technology isn't just a technical investment, it's a human one

Liberty University on why broadcast technology isn't just a technical invest...

08/05/2026

COW Jobs: UGC, On-Camera Video Content

COW Jobs: UGC, On-Camera Video Content Brie Clayton May 7, 2026 0 Comments UGC, On-Camera Video Content April 17, 2026Documentary Editor - US,......

07/05/2026

Sudan Digital Investigation Hub launches to support journalists navigating disinformation in conflict

Journalists reporting on Sudan are working in one of the most complex and fast-m...

07/05/2026

Victory+ CEO Neil Gruninger on the Impact of the Dallas Cowboys Deal, Creator-Led Content, and Free In-Market Access for Fans

Multi-year partnership positions Victory+ as a free home for Dallas Cowboys orig...

07/05/2026

SVG Students To Watch: Ella Peterson, University of Illinois

From sideline reporting to directing and producing, the talented sophomore is building a well-rounded foundation for a career in live sports production In the ...

07/05/2026

NAB 2026: Cobalt Digital Wins Two Future Best of Show Awards

Cobalt Digital has received two Future Best of Show awards at NAB Show 2026. The COBALT blueCORE platform was recognized by TV Tech, and the COBALT PACIFIC ULL-...

07/05/2026

Scripps National Spelling Bee Revamps Broadcast; Mina Kimes to Host, Embassy Rows Michael Davies to Produce

ESPN analyst Mina Kimes will host the televised semifinals and finals of the 202...

07/05/2026

MUBI Selects Bitmovin as Cloud VOD Encoding Partner

Bitmovin has announced that MUBI, the global film streaming platform, has selected Bitmovin as its cloud VOD encoding partner. Bitmovin's encoding infrastru...

07/05/2026

NBCUniversal Telemundo Enterprises and U.S. Soccer Extend Spanish-Language Media Rights Through 2030

NBCUniversal Telemundo Enterprises and the U.S. Soccer Federation have announced...

07/05/2026

Angel City Football Club and Victory+ Announce Regional Broadcast Partnership

Angel City Football Club (ACFC) and Victory have announced a regional broadcast partnership bringing live match coverage to fans across the greater Los Angeles...

07/05/2026

Spiideo Launches AI Highlights for Automated Highlight and Clip Creation

Spiideo has announced the launch of AI Highlights inside Spiideo Play, its automated sports production platform. AI Highlights combines video, event data, audio...

07/05/2026

Leostream Announces Unified HPC Remote Access Ecosystem for Post-Production and Enterprise Workflows

Leostream Corporation has announced a unified remote access ecosystem for high-p...

07/05/2026

Victory+ Adds Atlanta Dream Local Broadcasts to Free Streaming Platform

The Atlanta Dream has partnered with Victory to stream all locally broadcast Dream games for free, expanding the team's digital distribution strategy and g...

07/05/2026

Riedel Communications Appoints Marc Engroff as CFO; Frank Eischet Moves to Group COO Role

Riedel Communications has announced the appointment of Marc Engroff as Chief Fin...

07/05/2026

ARD Extends SES Satellite Distribution Agreement Through 2039

SES and ARD, Germany's largest public broadcasting network, have announced a long-term extension of their satellite distribution partnership through 2039. U...

07/05/2026

Case Study: How Pro Cheer League Built a Broadcast-First Arena Production

UpLight Technologies delivers a flexible video and lighting system for a new televised sportLaunching the Pro Cheer League required more than creating a compell...

07/05/2026

Full Day Productions and GSE Worldwide Launch Womens College Volleyball Showcase at AT&T Stadium

Full Day Productions and GSE Worldwide have announced Spikes Under the Lights, a...

07/05/2026

Deine persnliche Spotify DJ Experience ist jetzt in vier neuen Sprachen und weiteren Mrkten verfgbar

Seit dem Start im Jahr 2023 hat DJ (Beta) das personalisierte H rerlebnis von 94...

07/05/2026

Il DJ personalizzato di Spotify ora disponibile in 4 nuove lingue e in pi Paesi

Da quando stata presentata nel 2023, DJ (beta) ha aiutato a definire un'esperienza d'ascolto pi personalizzata per 94 milioni di utenti Spotify Premi...

07/05/2026

Your Personalized DJ Experience Is Expanding With 4 New Languages and More Markets

Since launching in 2023, DJ (beta) has helped shape a more personalized listenin...

07/05/2026

Seu DJ personalizado do Spotify t chegando em mais lugares - agora em 4 novos idiomas

Desde o lan amento em 2023, o DJ (beta) j ajudou a deixar a experi ncia de ouvi...

07/05/2026

Save Your Personal Podcast to Spotify and Listen Anywhere

From our earliest days, Spotify has been built on a simple principle: Great audio should be easy to reach. It's what's driven us to expand from music to...

07/05/2026

Spotify Celebrates API Heritage Month With New Music and Artist Spotlights

Asian and Pacific Islander artists continue to shape the global soundscape, pushing creative boundaries and connecting with fans worldwide. This Asian & Pacific...

07/05/2026

Symphonic Elements Chooir by UJAM

Company announce long-requested choir instrument In the latest expansion of their Symphonic Elements series, UJAM have introduced an all-new vocal instrumen...

07/05/2026

Buchla debut Ziggy synthesizer

Modular-inspired sound generation with digital control Buchla have introduced Ziggy, a new compact synthesizer built around the company's complex oscill...

07/05/2026

Polyend Drums hybrid drum machine

Combines analogue voices with digital synthesis and sequencing Polyend have announced Drums, a new hybrid analogue and digital drum machine that combines sy...

07/05/2026

Nektar Panorama CS12: Reaper & Fender Studio Pro support

Control surface now officially supports four DAWs When Nektar launched the Panorama CS12 control surface, it worked exclusively with Apple's Logic Pro, ...

07/05/2026

SGL Carbon: Expected lower sales, but EBITDA margin improved. Forecast confirmed

In the first quarter 2026, SGL Carbon generated consolidated sales of 184.5 million, which was 49.8 million, or 21.3%, lower than in the prior year (Q1 2025: ...

07/05/2026

Rohde & Schwarz and Greenerwave achieve precise and fast ESA antenna characterization using near-field technology

Rohde & Schwarz and Greenerwave achieve precise and fast ESA antenna characteriz...

07/05/2026

L3Harris Red Wolf + SKY RAIDER II INTERNATIONAL: Unmatched Flexibility for Kinetic Strike, EW, ISR and Support Missions

Two multi-role L3Harris products - the Red Wolf launched effects vehicle and SK...

07/05/2026

US Air Force Selects L3Harris to Develop Digital Infrastructure for its Advanced Battle Management System Network

L3Harris will be developing key features of a secure and resilient digital infra...

07/05/2026

New Nielsen Report Shows The Profound Impact of Asian Influence on Sports and Pop Culture

Alysa Liu and Shohei Ohtani Help Drive Viewership as 91 of the Top 100 Broadcast...

07/05/2026

IAB Tech Lab Introduces Standardized Signals For CTV Ad Portfolio

Share Copy link Facebook X Linkedin Bluesky Email...

07/05/2026

Cobalt Digital Wins Two Future Best of Show Awards at 202...

Cobalt Digital Wins Two Future Best of Show Awards at 2026 NAB Show Manufacturer Recognized by TV Tech and TVBEurope for Innovation in signal processing Cobal...

07/05/2026

FOR-A delivers creativity practicality and technical exce...

Software and hardware platforms, AI power and user-friendliness on show...

07/05/2026

Intinor brings advanced IP transport diagnostics and HDR...

Intinor will demonstrate its latest technical enhancements for the Direkt series at BroadcastAsia 2026. With a continued focus on reliable contribution and remo...

07/05/2026

Bitmovin Powers Global Film Streaming Platform MUBI

Bitmovin has announced that MUBI has chosen Bitmovin as its cloud VOD encoding partner, replacing MUBI's legacy on premises encoding setup to improve scalab...

07/05/2026

Meet Graduates from Berklees Class of 2026

Meet Graduates from Berklees Class of 2026 Members of this years graduating class reflect on their proudest moments at Berklee and look ahead to whats next. ...

07/05/2026

Convergence is Now Established Reality,' Says NAB 2026 Report

Share Copy link Facebook X Linkedin Bluesky Email...