Sony Pixel Power calrec Sony

World-Class: NVIDIA Research Builds AI Model to Populate Virtual Worlds With 3D Objects, Characters

23/09/2022

The massive virtual worlds created by growing numbers of companies and creators could be more easily populated with a diverse array of 3D buildings, vehicles, characters and more - thanks to a new AI model from NVIDIA Research.

Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. These 3D objects are created in the same format used by popular graphics software applications, allowing users to immediately import their shapes into 3D renderers and game engines for further editing.

The generated objects could be used in 3D representations of buildings, outdoor spaces or entire cities, designed for industries including gaming, robotics, architecture and social media.

GET3D can generate a virtually unlimited number of 3D shapes based on the data it's trained on. Like an artist who turns a lump of clay into a detailed sculpture, the model transforms numbers into complex 3D shapes.

With a training dataset of 2D car images, for example, it creates a collection of sedans, trucks, race cars and vans. When trained on animal images, it comes up with creatures such as foxes, rhinos, horses and bears. Given chairs, the model generates assorted swivel chairs, dining chairs and cozy recliners.

GET3D brings us a step closer to democratizing AI-powered 3D content creation, said Sanja Fidler, vice president of AI research at NVIDIA, who leads the Toronto-based AI lab that created the tool. Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and interesting objects.

GET3D is one of more than 20 NVIDIA-authored papers and workshops accepted to the NeurIPS AI conference, taking place in New Orleans and virtually, Nov. 26-Dec. 4.

It Takes AI Kinds to Make a Virtual World The real world is full of variety: streets are lined with unique buildings, with different vehicles whizzing by and diverse crowds passing through. Manually modeling a 3D virtual world that reflects this is incredibly time consuming, making it difficult to fill out a detailed digital environment.

Though quicker than manual methods, prior 3D generative AI models were limited in the level of detail they could produce. Even recent inverse rendering methods can only generate 3D objects based on 2D images taken from various angles, requiring developers to build one 3D shape at a time.

GET3D can instead churn out some 20 shapes a second when running inference on a single NVIDIA GPU - working like a generative adversarial network for 2D images, while generating 3D objects. The larger, more diverse the training dataset it's learned from, the more varied and detailed the output.

NVIDIA researchers trained GET3D on synthetic data consisting of 2D images of 3D shapes captured from different camera angles. It took the team just two days to train the model on around 1 million images using NVIDIA A100 Tensor Core GPUs.

Enabling Creators to Modify Shape, Texture, Material GET3D gets its name from its ability to Generate Explicit Textured 3D meshes - meaning that the shapes it creates are in the form of a triangle mesh, like a papier-m ch model, covered with a textured material. This lets users easily import the objects into game engines, 3D modelers and film renderers - and edit them.

Once creators export GET3D-generated shapes to a graphics application, they can apply realistic lighting effects as the object moves or rotates in a scene. By incorporating another AI tool from NVIDIA Research, StyleGAN-NADA, developers can use text prompts to add a specific style to an image, such as modifying a rendered car to become a burned car or a taxi, or turning a regular house into a haunted one.

The researchers note that a future version of GET3D could use camera pose estimation techniques to allow developers to train the model on real-world data instead of synthetic datasets. It could also be improved to support universal generation - meaning developers could train GET3D on all kinds of 3D shapes at once, rather than needing to train it on one object category at a time.

For the latest news from NVIDIA AI research, watch the replay of NVIDIA founder and CEO Jensen Huang's keynote address at GTC:
LINK: https://blogs.nvidia.com/blog/2022/09/23/3d-generative-ai-research-vir...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

30/04/2026

Study: Rokus Low-Cost, Ad-Free Howdy Streamer Hits 1 Million Subs

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

MLS Innovation Lab Selects AI Partners

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

NRB Files FCC Complaint Over Jimmy Kimmel Live!' Monologue

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

D.C. Court Denies Emergency Stay of Nexstar/Tegna Merger

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

NAB Criticizes FCC for Ordering Early Renewal of ABC-Owned Stations

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

FCC Approves Station Swaps Between Scripps and Gray Media

Share Copy link Facebook X Linkedin Bluesky Email...

30/04/2026

Telestream Introduces Pulse - a Software-Defined Test an...

A flexible monitoring platform designed to simplify ST 2110 operations, consolidate vendor tools, and support modern live production environments. See it at NAB...

30/04/2026

Langlev Takes Run Amok to Sundance with Zeiss Supreme Pri...

Sundance-premiering Run Amok is an expressive, unconventional take on today's teen experience, replete with musical numbers. When cinematographer Shachar ...

30/04/2026

Studio Technologies Elevates Jacksonville State Universit...

JACKSONVILLE, AL, APRIL 29, 2026 Jacksonville State University, known as Jax State and a proud NCAA Division I member of Conference USA, has transformed its a...

30/04/2026

Knowledge Network selects ThinkAnalytics to launch AI-pow...

Transforming viewer and content data into real-time intelligence to deliver relevant streaming experiences at scale ThinkAnalytics, the global leader in AI-pow...

30/04/2026

Sports Production, Delivery is Big Biz at NAB

Sports Production, Delivery is Big Biz at NAB Andy Marken April 29, 2026 0 Comments Hero image source: NAB One of the neat things about trade shows i...

30/04/2026

April 29, 2026

Scripps Research ranks third in 2026 Cure Innovation Index April 29, 2026 LA JOLLA, CA Scripps Research ranked third in the inaugural 2026 Cure Innovation In...

29/04/2026

Churchill Downs Racetrack Lifts Curtain on New Big Board Ahead of Kentucky Derby Week

It was a delicate job in a 150-year-old venue laden with traditions. Begun at th...

29/04/2026

How the MLS Innovation Lab Enhances Everything From Athlete Performance to Production to Ticket Sales

In annual event, the league gives startup companies the opportunity to prove the...

29/04/2026

Save the Date: SVG Venues & Teams Summit Travels South to Miamis Kaseya Center on Aug. 12

Panel discussions, networking, and a facility tour will take place in the renova...

29/04/2026

The Cast and Crew of Conbody VS Everybody Have Each Other's Backs for Life

(L-R) Derek Drescher, Coss Marte, and Syretta Wright have each other's backs. (Micheal Hurcomb/Shutterstock for Sundance Film Festival) By Veronika Lee Cla...

29/04/2026

Free Tilt EQ plug-in from Techivation

Combines EQ and harmonic distortion Techivation's latest release is a simple EQ designed to offer quick control over a source's overall tonal balanc...

29/04/2026

Expressive E launch the Osmose CE

Two new MPE controllers announced Expressive E caused quite a stir when they released the Osmose, making the sort of expression that was once reserved for p...

29/04/2026

iZotope RX 12 is here

New modules & enhanced machine-learning The latest version of iZotope's flagship restoration suite is now available, and now offers over 50 tools design...

29/04/2026

Surgeon Dr Jasmina Kevric wins 2026 Les Murray Award

Surgeon Dr Jasmina Kevric wins 2026 Les Murray Award 29 April, 2026 Media releases Australia for UNHCR and SBS are proud to announce that Dr Jasmina Kevric...

29/04/2026

YEP Spotlight: Julissa Padilla

Some people stumble into their passion. Julissa Padilla walked straight into a film vault. For her, entertainment was never just about the movies themselves. It...

29/04/2026

Clear-Com Promotes Brian Grahn, Ben Turnwell to Expanded Roles

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

The CW Unveils Major Streaming Deals with ESPN and Roku

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

Clear-Com Announces New Roles for Brian Grahn and Ben Tur...

Clear-Com has appointed Brian Grahn as Market Outreach Manager of the Americas and Ben Turnwell as Business Development Manager for EMEA live, expanding their ...

29/04/2026

Reduce Workflow Complexity with nxtedition at MPTS 2026

nxtedition is bringing its range of consolidated production tools to MPTS 2026, with new developments spanning transcription, editing, graphics and AI-assisted ...

29/04/2026

Telxius taps Synamedia to unlock seamless global multi-CD...

Quortex Switch to boost the streaming experience for Telxius customers, reaching millions of viewers worldwide Synamedia and Telxius, the leading global connec...

29/04/2026

freispace and Projective Announce Joint Showcase of Integ...

freispace, the leading ERP-as-a-Service platform for media and entertainment production, and Projective, a leading provider of post-production collaboration tec...

29/04/2026

DHD Celebrates 30th Anniversary at 2026 NAB Show

DHD reports strong interest in its broadcast audio product range, exhibited at the April 19th-22nd NAB Show in Las Vegas. The event attracted a claimed 58,000 a...

29/04/2026

Student Spotlight: Alan Catz

Student Spotlight: Alan Catz The Argentine film and game composer talks about working on League of Legends, receiving Berklee's BMI Award, and the lifelon...

29/04/2026

Jay Jennings Builds the Worlds You Hear on Screen

Jay Jennings Builds the Worlds You Hear on Screen The supervising sound designer behind A Minecraft Movie, The Meg, Letters from Iwo Jima, and dozens of other...

29/04/2026

Paramount Will be 49.5% Foreign Owned After WBD Merger

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

NAB Show 2026: AI, Vertical and BPS Dominate Broadcasters' Discussions

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

IAB Tech Lab Announces New OpenRTB Attributes

Share Copy link Facebook X Linkedin Bluesky Email...

29/04/2026

Meet the eight finalists for RT Today's TV Home Cook competition

Voting opens at 4pm today RT 's Today show have announced the eight finalists for their TV Home Cook competition. Amateur cooks from Cork, Dublin, Galway ...

29/04/2026

VEON and Kyivstar Fulfill Commitment to Invest USD 1 Billion in Ukraine over 2023-2027 Ahead of Schedule

29 Apr 2026 VEON and Kyivstar Fulfill Commitment to Invest USD 1 Billion in Ukr...

29/04/2026

U challenges stars and their dogs to compete for Wagging Rights

Rhod Gilbert, Harriet Kemsley, Kae Kurd, Sara Pascoe and Vicki Pattison to take part in brand new series on free streaming service U London, 29th April 2026: F...

29/04/2026

Katie Price: Nothing to Hide, a Sky Original documentary series from BAFTA-winning Mindhouse, coming this summer

Wednesday 29 April 2026 Katie Price: Nothing to Hide, a Sky Original documentar...

29/04/2026

Sky announces upcoming documentary series The Lie That Exposed The Truth

Re-examining the case of Ellie Williams and the wider story of grooming in the town of BarrowWednesday 29 April 2026 Sky announces upcoming documentary series ...

29/04/2026

Jennifer Garner to lead an all-star cast in new Sky Exclusive summer escape The Five Star Weekend, coming to screens this July

Wednesday 29 April 2026 Jennifer Garner to lead an all-star cast in new Sky Exc...

29/04/2026

'SUPERNOVA: GENESIS' Reached a Peak Audience of More Than 6.5 Million Viewers on Netflix

Back to All News SUPERNOVA: GENESIS Reached a Peak Audience of More Than 6.5 Mi...

29/04/2026

dB Sponsors College Sustainability Week

Students and staff from Hills Road Sixth Form College in Cambridge ran a 4.5km course around the roads of Cambridge as part of their annual programme of sustain...

29/04/2026

Listen to the birds sing! Experience mother nature in a seven-hour radio marathon

The Dawn Chorus airs Sunday 3 May from midnight to 7am on RT Radio 1 and RT ly...

29/04/2026

April 28, 2026

Jin-Quan Yu elected to the National Academy of Sciences Yu is recognized for his pioneering work in synthetic organic chemistry. April 28, 2026 LA JOLLA, CA S...

28/04/2026

Sound for the NFL Draft Show Balances Often Conflicting Goals

The audio team for the entertainment event must blend speech intelligibility with full-range music reproduction while considering the broadcast Last week's...