Sony Pixel Power calrec Sony

World-Class: NVIDIA Research Builds AI Model to Populate Virtual Worlds With 3D Objects, Characters

23/09/2022

The massive virtual worlds created by growing numbers of companies and creators could be more easily populated with a diverse array of 3D buildings, vehicles, characters and more - thanks to a new AI model from NVIDIA Research.

Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. These 3D objects are created in the same format used by popular graphics software applications, allowing users to immediately import their shapes into 3D renderers and game engines for further editing.

The generated objects could be used in 3D representations of buildings, outdoor spaces or entire cities, designed for industries including gaming, robotics, architecture and social media.

GET3D can generate a virtually unlimited number of 3D shapes based on the data it's trained on. Like an artist who turns a lump of clay into a detailed sculpture, the model transforms numbers into complex 3D shapes.

With a training dataset of 2D car images, for example, it creates a collection of sedans, trucks, race cars and vans. When trained on animal images, it comes up with creatures such as foxes, rhinos, horses and bears. Given chairs, the model generates assorted swivel chairs, dining chairs and cozy recliners.

GET3D brings us a step closer to democratizing AI-powered 3D content creation, said Sanja Fidler, vice president of AI research at NVIDIA, who leads the Toronto-based AI lab that created the tool. Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and interesting objects.

GET3D is one of more than 20 NVIDIA-authored papers and workshops accepted to the NeurIPS AI conference, taking place in New Orleans and virtually, Nov. 26-Dec. 4.

It Takes AI Kinds to Make a Virtual World The real world is full of variety: streets are lined with unique buildings, with different vehicles whizzing by and diverse crowds passing through. Manually modeling a 3D virtual world that reflects this is incredibly time consuming, making it difficult to fill out a detailed digital environment.

Though quicker than manual methods, prior 3D generative AI models were limited in the level of detail they could produce. Even recent inverse rendering methods can only generate 3D objects based on 2D images taken from various angles, requiring developers to build one 3D shape at a time.

GET3D can instead churn out some 20 shapes a second when running inference on a single NVIDIA GPU - working like a generative adversarial network for 2D images, while generating 3D objects. The larger, more diverse the training dataset it's learned from, the more varied and detailed the output.

NVIDIA researchers trained GET3D on synthetic data consisting of 2D images of 3D shapes captured from different camera angles. It took the team just two days to train the model on around 1 million images using NVIDIA A100 Tensor Core GPUs.

Enabling Creators to Modify Shape, Texture, Material GET3D gets its name from its ability to Generate Explicit Textured 3D meshes - meaning that the shapes it creates are in the form of a triangle mesh, like a papier-m ch model, covered with a textured material. This lets users easily import the objects into game engines, 3D modelers and film renderers - and edit them.

Once creators export GET3D-generated shapes to a graphics application, they can apply realistic lighting effects as the object moves or rotates in a scene. By incorporating another AI tool from NVIDIA Research, StyleGAN-NADA, developers can use text prompts to add a specific style to an image, such as modifying a rendered car to become a burned car or a taxi, or turning a regular house into a haunted one.

The researchers note that a future version of GET3D could use camera pose estimation techniques to allow developers to train the model on real-world data instead of synthetic datasets. It could also be improved to support universal generation - meaning developers could train GET3D on all kinds of 3D shapes at once, rather than needing to train it on one object category at a time.

For the latest news from NVIDIA AI research, watch the replay of NVIDIA founder and CEO Jensen Huang's keynote address at GTC:
LINK: https://blogs.nvidia.com/blog/2022/09/23/3d-generative-ai-research-vir...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

09/10/2026

Dolby Expands Dolby OptiView Platform with New Capabilities at IBC 2026

September 10 2026, 06:00 (PDT) Dolby Expands Dolby OptiView Platform with New Capabilities at IBC 2026 New Sports Intelligence helps providers better unders...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

29/09/2026

Oak View Group Names Cisco as Official Technology Partner

Oak View Group (OVG) has announced Cisco as an Official Technology Partner, with plans to standardize infrastructure across its venue network on Cisco solutions...

29/09/2026

SportsNet Pittsburgh Wins Three Mid-Atlantic Emmy Awards

SportsNet Pittsburgh (SNP) won three honors at the 2026 Mid-Atlantic Emmy Awards. Program awards went to Bucs BP in the Sports Program Live category (produced ...

29/09/2026

CBC, PWHL Revive Hockey Night in Canada' on Saturdays

The Professional Women's Hockey League has announced home openers for all 12 teams for the 2026-27 season, along with a broadcast partnership with CBC under...

29/09/2026

Manifold Technologies Recognized as MXL Contributor Following IBC Innovation Award

Manifold technologies has been recognized as a contributor to the Media eXchange...

29/09/2026

Fubo Launches My Local Sports RSN Add-On Package

FuboTV has announced My Local Sports, allowing subscribers in select markets to add their local regional sports network (RSN) as an add-on to a Fubo Sports subs...

29/09/2026

ANI Deploys Grass Valley AMPP and Framelight X for News Operations

Asian News International (ANI), a wire agency and video news provider based in India, has deployed Grass Valley's AMPP Media Operating System and Framelight...

29/09/2026

SABC Deploys Ikegami UHK-X600 Cameras for IP-Based Production

South African Broadcasting Corporation (SABC) has deployed Ikegami UHK-X600 cameras as part of an upgrade to IP-based production infrastructure. The deployment ...

29/09/2026

Busy in Bristol: ESPN Headquarters Come Alive With New Studio W, Remote-Production Workflows

REMI, REMCO models will drive the bulk of the broadcaster's NHL productions ...

29/09/2026

NASCAR Productions' Keith D'Alessandro on the Continued Evolution of Its Centralized Production Hub

Three years in, the facility is supporting more centralized race production, com...

29/09/2026

NBC Sports Brings Robust Sunday Night Baseball Production Playbook to MLB Wild Card Round

NBC, Peacock, and NBSN are carrying all four Wild Card series with a production ...

29/09/2026

Sundance Institute Announces 2026 Doc Fund Grantees

19 Nonfiction Projects Across All Stages of Production to Receive More Than $1 million in Support as Documentary Film Program Approaches 25th Anniversary LOS ...

29/09/2026

Zynaptiq introduce Loud & Wide

Two new plug-ins launch alongside updated bundles Zynaptiq have just announced the launch of two new plug-ins, alongside new and improved versions of two of...

29/09/2026

Wes Audio unveil the ngOmniBox

All-in-one bus processor revealed Wes Audio's ng product range has come a long way since it was first introduced, and now includes a tasty selection of ...

29/09/2026

First Baptist Nashville Drastically Upgrades Production Communications with...

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({ content_source:......

29/09/2026

Fubo TV Launches My Local Sports'

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

ARRI Targets Content Creators with New HONOR Smartphone

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

Asian News International Enhances Workflow with Grass Valley AMPP, FramelightX

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

SABC Chooses Ikegami UHK-X600 Cameras to Support IP Studio Transition

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

SABC Upgrades to IP-Based Production with Ikegami UHK-X60...

Ikegami announces that the South African Broadcasting Corporation, SABC, has chosen Ikegami UHK-X600 camera systems as part of an advance to IP-based operation ...

29/09/2026

Blendkit Offers 20% Discount to Celebrate Crossing 150K Asset Milestone

Blendkit Offers 20% Discount to Celebrate Crossing 150K Asset Milestone Brie Clayton September 29, 2026 0 Comments Blendkit, the largest 3D asset libr...

29/09/2026

Ease Live Reports over 75 Million Interactive Sessions Du...

Interactive streaming specialist Ease Live supported more than 76.5 million fan sessions across three major European media operators during this summers FIFA Wo...

29/09/2026

Ease Live to monetise key moments in live sports

Ease Live has added support for VAST (Video Ad Serving Template) to its graphical overlay technology, opening up new ways for broadcasters, streamers and rights...

29/09/2026

Rise AV launches awards celebrating the women advocates a...

Rise AV has announced the launch of the inaugural Rise AV Awards, a new global awards programme celebrating the women, advocates and organisations across the AV...

29/09/2026

Netflix Deploys Advanced Sony PTZ Cameras For Wonka's The Golden Ticket' Production

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

Massachusetts Broadcasters Association Names Gene Lavanchy 2026 Broadcaster of the Year

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

ESPN Reports Its Most-Watched WNBA Regular Season

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

Scripps Taps Digital News Veteran Sean McGarvy to Lead News Strategy

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

Gray Raises Political Ad Revenue Estimates

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

Edward Czarnecki Uses Game Theory to Assess ATSC 3.0 Transition Policies

Share Copy link Facebook X Linkedin Bluesky Email...

29/09/2026

First Baptist Nashville Drastically Upgrades Production C...

Clear-Com announced that First Baptist Nashville has upgraded its production communications with a Clear-Com intercom solution as part of a comprehensive sanct...

29/09/2026

Blackmagic Cinema Camera 6K Captured Action Shots for Sketch

Blackmagic Cinema Camera 6K Captured Action Shots for Sketch Brie Clayton September 28, 2026 0 Comments Camera's portable design and dynamic range...

29/09/2026

Kean Production Services Deploys AJA ColorBox to Deliver Consistent, Accurate Color Across In-venue Displays at Live Events

Kean Production Services Deploys AJA ColorBox to Deliver Consistent, Accurate Co...

29/09/2026

Berklee Alumni and Faculty Nominated for the 2026 Latin Grammy Awards

Berklee Alumni and Faculty Nominated for the 2026 Latin Grammy Awards A total of 28 Berklee alumni and faculty earned nominations across 20 categories this ye...

29/09/2026

Berklee's K-Pop & Beyond Explores Connections Between K-Pop and Hip-Hop

Berklee's K-Pop & Beyond Explores Connections Between K-Pop and Hip-Hop Now in its fifth year, the series brings together artists, industry leaders, and s...

29/09/2026

The energy transition will only succeed if it lasts

We tend to think we live in a digital world, with hopes and fears around artificial intelligence (AI), quantum computing, data centres and the next big technolo...

29/09/2026

Apple releases new guide to help families build healthy digital habits - UPDATE - Posted on 29 September 2026

Today, in partnership with the American Academy of Pediatrics (AAP), Apple relea...

29/09/2026

Final Cut Camera now supports variable aperture on iPhone\u00A018\u00A0Pro - UPDATE - Posted on 29 September 2026

Final Cut Camera, the professional filming app for iPhone, receives a major upda...

29/09/2026

RT lyric fm launch the hunt for Choirs for Christmas

RT lyric fm is proud to announce the annual Choirs for Christmas competition callout. Warm up those vocal chords and join us on RT lyric fm for a celebration ...

28/09/2026

NHL Names Adobe as an Official Creative, Marketing and AI Partner

The NHL has announced Adobe as an Official Creative, Marketing and AI Partner beginning with the 2026-27 season. The partnership will provide the league and its...

28/09/2026

MultiDyne to Showcase Expanded Portfolio Including MRMC Robotics at CABSAT 2026

MultiDyne will exhibit at CABSAT 2026 (October 5-7, Dubai Exhibition Centre), showcasing an expanded technology portfolio following its June 2026 acquisition of...

28/09/2026

ARRI and HONOR Launch Co-Branded HONOR Magic9 Pro Max

ARRI and HONOR have announced the HONOR Magic9 Pro Max, a flagship smartphone carrying ARRI branding on the device and packaging as the first product-level expr...

28/09/2026

Net Insight Adds Nimbra 1020G to Pan-Asian Live Media Trial Network

Net Insight has added the Nimbra 1020G to its pan-Asian live media trial network, giving customers and partners in the region a way to evaluate ST 2110 and JPEG...

28/09/2026

Vindral Named Recipient of 2026 Technology and Engineering Emmy Award

Vindral has been named a recipient of a 2026 Technology and Engineering Emmy Award by the National Academy of Television Arts and Sciences in the category of We...

28/09/2026

Interra Systems to Demonstrate QC and Monitoring Portfolio at CABSAT 2026

Interra Systems will exhibit at CABSAT 2026 (Stand B2-37, Dubai Exhibition Centre, October 5-7), demonstrating its BATON QC platform, ORION monitoring solutions...

28/09/2026

PGA TOUR's Michael Raimondo on Agentic Production, ShotLink Automation, and the Evolution of PGA TOUR Studios

Two years into its new production facility, the TOUR is expanding centralized wo...