Sony Pixel Power calrec Sony

World-Class: NVIDIA Research Builds AI Model to Populate Virtual Worlds With 3D Objects, Characters

23/09/2022

The massive virtual worlds created by growing numbers of companies and creators could be more easily populated with a diverse array of 3D buildings, vehicles, characters and more - thanks to a new AI model from NVIDIA Research.

Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. These 3D objects are created in the same format used by popular graphics software applications, allowing users to immediately import their shapes into 3D renderers and game engines for further editing.

The generated objects could be used in 3D representations of buildings, outdoor spaces or entire cities, designed for industries including gaming, robotics, architecture and social media.

GET3D can generate a virtually unlimited number of 3D shapes based on the data it's trained on. Like an artist who turns a lump of clay into a detailed sculpture, the model transforms numbers into complex 3D shapes.

With a training dataset of 2D car images, for example, it creates a collection of sedans, trucks, race cars and vans. When trained on animal images, it comes up with creatures such as foxes, rhinos, horses and bears. Given chairs, the model generates assorted swivel chairs, dining chairs and cozy recliners.

GET3D brings us a step closer to democratizing AI-powered 3D content creation, said Sanja Fidler, vice president of AI research at NVIDIA, who leads the Toronto-based AI lab that created the tool. Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and interesting objects.

GET3D is one of more than 20 NVIDIA-authored papers and workshops accepted to the NeurIPS AI conference, taking place in New Orleans and virtually, Nov. 26-Dec. 4.

It Takes AI Kinds to Make a Virtual World The real world is full of variety: streets are lined with unique buildings, with different vehicles whizzing by and diverse crowds passing through. Manually modeling a 3D virtual world that reflects this is incredibly time consuming, making it difficult to fill out a detailed digital environment.

Though quicker than manual methods, prior 3D generative AI models were limited in the level of detail they could produce. Even recent inverse rendering methods can only generate 3D objects based on 2D images taken from various angles, requiring developers to build one 3D shape at a time.

GET3D can instead churn out some 20 shapes a second when running inference on a single NVIDIA GPU - working like a generative adversarial network for 2D images, while generating 3D objects. The larger, more diverse the training dataset it's learned from, the more varied and detailed the output.

NVIDIA researchers trained GET3D on synthetic data consisting of 2D images of 3D shapes captured from different camera angles. It took the team just two days to train the model on around 1 million images using NVIDIA A100 Tensor Core GPUs.

Enabling Creators to Modify Shape, Texture, Material GET3D gets its name from its ability to Generate Explicit Textured 3D meshes - meaning that the shapes it creates are in the form of a triangle mesh, like a papier-m ch model, covered with a textured material. This lets users easily import the objects into game engines, 3D modelers and film renderers - and edit them.

Once creators export GET3D-generated shapes to a graphics application, they can apply realistic lighting effects as the object moves or rotates in a scene. By incorporating another AI tool from NVIDIA Research, StyleGAN-NADA, developers can use text prompts to add a specific style to an image, such as modifying a rendered car to become a burned car or a taxi, or turning a regular house into a haunted one.

The researchers note that a future version of GET3D could use camera pose estimation techniques to allow developers to train the model on real-world data instead of synthetic datasets. It could also be improved to support universal generation - meaning developers could train GET3D on all kinds of 3D shapes at once, rather than needing to train it on one object category at a time.

For the latest news from NVIDIA AI research, watch the replay of NVIDIA founder and CEO Jensen Huang's keynote address at GTC:
LINK: https://blogs.nvidia.com/blog/2022/09/23/3d-generative-ai-research-vir...
See more stories from nvidia

Most recent headlines

23/12/2025

Nielsen, Roku Expand Measurement Partnership

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

23/12/2025

PwC: Streaming Market Shifting to 'Scale and Sustainability'

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

23/12/2025

Inside the Gray Innovation Lab

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

23/12/2025

ESPN Renews Deal for Heisman Trophy Coverage

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

23/12/2025

Gray Media to Acquire WBBJ from Bahakel Communications

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

23/12/2025

Taking the Stage at Carnegie Hall-On a Global Scale

Taking the Stage at Carnegie Hall-On a Global Scale Boston Conservatory Orchestra students reflect on their epic concert marking the 80th session of the UN Gene...

22/12/2025

SVG New Sponsor Spotlight: Presidio's Neerav Shah on the Role of Its Captivate and Resonate Platforms in Sports Production

SVG New Sponsor Spotlight: Presidio's Neerav Shah on the Role of Its Captiva...

22/12/2025

Hitting the Bullseye: Sky Sports Readies Itself for the Biggest PDC World Darts Championship to Hit Ally Pally Yet

Hitting the bullseye: Sky Sports readies itself for the biggest PDC World Darts ...

22/12/2025

Unique Skillset: Bringing New Directors to the World of Darts at The Worlds with Sky Sports

Unique skillset: Bringing new directors to the world of darts at The Worlds with...

22/12/2025

Gravity Media Prepares for a Flight of Fancy With the PDC World Darts Championship 2025 for Sky Sports

Gravity Media prepares for a flight of fancy with the PDC World Darts Championsh...

22/12/2025

One Hundred and Eighty: Gravity Media on Hitting the Production Bullseye at the World Darts Championship 2025

One hundred and eighty: Gravity Media on hitting the production bullseye at the ...

22/12/2025

The Famous Group's Jon Slusser on Fascinating Fans Through Immersive Content Experiences

The Famous Group's Jon Slusser on Fascinating Fans Through Immersive Content...

22/12/2025

ESPN's Meg Aronowitz on Continuing High-Quality Broadcasts of Collegiate Sports, Expanding Growth of Internal Production Team

ESPN's Meg Aronowitz on Continuing High-Quality Broadcasts of Collegiate Spo...

22/12/2025

ESPN Takes Data-Driven Storytelling to New Heights with MNF Playbook with Next Gen Stats' NFL Altcasts

ESPN Takes Data-Driven Storytelling to New Heights with MNF Playbook with Next ...

22/12/2025

A Decade of Giving: Fest & Flauschig' Christmas Circus Celebrates Record Turnout and Generosity

For a decade, popular German podcast Fest & Flauschig has hosted an annual Chris...

22/12/2025

Paramount and Netflix Boast Double-Digit Gains in Nielsen's November Media Distributor Gauge

Paramount Scores Largest Share Increase Among Distributors as Paramount and CBS...

22/12/2025

Nielsen and Roku Expand Strategic Measurement Partnership

New multi-year deal integrates Roku's data to fuel Nielsen's measurement suite Roku gains access to Nielsen's streaming ratings, showing The Roku C...

22/12/2025

Allen Media Group to Deploy Infillion TrueX for Streaming Services

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

22/12/2025

Berklee Wrapped 2025: Our Top News and Stories

Berklee Wrapped 2025: Our Top News and Stories A look back at a year highlighted by faculty milestones, major film and television projects, Bob Dylan's ho...

22/12/2025

Marine Biological Laboratory Explores Human Memory With AI and Virtual Reality

The works of Plato state that when humans have an experience, some level of change occurs in their brain, which is powered by memory - specifically long-term me...

22/12/2025

Space42 and LatConnect 60 Expand Access to Advanced Geospatial Intelligence

Partnership integrates complementary satellite data and AI analytics to enhance security, infrastructure, and environmental monitoring solutions for global cust...

22/12/2025

Simplify Playlist Management with Workflows in WO Automation for Radio

Workflows allow you to create a sequence of planned events which may be added to your template(s) or inserted directly into your sequential or background playli...

22/12/2025

Sky extends PGA TOUR partnership until 2029, as Sky Sports remains the unrivalled home for golf fans in the UK and Ireland

Monday 22 December 2025 Sky extends PGA TOUR partnership until 2029, as Sky Spo...

22/12/2025

Global Anime Hits and New Releases Take Center Stage at Jump Festa 2026

Back to All News Global Anime Hits and New Releases Take Center Stage at Jump Festa 2026 Entertainment 22 December 2025 GlobalJapan Link copied to clipboar...

22/12/2025

Christmas with Oliver Callan on RT

Siobh n McSweeney, Rory McIlroy, Elon Musk, Catherine Connolly, Jim Gavin, Ivan Yates and Traitor Paudie Moloney lead new characters for Callan Kicks the Year 2...

22/12/2025

Monaghan's McKenna family crowned Ireland's Fittest Family 2025

Winner announced in the picturesque surroundings of Wicklow's Avondale Tower and Treetop Walk Andrew Trimble wins the show in his first series as coach Th...

22/12/2025

RT lyric fm Choirs for Christmas 2025 Winners Announced

The 2025 winners have been announced today, Sunday 21 December, for Ireland's largest choral competition Choirs for Christmas hosted by RT lyric fm. Ove...

21/12/2025

Legoshi and Haru's Story Reaches Its Finale: BEASTARS Final Season Part 2' Premieres March 2026, Main Trailer Out Now

Back to All News Legoshi and Haru's Story Reaches Its Finale: BEASTARS Fin...

21/12/2025

Rory McIlroy caps stellar year by winning the RT Sport Sportsperson of the Year 2025

John Shortt named Young Sportsperson of the Year Kerry are the Team of the Year ...

20/12/2025

Atomos Updates Ninja TX GO-Ninja TX With ProRes RAW and C...

Atomos announced the immediate availability of a new firmware update for its Ninja TX GO and Ninja TX monitor-recorders, unlocking ProRes RAW recording from the...

20/12/2025

CJP Broadcast Completes Digitisation of European Gymnasti...

CJP Broadcast has completed the digitisation of the European Gymnastics tape archive, converting 328 tapes containing more than forty years of recorded material...

20/12/2025

Bitmovin Launches Stream Lab MCP Server

Bitmovin, the leading provider of video streaming solutions, today announced the launch of the Stream Lab MCP Server, to give AI agents and large language model...

20/12/2025

Gracenote Unveils New Immersive Features for Sports Hubs

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/12/2025

Morgan Murphy Media Promotes Jill Shiroma to VP of Digital Strategy

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/12/2025

Samsung Makes GameBreaks Ad Format Available Programmatically

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/12/2025

FCC Extends Deadline for Comments on Upper C-Band Proposals

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/12/2025

Scripps Sports Inks Deals for Soccer and Professional Cheerleading

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/12/2025

Atomos Unveils Firmware Update For Ninja TX and TX GO

Share Share by: Copy link Facebook X Whatsapp Pinterest Flipboard...

20/12/2025

Barack Obama Includes Laufey on His 2025 Favorite Music List

Barack Obama Includes Laufey on His 2025 Favorite Music List The former presidents roundup of books, music, and movies includes a song from the Berklee alums ...

20/12/2025

December 19, 2025

Study reveals a key hormonal circuit in the kidneys Scripps Research scientists identify the protein that helps kidney cells regulate renin, providing foundatio...

19/12/2025

With Playout Release 2025.4, ToolsOnAir continues to push professional playout workflows forward on macOS.

With Playout Release 2025.4, ToolsOnAir continues to push professional playout w...

19/12/2025

SVG Sit-Down: Diversified's Jared Timmins on AI for Broadcast Sports and Creating the Smart Venue'

SVG Sit-Down: Diversified's Jared Timmins on AI for Broadcast Sports and Cre...

19/12/2025

2025 SVG Summit Audio Recap: Say What?

2025 SVG Summit Audio Recap: Say What?The Audio Production and Distribution Workshop at the SVG Summit 20 took on issues including speech intelligibility, Next-...

19/12/2025

Gamified Fun: Channel 5 on its NFL Big Game Night Ambitions with Hungry Bear Media

Gamified fun: Channel 5 on its NFL Big Game Night ambitions with Hungry Bear Med...

19/12/2025

College Football Playoff Preview: For ESPN, Round 1 is a Fantastic Yet Familiar Saturday of Production

College Football Playoff Preview: For ESPN, Round 1 is a Fantastic Yet Familia...

19/12/2025

AWS's Jason Dvorkin on Developing Partnerships With the NBA and PGA Tour, Embracing the Use of Agentic AI

AWS's Jason Dvorkin on Developing Partnerships With the NBA and PGA Tour, Em...

19/12/2025

Netflix Kicks Off Packed Sports Week with Paul-Joshua Fight Before Shifting to NFL Christmas Doubleheader

Netflix Kicks Off Packed Sports Week with Paul-Joshua Fight Before Shifting to N...

19/12/2025

SVG New Sponsor Spotlight: Presidio's Nareev Shah on the Role of Its Captivate and Resonate Platforms in Sports Production

SVG New Sponsor Spotlight: Presidio's Nareev Shah on the Role of Its Captiva...

19/12/2025

US Marine Corps Increases Affordable Mass Demonstrations with Successful Red Wolf Low-Altitude Live Fire

Mounted to the pylon of an AH-1Z Viper helicopter, a Red Wolf vehicle successful...