Sony Pixel Power calrec Sony

World-Class: NVIDIA Research Builds AI Model to Populate Virtual Worlds With 3D Objects, Characters

23/09/2022

The massive virtual worlds created by growing numbers of companies and creators could be more easily populated with a diverse array of 3D buildings, vehicles, characters and more - thanks to a new AI model from NVIDIA Research.

Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. These 3D objects are created in the same format used by popular graphics software applications, allowing users to immediately import their shapes into 3D renderers and game engines for further editing.

The generated objects could be used in 3D representations of buildings, outdoor spaces or entire cities, designed for industries including gaming, robotics, architecture and social media.

GET3D can generate a virtually unlimited number of 3D shapes based on the data it's trained on. Like an artist who turns a lump of clay into a detailed sculpture, the model transforms numbers into complex 3D shapes.

With a training dataset of 2D car images, for example, it creates a collection of sedans, trucks, race cars and vans. When trained on animal images, it comes up with creatures such as foxes, rhinos, horses and bears. Given chairs, the model generates assorted swivel chairs, dining chairs and cozy recliners.

GET3D brings us a step closer to democratizing AI-powered 3D content creation, said Sanja Fidler, vice president of AI research at NVIDIA, who leads the Toronto-based AI lab that created the tool. Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and interesting objects.

GET3D is one of more than 20 NVIDIA-authored papers and workshops accepted to the NeurIPS AI conference, taking place in New Orleans and virtually, Nov. 26-Dec. 4.

It Takes AI Kinds to Make a Virtual World The real world is full of variety: streets are lined with unique buildings, with different vehicles whizzing by and diverse crowds passing through. Manually modeling a 3D virtual world that reflects this is incredibly time consuming, making it difficult to fill out a detailed digital environment.

Though quicker than manual methods, prior 3D generative AI models were limited in the level of detail they could produce. Even recent inverse rendering methods can only generate 3D objects based on 2D images taken from various angles, requiring developers to build one 3D shape at a time.

GET3D can instead churn out some 20 shapes a second when running inference on a single NVIDIA GPU - working like a generative adversarial network for 2D images, while generating 3D objects. The larger, more diverse the training dataset it's learned from, the more varied and detailed the output.

NVIDIA researchers trained GET3D on synthetic data consisting of 2D images of 3D shapes captured from different camera angles. It took the team just two days to train the model on around 1 million images using NVIDIA A100 Tensor Core GPUs.

Enabling Creators to Modify Shape, Texture, Material GET3D gets its name from its ability to Generate Explicit Textured 3D meshes - meaning that the shapes it creates are in the form of a triangle mesh, like a papier-m ch model, covered with a textured material. This lets users easily import the objects into game engines, 3D modelers and film renderers - and edit them.

Once creators export GET3D-generated shapes to a graphics application, they can apply realistic lighting effects as the object moves or rotates in a scene. By incorporating another AI tool from NVIDIA Research, StyleGAN-NADA, developers can use text prompts to add a specific style to an image, such as modifying a rendered car to become a burned car or a taxi, or turning a regular house into a haunted one.

The researchers note that a future version of GET3D could use camera pose estimation techniques to allow developers to train the model on real-world data instead of synthetic datasets. It could also be improved to support universal generation - meaning developers could train GET3D on all kinds of 3D shapes at once, rather than needing to train it on one object category at a time.

For the latest news from NVIDIA AI research, watch the replay of NVIDIA founder and CEO Jensen Huang's keynote address at GTC:
LINK: https://blogs.nvidia.com/blog/2022/09/23/3d-generative-ai-research-vir...
See more stories from nvidia

Most recent headlines

07/12/2022

Dalet Flex Takes a Leap Forward

Dalet, a leading technology and service provider for media-rich organizations, today announced key new capabilities available in Dalet Flex that support importa...

04/11/2022

Game better with DTS Headphone:X

The best gamers know DTS Headphone:X technology ups their play. Now major magazines have caught on, too. DTS Headphone:X technology creates immersive soundsc...

08/10/2022

Dalet Next-Gen Solutions at IBC 2022

Dalet, a leading technology and service provider for media-rich organizations, is pleased to announce its return to the IBC Show, held in Amsterdam from Septemb...

01/10/2022

BARDAS, Bain Capital to Develop $600M TV Studio in Hollywood

HOLLYWOOD & BOSTON A joint venture between the BARDAS Investment Group and Bain Capital Real Estate has announced plans for a $600 million redevelopment of the ...

01/10/2022

FCC Chairwoman Rosenworcel Provides Agency Update on Hurricane Ian

WASHINGTON Federal Communications Commission Chairwoman Jessica Rosenworcel provided an update today on the agency's efforts to support the operation and re...

01/10/2022

Alisa Bowen Named President Of Disney+

LOS ANGELES Alisa Bowen has been named president of Disney+, where she will report to Michael Paull, president, Direct to Consumer, Disney Media Entertainment &...

01/10/2022

Ms. Marvel Finished in DaVinci Resolve Studio

FREMONT, Calif. Blackmagic Design has announced that one of the latest installments in the Marvel Cinematic Universe (MCU), the Ms. Marvel series, was graded ...

01/10/2022

Ian Aftermath: 5 TV Stations, 21 Radio Stations Still Out of Service in Florida

WASHINGTON, D.C. As hurricane Ian continues on its path of devastation into South Carolina, the FCC is reporting that five TV stations and 21 ratio stations are...

01/10/2022

Johnny Weir, Tara Lipinski, Debbie Gibson, Donny Osmond To Guest on Criss Angels Magic With the Stars

Criss Angel's Magic With the Stars premieres Saturday, Oct. 22 on The CW. Ea...

01/10/2022

End of the Road for Bridge and Tunnel

The Edward Burns dramedy Bridge and Tunnel will not see season three on Epix. Season two premiered in July....

01/10/2022

Hurricane Ian Knocks Out Cable/Broadband to Half Million

Hurricane Ian has taken a heavy toll on Florida communications, according to the first reporting out of the Federal Communications Commission's voluntary Di...

01/10/2022

Comscore Restructures with Staff Cuts To Cost $5-8 Million in Severance

Measurement company Comscore said it is restructuring its operation, resulting in employee layoffs that will cost the company $6 million to $8 million in severa...

01/10/2022

TMZ, TMZ Live Renewed by Fox Television Stations

TMZ and TMZ Live have been renewed on the Fox Television Stations through the 2024-25 TV season, said Frank Cicha, executive VP of programming, Fox Television S...

01/10/2022

Say Goodbye Newsy and Hello To Scripps News on Jan. 1

E.W. Scripps said it is combining the national news assets of its local station group with its Newsy network to create Scripps News....

01/10/2022

Series Business: The Equalizer Showrunners Joseph C. Wilson and Adam Glass

Broadcasting + Cables Michael Malone speaks with Joseph C. Wilson and Adam Glass, showrunners on The Equalizer, about the upcoming season the CBS series....

01/10/2022

Bill Plante, Longtime CBS News Correspondent, Has Died

Bill Plante, a correspondent who spent 52 years at CBS News, died September 28 in Washington. He was 84 and the cause was respiratory failure....

01/10/2022

FCC Seeks Staffing Info on Standard General-Tegna

The Federal Communications Commission has asked Tegna and Standard General for yet more documents, with a particular focus on any potential layoffs or impact on...

01/10/2022

Best Ever Channels, Stand Group Bringing Funny to FAST with Witz-Comedy TV

Knock knock. Who's there? Witz-Comedy TV, a free, ad supported streaming TV channel being launched by Best Ever Channels and The Stand Group....

01/10/2022

Soapie teasers: What's happening in The River this October 2022?

Soapie teasers: What's happening in The River this October 2022?This month is dramatic in The River. by SA TV and Soapies 01-10-2022 08:23 in Soapies ...

01/10/2022

Scandal soapie teasers: What's happening this October 2022?

Scandal soapie teasers: What's happening this October 2022?This month is going to be intense in Scandal. by SA TV and Soapies 01-10-2022 08:25 in Scand...

01/10/2022

Soapie teasers: What's happening in Gomora on October 2022?

Soapie teasers: What's happening in Gomora on October 2022?Are you ready to be entertained this month? by SA TV and Soapies 01-10-2022 08:30 in Gomora ...

01/10/2022

Diep City soapie teasers: What's happening this October 2022?

Diep City soapie teasers: What's happening this October 2022?This is an exciting month in DiepCity. by SA TV and Soapies 01-10-2022 08:43 in Soapies D...

01/10/2022

The Queen soapie teasers: What's happening this October 2022?

The Queen soapie teasers: What's happening this October 2022?The Queen is full of twists and exciting drama this month. by SA TV and Soapies 01-10-2022 0...

01/10/2022

Skeem Saam Soapie teasers: What's happening in October 2022?

Skeem Saam Soapie teasers: What's happening in October 2022?Are you ready for another Skeem Saam rollercoaster ride this month? by SA TV and Soapies 01-1...

01/10/2022

Muvhango soapie teasers: What's happening this October 2022?

Muvhango soapie teasers: What's happening this October 2022?Muvhango is full of twists and exciting drama this month. by SA TV and Soapies 01-10-2022 08:...

01/10/2022

Uzalo soapie teasers: What's happening this October 2022?

Uzalo soapie teasers: What's happening this October 2022?Uzalo continues its fine form this month. by SA TV and Soapies 01-10-2022 08:06 in Soapies Uz...

01/10/2022

Soapie teasers: What's happening in Generations: The Legacy in October 2022?

Soapie teasers: What's happening in Generations: The Legacy in October 2022?What an entertaining month we have in store. by SA TV and Soapies 01-10-2022 ...

30/09/2022

From Almada to Irritu: Sundance Institute's Partnership with Mexican Storytellers Has Birthed Countless Cinematic Classics

By Stephanie Ornelas When it comes to Mexican cinema, Sundance Institute's...

30/09/2022

Celebrate Latinx Heritage Month With the Launch of Our New Playlist, De TODO

Taking place in the U.S. between September 15 and October 15 each year, Latinx Heritage Month honors the many contributions this unique and vibrant culture has ...

30/09/2022

This International Podcast Day, We're Dishing Out Trivia on Some of Our Favorite Shows

-- -- Today marks International Podcast Day, and as the world's most popu...

30/09/2022

Object Matrix partners with Pebble for efficient multi-channel playout

Object Matrix has partnered with Pebble, a leader in the highly specialized and mission-critical field of broadcast playout automation. The integrated solution...

30/09/2022

Hitachi to Make Hitachi Energy a Wholly Owned Subsidiary

Tokyo, September 30, 2022 --- Hitachi, Ltd. (TSE: 6501, Hitachi) today announced that it has signed a share transfer agreement with ABB Ltd (SIX: ABBN, ABB) to ...

30/09/2022

Bravo to a Flying Engineering Marvel and to the People Who Helped Imaginations Soar

The Stratospheric Observatory for Infrared Astronomy, or SOFIA, completed its fi...

30/09/2022

L3Harris Technologies Sets Date for Third Quarter Earnings Release

MELBOURNE, Fla. - L3Harris Technologies (NYSE:LHX) will release financial results for its third quarter of fiscal year 2022 on Thursday, October 27, 2022. The c...

30/09/2022

John Small Appointed Solutions Engineer at ASG

EMERYVILLE, CALIF. Advanced Systems Group has hired John Small as a solutions engineer. He will provide technical expertise on infrastructure and workflows to c...

30/09/2022

FCC Seeks More Tegna Deal Documents

WASHINGTON D.C. The FCC has asked for a wide range of new documents as part of its assessment of the $8 billion merger of Apollo Global Management, Standard Gen...

30/09/2022

Ms. Marvel Finished in DaVinci Resolve Studio

Ms. Marvel Finished in DaVinci Resolve Studio Brie Clayton September 30, 2022 0 Comments Blackmagic Design today announced that one of the latest insta...

30/09/2022

Pixotope Announces New EMEA Partners and Distributors

Pixotope Announces New EMEA Partners and Distributors Brie Clayton September 30, 2022 0 Comments Following a suite of new partnerships in the APAC regi...

30/09/2022

Digital Domain Combines Artistry and Machine Learning to De-Age Joe Montana for Quantum Leap' Trailer

Digital Domain Combines Artistry and Machine Learning to De-Age Joe Montana for ...

30/09/2022

Roku Expands Cross-Platform Measurement Capabilities With Nielsen

NEW YORK Nielsen has announced that it is working with Roku to enable four-screen measurement for the first time across traditional TV, connected TV, desktop, a...

30/09/2022

Newsy Rebranded as Scripps News'

CINCINNATI E.W. Scripps Company has announced that it is rebranding its Newsy free 24/7 OTA news channel to Scripps News, launching Jan. 1, 2023....

30/09/2022

BBC Studioworks officially opens new Kelvin Hall production hub

The company has invested in the latest technology for the hub, including HD facilities, and high-quality audio equipment By Jenny Priestley Published: Septemb...

30/09/2022

Meet the global head of customer success

Vizrts Vanessa Walmsley describes how shes inspired to work in the media industry by the people and their passion for the technology theyre creating By Contrib...

30/09/2022

Sky extends Formula 1 rights deal across all its markets, will offer multi-screen and second-screen options

Sky also said viewers will be able to continue to watch every race in Ultra HD v...

30/09/2022

FCC Extends Fee Filing Deadline to Friday, Sept. 30 Due to Hurricane Ian

WASHINGTON The FCC announced yesterday that due to Hurricane Ian, it was extending the Fiscal Year (FY) 2022 regulatory fee filing deadline from Wednesday, Sept...

30/09/2022

FCC Updates Emergency Alert Rules

WASHINGTON, D.C. The Federal Communications Commission (FCC) has updated its Emergency Alert System rules so that alerts delivered over television and radio are...

30/09/2022

Dish Tops J.D. Power National Pay TV Consumer Satisfaction Rankings

TROY, Mich. J.D. Power has released its annual consumer survey of consumer satisfaction with TV service providers and for the fifth year in a row, Dish received...

30/09/2022

FCC Seeks to Remove Obsolete Analog TV Rules

WASHINGTON With the cessation of LPTV analog broadcasts last year, the FCC is marking the official end of analog TV broadcasting in the U.S. by updating or outr...

30/09/2022

FCC: Ian Knocks Out 6 TV, 21 Radio Stations

WASHINGTON, D.C. In the wake of hurricane Ian, the FCC has issued a report (opens in new tab) showing widespread damage to media and communications outlets in F...

30/09/2022

NAB Show NY: Artel to Highlight IP Solutions

NEW YORK At the 2022 NAB Show New York, Oct. 19-20 at the Javits Center, Artel will feature products that simplify broadcasters shift to hybrid IP/SDI and all-I...