Sony Pixel Power calrec Sony

Knight Rider Rides a GAN: Bringing KITT to Life with AI, NVIDIA Omniverse

16/04/2021

Fasten your seatbelts. NVIDIA Research is revving up a new deep learning engine that creates 3D object models from standard 2D images - and can bring iconic cars like the Knight Rider's AI-powered KITT to life - in NVIDIA Omniverse.

Developed by the NVIDIA AI Research Lab in Toronto, the GANverse3D application inflates flat images into realistic 3D models that can be visualized and controlled in virtual environments. This capability could help architects, creators, game developers and designers easily add new objects to their mockups without needing expertise in 3D modeling, or a large budget to spend on renderings.

A single photo of a car, for example, could be turned into a 3D model that can drive around a virtual scene, complete with realistic headlights, tail lights and blinkers.

To generate a dataset for training, the researchers harnessed a generative adversarial network, or GAN, to synthesize images depicting the same object from multiple viewpoints - like a photographer who walks around a parked vehicle, taking shots from different angles. These multi-view images were plugged into a rendering framework for inverse graphics, the process of inferring 3D mesh models from 2D images.

Once trained on multi-view images, GANverse3D needs only a single 2D image to predict a 3D mesh model. This model can be used with a 3D neural renderer that gives developers control to customize objects and swap out backgrounds.

When imported as an extension in the NVIDIA Omniverse platform and run on NVIDIA RTX GPUs, GANverse3D can be used to recreate any 2D image into 3D - like the beloved crime-fighting car KITT, from the popular 1980s Knight Rider TV show.

Previous models for inverse graphics have relied on 3D shapes as training data.

Instead, with no aid from 3D assets, We turned a GAN model into a very efficient data generator so we can create 3D objects from any 2D image on the web, said Wenzheng Chen, research scientist at NVIDIA and lead author on the project.

Because we trained on real images instead of the typical pipeline, which relies on synthetic data, the AI model generalizes better to real-world applications, said NVIDIA researcher Jun Gao, an author on the project.

The research behind GANverse3D will be presented at two upcoming conferences: the International Conference on Learning Representations in May, and the Conference on Computer Vision and Pattern Recognition, in June.

From Flat Tire to Racing KITT Creators in gaming, architecture and design rely on virtual environments like the NVIDIA Omniverse simulation and collaboration platform to test out new ideas and visualize prototypes before creating their final products. With Omniverse Connectors, developers can use their preferred 3D applications in Omniverse to simulate complex virtual worlds with real-time ray tracing.

But not every creator has the time and resources to create 3D models of every object they sketch. The cost of capturing the number of multi-view images necessary to render a showroom's worth of cars, or a street's worth of buildings, can be prohibitive.

That's where a trained GANverse3D application can be used to convert standard images of a car, a building or even a horse into a 3D figure that can be customized and animated in Omniverse.

To recreate KITT, the researchers simply fed the trained model an image of the car, letting GANverse3D predict a corresponding 3D textured mesh, as well as different parts of the vehicle such as wheels and headlights. They then used NVIDIA Omniverse Kit and NVIDIA PhysX tools to convert the predicted texture into high-quality materials that give KITT a more realistic look and feel, and placed it in a dynamic driving sequence.

Omniverse allows researchers to bring exciting, cutting-edge research directly to creators and end users, said Jean-Francois Lafleche, deep learning engineer at NVIDIA. Offering GANverse3D as an extension in Omniverse will help artists create richer virtual worlds for game development, city planning or even training new machine learning models.

GANs Power a Dimensional Shift Because real-world datasets that capture the same object from different angles are rare, most AI tools that convert images from 2D to 3D are trained using synthetic 3D datasets like ShapeNet.

To obtain multi-view images from real-world data - like images of cars available publicly on the web - the NVIDIA researchers instead turned to a GAN model, manipulating its neural network layers to turn it into a data generator.

The team found that opening the first four layers of the neural network and freezing the remaining 12 caused the GAN to render images of the same object from different viewpoints.

Keeping the first four layers frozen and the other 12 layers variable caused the neural network to generate different images from the same viewpoint. By manually assigning standard viewpoints, with vehicles pictured at a specific elevation and camera distance, the researchers could rapidly generate a multi-view dataset from individual 2D images.

The final model, trained on 55,000 car images generated by the GAN, outperformed an inverse graphics network trained on the popular Pascal3D dataset.

Read the full ICLR paper, authored by Wenzheng Chen, fellow NVIDIA researchers Jun Gao and Huan Ling, Sanja Fidler, director of NVIDIA's Toronto research lab, University of Waterloo student Yuxuan Zhang, Stanford student Yinan Zhang and MIT professor Antonio Torralba. Additional collaborators on the CVPR paper include Jean-Francois Lafleche, NVIDIA researcher Kangxue Yin and Adela Barriuso.

The NVIDIA Research team consists of more than 200 scientists around the globe, focusing on areas such as AI, computer vision, self-driving cars, robotics and graphics. Learn more about the company's latest research and industry breakthroughs in NVIDIA CEO Jensen Huang's k
LINK: https://blogs.nvidia.com/blog/2021/04/16/gan-research-knight-rider-ai-...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

31/03/2026

Boland Communications Introduces New QD-OLED Series Monitors

Share Copy link Facebook X Linkedin Bluesky Email...

31/03/2026

Carr Warns NFL Over Streaming Rights, Consumer Costs

Share Copy link Facebook X Linkedin Bluesky Email...

31/03/2026

Farhan Khan Appointed FCC Chief Information Officer

Share Copy link Facebook X Linkedin Bluesky Email...

31/03/2026

Federal Judge Pauses Nexstar/Tegna Merger

Share Copy link Facebook X Linkedin Bluesky Email...

31/03/2026

Boris FX Acquires Vegas Pro, Sound Forge, and Acid Pro

Boris FX Acquires Vegas Pro, Sound Forge, and Acid Pro Jessie Electa Petrov March 30, 2026 0 Comments The acclaimed VFX developer bolsters its award-w...

31/03/2026

Berklee India Exchange Presents Anirudh Varma Collective

Berklee India Exchange Presents Anirudh Varma Collective The collective will present a workshop on Hindustani classical music and a concert in collaboration w...

30/03/2026

NAB 2026: Manifold to Demonstrate 400GbE COTS FPGA Support

Manifold Technologies, a Germany-based provider of cloud infrastructure for live broadcast production, will demonstrate support for 400GbE COTS FPGA accelerator...

30/03/2026

NAB 2026: Boland Communications Introduces QD-OLED Series Monitors

Boland Communications will introduce its QD4K315HDR10, a 31.5-inch QD-OLED monitor, at NAB Show 2026 (Booth C3519, April 18-22). The company is also introducing...

30/03/2026

NAB 2026: PTZOptics to Showcase Move 4K and Horizon Platform

PTZOptics will demonstrate its Move 4K PTZ cameras and Horizon web-based control platform at NAB Show 2026 (Booth N1902). Move 4K with Horizon is now available...

30/03/2026

NAB 2026: Net Insight to Showcase Updated Nimbra Edge

Net Insight will demonstrate the next version of Nimbra Edge, its orchestration and control layer for live media services across multi-domain environments, at N...

30/03/2026

NAB 2026: Appear to Showcase Live Production Processing

Appear ASA will exhibit at NAB Show 2026 (Booth W1531, April 19-22, Las Vegas). The company completed an IPO in November 2025. Our customer-first approach is ...

30/03/2026

NAB 2026: Harmonic Announces New Live Sports Streaming Capabilities

Harmonic has announced new capabilities for its sports streaming platform, covering multiview, programmatic advertising, in-stream advertising, and content wate...

30/03/2026

NAB 2026: Ateme to Showcase GenAI, Agentic AI, and Streaming

Ateme (Booth W1723) will demonstrate broadcast, streaming, and AI-driven media workflow solutions at NAB Show 2026. GenAI and Agentic AI Ateme will demonstrat...

30/03/2026

NAB 2026: Bitmovin's Player Web X Adds Advertising Support, Vertical Video, and Proprietary ABR Algorithm

Bitmovin has announced new capabilities for Player Web X, its web video player, ...

30/03/2026

NAB 2026: Brazil's Minister of Communications and FCC Commissioner To Speak

The 2026 NAB Show (April 18-22, exhibits April 19-22, Las Vegas Convention Center) will host Brazil's Minister of Communications, Frederico de Siqueira Filh...

30/03/2026

NAB 2026: EVS To Showcase Expanded Live Production Ecosystem

EVS will exhibit at NAB Show 2026 (Booth N1841), highlighting new products and updates across its live production portfolio, including the debut of T-Motion med...

30/03/2026

NAB 2026: Solid State Logic To Demonstrate Expanded Virtual System T Platform

Solid State Logic will demonstrate its virtualized System T platform at NAB Show 2026 (Booth C6907). Demonstrations will include the VTE1 virtual DSP engine, ne...

30/03/2026

NAB 2026: Globecast To Showcase Managed Media Services Approach

Globecast will exhibit at NAB Show 2026 (Booth W3335), highlighting its hybrid service model spanning satellite, IP, fiber, and cloud. The company will demonst...

30/03/2026

NAB 2026: IP Showcase Returns as IPMX Moves to Deployment

The Alliance for IP Media Solutions (AIMS), Advanced Media Workflow Association (AMWA), and the Video Services Forum (VSF) have announced that the IP Showcase w...

30/03/2026

NAB 2026: BBright To Demonstrate Single-Stream ST 2110 Playout

At NAB Show 2026 BBright will present a demonstration of its One Stream for the World concept, showing how a single ST 2110 playout stream can simultaneously ...

30/03/2026

NAB 2026: OpenDrives To Demonstrate New Storage and Edge Products

OpenDrives will demonstrate new products at NAB Show 2026, with two locations in the West Hall: a pod (W3443-E) in the Sports Business Hub and a cabana at W1158...

30/03/2026

Behind the Mic: Amazon Prime Hosts 90th Master Tournament With Host Terry Gannon

Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...

30/03/2026

Op-Ed: Preparing for Agentic AI in Live Sports

The economics of live sports streaming have changed. New rights models, cloud production tools, and lower-cost distribution have made it possible for high schoo...

30/03/2026

Movimento Strings from Sonora Cinematic

MPE-capable chamber strings library announced Alongside their collection of Kontakt instruments, Sonora Cinematic have been steadily introducing a series of...

30/03/2026

UJAM release Groovemate Latigo

Latin-inspired percussion instrument announced Built on a newly developed engine and interface, UJAM's latest instrument has been designed to create Lat...

30/03/2026

Best Service launch Desert Winds

Latest Eduardo Tarilonte collaboration announced The latest library to join Best Service's ever-growing range includes four solo wind instruments that c...

30/03/2026

SOS Music Creators Survey 2026

We want to hear from you! Complete our SOS Quick Survey and enter the prize draw for a chance to win one of three $50 Amazon vouchers! Sound On Sound carri...

30/03/2026

Government of Canada Selects MAS for Strategic Tanker Fleet Sustainment

CC-330 Husky. 2024 Eric Desbiens Photography. Used with permission for the announcement and related communications. No residual rights....

30/03/2026

L3Harris Included in MDA Space Solution for RCN ISTAR Program

L3Harris Technologies will provide WESCAM CMX -8 sensor systems for integration on new Uncrewed Aircraft Systems from MDA Space, enhancing the Royal Canadian Na...

30/03/2026

EVS to Debut T-Motion Robotics at 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

30/03/2026

SDVI To Feature New Rally Media Supply Chain Management Enhancements

Share Copy link Facebook X Linkedin Bluesky Email...

30/03/2026

Boland Communications Introduces QD4K315HDR10 QD-OLED Series Monitors

Share Copy link Facebook X Linkedin Bluesky Email...

30/03/2026

Mileto Tecnologia accelerates streaming growth with Synam...

Synamedia today announced that Mileto Tecnologia, one of Brazil's largest pay-TV operators, has chosen the Synamedia Go platform to support its rapid OTT ex...

30/03/2026

FOR-A's Software-Defined, AI-Powered Development Advances with Nippon TV and NVIDIA Technology

FOR-A's Software-Defined, AI-Powered Development Advances with Nippon TV and...

30/03/2026

Give Your Astrophotography REAL Depth - After Effects Tutorial

Give Your Astrophotography REAL Depth - After Effects Tutorial Graham Quince March 30, 2026 0 Comments In this tutorial, I talk you through the full w...

30/03/2026

Alfalite returns to NAB Show alongside FOR-A, showcasing LED solutions for broadcast and mission-critical environments

Alfalite returns to NAB Show alongside FOR-A, showcasing LED solutions for broad...

30/03/2026

WideOrbit Announces New Name, New Features for Flagship Radio Automation Software

Introducing WO Aurora WideOrbit is pleased to introduce WO Aurora, a new name fo...

30/03/2026

Sky announces changes to its Diversity Advisory Council

Sky welcomes Karen Blackett CBE to its DAC and thanks Baroness Prashar and Ndidi Okezie as they step down after five yearsMonday 30 March 2026 Sky announces ch...

30/03/2026

Netflix Announces the Reunion for Love is Blind: Sweden Season 3 - Premiering April 2

Back to All News Netflix Announces the Reunion for Love is Blind: Sweden Season...

30/03/2026

Netflix unveils new images from the second season of 'Gangs of Galicia'

Back to All News Netflix unveils new images from the second season of Gangs of Galicia Entertainment 30 March 2026 GlobalSpain Link copied to clipboard Do...

30/03/2026

The Latest on Netflix Anime, Unveiled at AnimeJapan 2026

Back to All News The Latest on Netflix Anime, Unveiled at AnimeJapan 2026 Entertainment 30 March 2026 GlobalJapan Link copied to clipboard From romance an...

30/03/2026

KBRO Leverages Harmonic's Fiber-on-Demand Solution for Network Upgrades

Leading Taiwan Broadband Operator Drives Fiber Deeper with Harmonic SAN JOSE, Calif. - March 30, 2026 - Harmonic (NASDAQ: HLIT) today announced that KBRO, a lea...

30/03/2026

Top 10 Reasons Government Meetings Need Transcriptions (and Why It Matters More Than Ever)

Tyngsboro, Mass., March 30, 2026 - City councils, county commissions, school boa...

29/03/2026

Victory+ Turns to Creator Economy, Bringing In Popular Women's Sports Influencer Coach Jackie J to Host Live NWSL Alt-Cast

Cloud-based production, real-time engagement, and creator-driven storytelling ai...

28/03/2026

Harrison launch LiveTrax 3

Now features DiGiCo console integration Harrison's live recording and virtual soundcheck software has just reached its third major version, which among ...