Sony Pixel Power calrec Sony

Moving Pictures: Transform Images Into 3D Scenes With NVIDIA Instant NeRF

17/04/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and which showcases new hardware, software, tools and accelerations for RTX PC users.

Imagine a gorgeous vista, like a cliff along the water's edge. Even as a 2D image, the scene would be beautiful and inviting. Now imagine exploring that same view in 3D - without needing to be there.

NVIDIA RTX technology-powered AI helps turn such an imagination into reality. Using Instant NeRF, creatives are transforming collections of still images into digital 3D scenes in just seconds.

Simply Radiant A NeRF, or neural radiance field, is an AI model that takes 2D images representing a scene as input and interpolates between them to render a complete 3D scene. The model operates as a neural network - a model that replicates how the brain is organized and is often used for tasks that require pattern recognition.

Using spatial location and volumetric rendering, a NeRF uses the camera pose from the images to render a 3D iteration of the scene. Traditionally, these models are computationally intensive, requiring large amounts of rendering power and time.

A recent NVIDIA AI research project changed that.

Instant NeRF takes NeRFs to the next level, using AI-accelerated inverse rendering to approximate how light behaves in the real world. It enables researchers to construct a 3D scene from 2D images taken at different angles. Scenes can now be generated in seconds, and the longer the NeRF model is trained, the more detailed the resulting 3D renders.

NVIDIA researchers released four neural graphics primitives, or pretrained datasets, as part of the Instant-NGP training toolset at the SIGGRAPH computer graphics conference in 2022. The tools let anyone create NeRFs with their own data. The researchers won a best paper award for the work, and TIME Magazine named Instant NeRF a best invention of 2022.

In addition to speeding rendering for NeRFs, Instant NeRF makes the entire image reconstruction process accessible using NVIDIA RTX and GeForce RTX desktop and laptop GPUs. While the time it takes to render a scene depends on factors like dataset size and the mix of image and video source content, the AI training doesn't require server-grade or cloud-based hardware.

NVIDIA RTX workstations and GeForce RTX PCs are ideally suited to meet the computational demands of rendering NeRFs. NVIDIA RTX and GeForce RTX GPUs feature Tensor Cores, dedicated AI hardware accelerators that provide the horsepower to run generative AI locally.

Ready, Set, Go Get started with Instant NeRF to learn about radiance fields and experience imagery in a new way.

Developers and tech enthusiasts can download the source-code base to compile. Nontechnical users can download the Windows installers for Instant-NGP software, available on GitHub.

While the installer is available for a wide range of RTX GPUs, the program performs best on the latest-architecture GeForce RTX 40 Series and NVIDIA RTX Ada Generation GPUs.

The Getting Started With Instant NeRF guide walks users through the process, including loading one of the primitives, such as NeRF Fox, to get a sense of what's possible. Detailed instructions and video walkthroughs - like the one above - demonstrate how to create NeRFs with custom data, including tips for capturing good input imagery and compiling codebases (if built from source). The guide also covers using the Instant NeRF graphical user interface, optimizing scene parameters and creating an animation from the scene.

The NeRF community also offers many tips and tricks to help users get started. For example, check out the livestream below and this technical blog post.

Show and Tell Digital artists are composing beautiful scenes and telling fresh stories with NVIDIA Instant NeRF. The Instant NeRF gallery showcases some of the most innovative and thought-provoking examples, viewable as video clips in any web browser.

Here are a few:

Through the Looking Glass by Karen X. Cheng and James Perlman A pianist practices her song, part of her daily routine, though there's nothing mundane about what happens next. The viewer peers into the mirror, a virtual world that can be observed but not traversed; it's unreachable by normal means. Then, crossing the threshold, it's revealed that this mirror is in fact a window into an inverted reality that can be explored from within. Which one is real?

https://blogs.nvidia.com/wp-content/uploads/2024/04/studio-ai-decoded-karenx-v9-1280x680-1.mp4

Meditation by Franc Lucent As soon as they walked into one of many rooms in Nico Santucci's estate, Lucent knew they needed to turn it into a NeRF. Playing with the dynamic range and reflections in the pond, it presented the artist with an unknown exploration. They were pleased with the softness of the light and the way the NeRF elevates the room into what looks like something out of a dream - the perfect place to meditate. A NeRF can freeze a moment in a way that's more immersive than a photo or video.

https://blogs.nvidia.com/wp-content/uploads/2024/04/studio-ai-decoded-franc-lucent-meditation-1280x680-1.mp4

Zeus by Hugues Bruy re These rendered 3D scenes with Instant NeRF use the data Bruy re previously captured for traditional photogrammetry using mirrorless digital cameras, smartphones, 360-degree cameras and drones. Instant NeRF gives him a powerful tool to help preserve and share cultural artifacts through online libraries, museums, virtual-reality experiences and heritage-conservation projects. This NeRF was trained using a dataset of photos taken with an iPhone at the Royal Ontario Museum.

https://blogs.nvidia.com/wp-content/uploads/2024/04/studio-ai-decoded-hugues-zeus-1280x680-1.mp4

From Image to Video to Reality Transforming images into a 3D scene with AI is cool. Stepp
LINK: https://blogs.nvidia.com/blog/ai-decoded-instant-nerf/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

13/08/2026

CMG Expands Local Sports Offering With Two NBA Deals

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

Olympic Broadcasting Services to receive IBC2026 Internat...

IBC today announced that its most prestigious honour, the International Honour for Excellence (IHFE), will be presented to Olympic Broadcasting Services (OBS), ...

13/08/2026

Dalet Demonstrates the Future of AI-Orchestrated Live Spo...

Dalet, a leading technology and service provider for media-rich organizations, today announced its participation in the IBC2026 Accelerator Media Innovation Pro...

13/08/2026

dzjinius to showcase new dzjinius on-set application at I...

dzjinius, the all-in-one production management platform for the media and entertainment industry, will showcase its new application, dzjinius on-set , at IBC20...

13/08/2026

BCE Demonstrates Sovereign Cloud Media Workflows powered...

Broadcasting Center Europe (BCE), a leading European provider of media services, system integration and product development, will demonstrate an end-to-end sove...

13/08/2026

Clear-Com to Showcase Communications Solutions for Broadc...

Clear-Com is returning to IBC 2026, taking place September 11-14 in Amsterdam, Netherlands. At Stand 10.D29, Clear-Com will spotlight the award-winning FreeSpe...

13/08/2026

Leader to highlight advanced Test and Measurement solutio...

Test & measurement innovator, Leader Electronics of Europe, will showcase its comprehensive portfolio of Leader, PHABRIX and LeaderPhabrix test and measurement ...

13/08/2026

KRK Brings Advanced Wireless Control to Professional Monitoring With New V Series Five Studio MonitorsKRK Brings Advanced Wireless

KRK Brings Advanced Wireless Control to Professional Monitoring With New V Serie...

13/08/2026

Anthony Noack Captures Artist's Journey in Estonia with Blackmagic Design

Anthony Noack Captures Artist's Journey in Estonia with Blackmagic Design Brie Clayton August 12, 2026 0 Comments Solo filmmaker documents concept...

13/08/2026

ARRI Image Science makes its mobile debut in the HONOR Robot Phone

ARRI Image Science makes its mobile debut in the HONOR Robot Phone Brie Clayton August 12, 2026 0 Comments Hero image displays the HONOR Robot Phone, ...

13/08/2026

ASG Adds Eran Binyamin Zeitoun as Senior Technical Program Manager

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

Vinten To Feature Manual, Robotic Camera Support Offerings At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

SMPTE Unveils IBC 2026 Plans

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

IP Showcase on the Water Announces Schedule for IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

Spectrum, Optimum Ink New Deals for News Carriage and Ads

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

IAB Tech Lab Updates Privacy Standards

Share Copy link Facebook X Linkedin Bluesky Email...

13/08/2026

Suitest Highlights New AI-Powered Device Testing and Qual...

Suitest's Test Automation Platform Simplifies Video Quality Assurance, Monitoring, and Device Certification Across the OTT Ecosystem At IBC2026, Suitest, a...

13/08/2026

Cinegy showcases powerful, agile, cost-effective media so...

Cinegy GmbH, the premier provider of software-defined television technology, will have its key software solutions ready to demonstrate at SET Expo 2026 (18 20...

13/08/2026

Scripps Research establishes endowed chair honoring Peter K. Vogt, influential pioneer in cancer research

Scripps Research has established the Peter K. Vogt Endowed Chair, a new faculty ...

13/08/2026

New research offers fresh insights for how to make better vaccines

LA JOLLA, CA-A vaccine's molecular structure plays a key role in determining how the immune system responds and prepares to protect the body from future exp...

13/08/2026

Event Marks New Season with the RT Concert Orchestra

RT 's Miriam O'Callaghan welcomed a range of specially invited guests to the National Concert Hall last week for a unique event to mark a new season wit...

13/08/2026

Seattle Seahawks Become First Team to Bring Dolby OptiView with Dolby Vision to Fans

August 13 2026, 06:00 (PDT) Seattle Seahawks Become First Team to Bring Dolby O...

13/08/2026

Class Is in Session: GeForce NOW Levels Up Linux, Chromebooks and More

GeForce NOW is giving cloud gaming an extra-credit upgrade just in time for back-to-school season. The native Linux app for GeForce NOW is officially out of be...

12/08/2026

Op-Ed: The Versioning Playbook - How Rights Holders are Getting More From Live Events

Traditional live sports production has focused on creating the best possible pro...

12/08/2026

IBC 2026: FOR-A to Demonstrate Production and Processing Solutions

FOR-A will showcase a range of broadcast and production products at IBC2026 (Stand 2.B53, Amsterdam RAI, September 11-14), including software-based switching, A...

12/08/2026

IBC 2026: AMWA and EBU to Demo NMOS BCP-008 Multi-Vendor IP Status Reporting

The Advanced Media Workflow Association (AMWA) and EBU will host live demonstrations of NMOS BCP-008 Minimum Status Reporting at IBC2026 (September 11-14) on th...

12/08/2026

IBC 2026: Dalet Launches Solution Suites for Media Production and Archive Management

Dalet has announced Dalet Solution Suites, a portfolio of cloud-native solutions...

12/08/2026

Net Insight Enables Centralized Remote Production of Blinkfestivalen 2026 for NRK

Net Insight has announced that DMC Production Norway has selected its Open Live ...

12/08/2026

Zixi Appoints Alyson Stewart as Senior Director of Sales

Zixi has appointed Alyson Stewart as Senior Director of Sales. Stewart will be responsible for direct sales and partnerships, with a focus on broadcasters, medi...

12/08/2026

LiveU Powers Host Broadcast Operations at 2026 Glasgow Games

LiveU's LU900Q intelligent production unit was deployed as part of the host broadcast operation for the Games in Glasgow (July 23-August 2, 2026). Host broa...

12/08/2026

Disney+ and ESPN Acquire Multi-Year Formula E Rights Beginning 2026/27 Season

Disney , ESPN, and Formula E have announced a multi-year deal bringing all Formula E races to Disney and ESPN in the U.S. and to Disney in most international...

12/08/2026

PSSI Launches X-LIVE Managed Service for Live Event Transmission

PSSI has launched X-LIVE, a managed service offering dedicated engineering resources and Appear encoding infrastructure for live event signal processing and tra...

12/08/2026

MoonPay X Games League Draws 18.3 Million Viewers in Inaugural Summer Season

The inaugural MoonPay X Games League Summer Season reached 18.3 million unique live viewers across three stops on ABC and ESPN, a 78.5% increase compared to the...

12/08/2026

IBC 2026: BBright Adds RIST Support to IP-Gateway

BBright will showcase an expanded IP-Gateway at IBC2026, adding support for all RIST profiles to its existing support for MPEG Transport Stream over IP with SMP...

12/08/2026

IBC 2026: BBright Adds EBU OGraf Graphics Support to UHD and ST 2110 Playout Platform

BBright will demonstrate EBU OGraf animated graphics support in its software-bas...

12/08/2026

Interra Systems to Showcase QC and Monitoring Solutions at SET Expo 2026

Interra Systems will demonstrate its ORION, BATON, BATON Captions, and VEGA solutions at SET Expo 2026 (August 17-20, Distrito Anhembi, S o Paulo) at partner Li...

12/08/2026

DPA Microphones Appoints Raul Contreras as West Coast Regional Sales Manager

DPA Microphones has appointed Raul Contreras as West Coast Regional Sales Manager. In the role, Contreras will oversee sales and customer relationships througho...

12/08/2026

Save the Date: SVG LIVE! Returns to LA Nov. 10 to Spotlight Sports & Entertainment Production

Live spectacles ranging from the Super Bowl and FIFA World Cup to the Oscars and...

12/08/2026

Love, Separation, and the Cost of Comfort in How to Divorce During the War

Writer-Director Andrius Bla evi ius and crew attend the premiere of How to Divorce During the War at The Ray Theatre on January 26, 2026, in Park City, UT. (P...

12/08/2026

SSL 360 gets a major update

Enhanced DAW & plug-in control for the UC1, UF8 and UF1 The software that powers SSL's range of control surfaces has just received a major update that d...

12/08/2026

Triton Audio launch the Tilt EQ

In-line preamp features one-knob EQ In addition to their range of in-line preamps, Triton Audio recently managed to squeeze a compressor into an enclosure t...

12/08/2026

AudioRoute from Arvital Audio

Captures system audio with or without a DAW Depending on your setup, capturing system audio in a DAW can be a tedious process, so much so that Arvital Audio...

12/08/2026

2026 NATIONAL NAIDOC AWARDS TO BROADCAST ON NITV LIVE AND DIRECT FROM MPARNTWE

2026 NATIONAL NAIDOC AWARDS TO BROADCAST ON NITV LIVE AND DIRECT FROM MPARNTWE 12 August, 2026 Media releases First Nations excellence will be celebrated l...

12/08/2026

More than six million sports fans stay with SBS for Tour de France and Tour de France Femmes Avec Zwift

More than six million sports fans stay with SBS for Tour de France and Tour de F...

12/08/2026

Nielsen Introduces Groundbreaking Streaming Ad Measurement in Brazil

Nielsen is cementing Digital Ad Intel as the new go-to currency for digital ad intelligence in Brazil-now with streaming monitoring in the mix. S o Paulo, Augu...

12/08/2026

LPTV Advocates Petition FCC for Voluntary 5G Broadcasts

Share Copy link Facebook X Linkedin Bluesky Email...

12/08/2026

Tubi Media Group Inks Content Discovery Deal with Gracenote

Share Copy link Facebook X Linkedin Bluesky Email...