Sony Pixel Power calrec Sony

Moving Pictures: Transform Images Into 3D Scenes With NVIDIA Instant NeRF

17/04/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and which showcases new hardware, software, tools and accelerations for RTX PC users.

Imagine a gorgeous vista, like a cliff along the water's edge. Even as a 2D image, the scene would be beautiful and inviting. Now imagine exploring that same view in 3D - without needing to be there.

NVIDIA RTX technology-powered AI helps turn such an imagination into reality. Using Instant NeRF, creatives are transforming collections of still images into digital 3D scenes in just seconds.

Simply Radiant A NeRF, or neural radiance field, is an AI model that takes 2D images representing a scene as input and interpolates between them to render a complete 3D scene. The model operates as a neural network - a model that replicates how the brain is organized and is often used for tasks that require pattern recognition.

Using spatial location and volumetric rendering, a NeRF uses the camera pose from the images to render a 3D iteration of the scene. Traditionally, these models are computationally intensive, requiring large amounts of rendering power and time.

A recent NVIDIA AI research project changed that.

Instant NeRF takes NeRFs to the next level, using AI-accelerated inverse rendering to approximate how light behaves in the real world. It enables researchers to construct a 3D scene from 2D images taken at different angles. Scenes can now be generated in seconds, and the longer the NeRF model is trained, the more detailed the resulting 3D renders.

NVIDIA researchers released four neural graphics primitives, or pretrained datasets, as part of the Instant-NGP training toolset at the SIGGRAPH computer graphics conference in 2022. The tools let anyone create NeRFs with their own data. The researchers won a best paper award for the work, and TIME Magazine named Instant NeRF a best invention of 2022.

In addition to speeding rendering for NeRFs, Instant NeRF makes the entire image reconstruction process accessible using NVIDIA RTX and GeForce RTX desktop and laptop GPUs. While the time it takes to render a scene depends on factors like dataset size and the mix of image and video source content, the AI training doesn't require server-grade or cloud-based hardware.

NVIDIA RTX workstations and GeForce RTX PCs are ideally suited to meet the computational demands of rendering NeRFs. NVIDIA RTX and GeForce RTX GPUs feature Tensor Cores, dedicated AI hardware accelerators that provide the horsepower to run generative AI locally.

Ready, Set, Go Get started with Instant NeRF to learn about radiance fields and experience imagery in a new way.

Developers and tech enthusiasts can download the source-code base to compile. Nontechnical users can download the Windows installers for Instant-NGP software, available on GitHub.

While the installer is available for a wide range of RTX GPUs, the program performs best on the latest-architecture GeForce RTX 40 Series and NVIDIA RTX Ada Generation GPUs.

The Getting Started With Instant NeRF guide walks users through the process, including loading one of the primitives, such as NeRF Fox, to get a sense of what's possible. Detailed instructions and video walkthroughs - like the one above - demonstrate how to create NeRFs with custom data, including tips for capturing good input imagery and compiling codebases (if built from source). The guide also covers using the Instant NeRF graphical user interface, optimizing scene parameters and creating an animation from the scene.

The NeRF community also offers many tips and tricks to help users get started. For example, check out the livestream below and this technical blog post.

Show and Tell Digital artists are composing beautiful scenes and telling fresh stories with NVIDIA Instant NeRF. The Instant NeRF gallery showcases some of the most innovative and thought-provoking examples, viewable as video clips in any web browser.

Here are a few:

Through the Looking Glass by Karen X. Cheng and James Perlman A pianist practices her song, part of her daily routine, though there's nothing mundane about what happens next. The viewer peers into the mirror, a virtual world that can be observed but not traversed; it's unreachable by normal means. Then, crossing the threshold, it's revealed that this mirror is in fact a window into an inverted reality that can be explored from within. Which one is real?

https://blogs.nvidia.com/wp-content/uploads/2024/04/studio-ai-decoded-karenx-v9-1280x680-1.mp4

Meditation by Franc Lucent As soon as they walked into one of many rooms in Nico Santucci's estate, Lucent knew they needed to turn it into a NeRF. Playing with the dynamic range and reflections in the pond, it presented the artist with an unknown exploration. They were pleased with the softness of the light and the way the NeRF elevates the room into what looks like something out of a dream - the perfect place to meditate. A NeRF can freeze a moment in a way that's more immersive than a photo or video.

https://blogs.nvidia.com/wp-content/uploads/2024/04/studio-ai-decoded-franc-lucent-meditation-1280x680-1.mp4

Zeus by Hugues Bruy re These rendered 3D scenes with Instant NeRF use the data Bruy re previously captured for traditional photogrammetry using mirrorless digital cameras, smartphones, 360-degree cameras and drones. Instant NeRF gives him a powerful tool to help preserve and share cultural artifacts through online libraries, museums, virtual-reality experiences and heritage-conservation projects. This NeRF was trained using a dataset of photos taken with an iPhone at the Royal Ontario Museum.

https://blogs.nvidia.com/wp-content/uploads/2024/04/studio-ai-decoded-hugues-zeus-1280x680-1.mp4

From Image to Video to Reality Transforming images into a 3D scene with AI is cool. Stepp
LINK: https://blogs.nvidia.com/blog/ai-decoded-instant-nerf/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

11/03/2026

Techex Names Matt McKee as Senior Director of Sales, Americas

Share Copy link Facebook X Linkedin Bluesky Email...

11/03/2026

IAB Tech Lab Announces Content Monetization Protocol for AI LLMs

Share Copy link Facebook X Linkedin Bluesky Email...

11/03/2026

Mondae Hott Joins Kokusai Denki as Northeast Sales Manager

Share Copy link Facebook X Linkedin Bluesky Email...

11/03/2026

Gray Media to Air Cincinnati Reds' Games on WXIX FOX19

Share Copy link Facebook X Linkedin Bluesky Email...

11/03/2026

Shure Audio Solutions Deliver Super Bowl Win

Share Copy link Facebook X Linkedin Bluesky Email...

11/03/2026

UK's First Live Broadcast Using New n40 Private 5G Spectrum

Share Copy link Facebook X Linkedin Bluesky Email...

11/03/2026

Utah Scientific Expands Technology Partner Program With I...

Utah Scientific today announced the expansion of its Technology Partner Program with the addition of Audinate, Bitfocus, and Skaarhoj, three industry leaders wh...

11/03/2026

DigitalGlue Ends the Post Production Tax creativespace In...

DigitalGlue, creator of the creative.space on-premise managed storage platform, today revealed plans to launch creative.space Intelligence (CSI) at NAB 2026 (Bo...

11/03/2026

Maxon and Tencent Cloud Partner to Integrate HY 3D into C...

Maxon, maker of powerful, approachable software solutions for creators working in 2D and 3D design, motion graphics, visual effects, gaming, and more, has annou...

11/03/2026

NUGEN Audio Halo Vision Plug In Serves as Spatial Compass...

Composer and Re-recording Mixer Michael Phillips Keeley has built his career around immersive storytelling. Working from his Dolby Atmos-equipped studio, Sound ...

11/03/2026

YES selects Synamedia Iris to power advanced advertising

Leading video software provider Synamedia today announced that YES, the pay-TV subsidiary of the telco Bezeq (TASE: BEZQ), has selected Synamedia Iris to delive...

11/03/2026

Cost Savings Scalability and Smarter Monetization Viacces...

As media companies face increasing cost pressures and operational complexity, at the 2026 NAB Show in Las Vegas, Viaccess-Orca (VO), a global leader in OTT / TV...

11/03/2026

Digital Alert Systems Unveils Version 6 Software for DASD...

Digital Alert Systems, a global leader in emergency communications solutions for media providers, today announced the release of Version 6 software for its DASD...

11/03/2026

SES Brings Satellite Connectivity to Refugees in Chad

First Medium-Earth Orbit (MEO) deployment of the emergency.lu platform for refugees and their host communities' use provides dependable broadband for humani...

11/03/2026

Foundry releases Nuke 17.0

Foundry releases Nuke 17.0 Brie Clayton March 1, 2026 0 Comments Native Gaussian Splat support, new 3D system based on USD, expanded machine learning ca...

11/03/2026

Preserving UNESCO World Heritage with URSA Cine Immersive

Preserving UNESCO World Heritage with URSA Cine Immersive Brie Clayton March 1, 2026 0 Comments The Explorers turned to France's cultural landmark...

11/03/2026

I Clicked This By Accident And It Made After Effects SO Much Faster

I Clicked This By Accident And It Made After Effects SO Much Faster Graham Quince March 1, 2026 0 Comments Discover how Region of Interest in Adobe A...

11/03/2026

Cine Gear Connect Brings a Focused All-Day Experience to Industry City, NY

Cine Gear Connect Brings a Focused All-Day Experience to Industry City, NY Brie Clayton March 4, 2026 0 Comments Registration is now open for Cine Gea...

11/03/2026

La Vorgine Edited and Finished with DaVinci Resolve Studio

La Vor gine Edited and Finished with DaVinci Resolve Studio Brie Clayton March 4, 2026 0 Comments One of Colombia's most ambitious projects goes g...

11/03/2026

SoundMarket Launches 18,000+ Tracks of Real Music by Award-Winning Composers for Editors and Post Professionals

SoundMarket Launches 18,000 Tracks of Real Music by Award-Winning Composers for...

11/03/2026

Capta Center Supports NOVO19 Remote Production with Blackmagic Design

Capta Center Supports NOVO19 Remote Production with Blackmagic Design Brie Clayton March 5, 2026 0 Comments The facility provides production and playo...

11/03/2026

DigitalGlue Ends the Post-Production Tax: creative.space Intelligence (CSI) Unifies On-Premise Storage with Forensic AI at NAB 2026

DigitalGlue Ends the Post-Production Tax: creative.space Intelligence (CSI) Unif...

11/03/2026

Kochi Sun Sun Uses Blackmagic Replay for High School Volleyball Finals

Kochi Sun Sun Uses Blackmagic Replay for High School Volleyball Finals Brie Clayton March 9, 2026 0 Comments Versatile Blackmagic Replay system proves...

11/03/2026

Richard Bona Joins Berklee for Signature Series Concert

Richard Bona Joins Berklee for Signature Series Concert The Grammy-winning Cameroonian bassist and vocalist collaborates with students and faculty in a progra...

11/03/2026

March 10, 2026

Scripps Research scientists awarded nearly $5 million by NIH to investigate cancer growth Researchers will investigate how a common dietary nutrient may control...

11/03/2026

The 2026 RT Short Story Competition is now open for entries

TELL US YOUR (SHORT) STORIES The 2026 RT Short Story Competition is now open for entries Recognising and rewarding the best new Irish fiction writing for...

11/03/2026

Celtic Media Festival 2026 Nominees Announced

RT 's The Traitors Ireland is among the nominees for the 2026 Celtic Media Festival Torc Awards for Excellence, announced today....

10/03/2026

Harvey Arnold, Bert Goldman to Be Honored at the 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

10/03/2026

Senators Urge FCC to Preserve Citizens Broadband Radio Service

Share Copy link Facebook X Linkedin Bluesky Email...

10/03/2026

SCTE TechExpo26 Issues Call for Content, Technical Papers

Share Copy link Facebook X Linkedin Bluesky Email...

10/03/2026

Zefr Receives MRC Accreditation

Share Copy link Facebook X Linkedin Bluesky Email...

10/03/2026

Study: Overloaded Sports Fans Fed Up with Fragmented Viewing Options

Share Copy link Facebook X Linkedin Bluesky Email...

10/03/2026

NVIDIA and ComfyUI Streamline Local AI Video Generation for Game Developers and Creators at GDC

Game developers and artists are building cinematic worlds and iconic characters ...

10/03/2026

NVIDIA Virtualizes Game Development With RTX PRO Server

Game development teams are working across larger worlds, more complex pipelines and more distributed teams than ever. At the same time, many studios still rely ...

10/03/2026

As Open Models Spark AI Boom, NVIDIA Jetson Brings It to Life at the Edge

The Cat 306 CR mini-excavator weighs just under eight tons and fits inside a standard shipping container. It's the machine a contractor rents when the job s...

10/03/2026

NVIDIA and Thinking Machines Lab Announce Long-Term Gigawatt-Scale Strategic Partnership

NVIDIA and Thinking Machines Lab announced today a multiyear strategic partnersh...

09/03/2026

Foos Gone Wild, Combate Global Launch New Televised MMA Fight Series

Foos Gone Wild and Combate Global have teamed up to create a twist on combat sports competition, announcing the launch of a special amateur Mixed Martial Arts (...

09/03/2026

Harmonic Accelerates Streaming and Broadcast Transformations

At the 2026 NAB Show, Harmonic will introduce significant enhancements to its video appliances and SaaS solutions, highlighted by a next-generation media server...

09/03/2026

ESPN Delivers Most-Watched MLB Spring Training Game in 10 years with Team USA vs. San Francisco Giants

ESPN's March 3 spring training matchup between Team USA and the San Francisc...

09/03/2026

Most Valuable Promotions Launches Women's Boxing Platform, Signs Multi-Year Deal with ESPN

Most Valuable Promotions (MVP) announces the launch of MVPW, a new global platfo...

09/03/2026

Behind The Mic: CBS Sports and TNT Sports Share NCAA Division 1 Mens's Basketball Tournament Commentators

Behind The Mic provides a roundup of recent news regarding on-air talent, includ...

09/03/2026

SVG All-Stars: Jenna McKeon, Senior Director, Remote Technical Operations, CBS Sports

From Super Bowl compounds to Final Four setups, the Hofstra graduate helps coord...

09/03/2026

NBC's Paralympic Effort Embraces Cloud for Signal Transport

Stamford plays a key role, but a small team in Cortina and Milan powers local presence and mixed-zone coverage...

09/03/2026

Save the Date: SVG's New Cloud & Content Workflows Summit in NYC on July 28

The event brings together SVG's previous Cloud Production and Content Management Forums into a single, comprehensive day of programming...

09/03/2026

2,000-Year-Old Arena Hosts Paralympics Opening Ceremony, Olympics Closing Ceremony

Updated Mar 9, 2026 Live surround sound has been a part of the plan for Roman a...