Content streaming and engagement are entering a new dimension with QUEEN, an AI model by NVIDIA Research and the University of Maryland that makes it possible to stream free-viewpoint video, which lets viewers experience a 3D scene from any angle.
QUEEN could be used to build immersive streaming applications that teach skills like cooking, put sports fans on the field to watch their favorite teams play from any angle, or bring an extra level of depth to video conferencing in the workplace. It could also be used in industrial environments to help teleoperate robots in a warehouse or a manufacturing plant.
The model will be presented at NeurIPS, the annual conference for AI research that begins Tuesday, Dec. 10, in Vancouver.
To stream free-viewpoint videos in near real time, we must simultaneously reconstruct and compress the 3D scene, said Shalini De Mello, director of research and a distinguished research scientist at NVIDIA. QUEEN balances factors including compression rate, visual quality, encoding time and rendering time to create an optimized pipeline that sets a new standard for visual quality and streamability.
Reduce, Reuse and Recycle for Efficient Streaming Free-viewpoint videos are typically created using video footage captured from different camera angles, like a multicamera film studio setup, a set of security cameras in a warehouse or a system of videoconferencing cameras in an office.
Prior AI methods for generating free-viewpoint videos either took too much memory for livestreaming or sacrificed visual quality for smaller file sizes. QUEEN balances both to deliver high-quality visuals - even in dynamic scenes featuring sparks, flames or furry animals - that can be easily transmitted from a host server to a client's device. It also renders visuals faster than previous methods, supporting streaming use cases.
In most real-world environments, many elements of a scene stay static. In a video, that means a large share of pixels don't change from one frame to another. To save computation time, QUEEN tracks and reuses renders of these static regions - focusing instead on reconstructing the content that changes over time.
Using an NVIDIA Tensor Core GPU, the researchers evaluated QUEEN's performance on several benchmarks and found the model outperformed state-of-the-art methods for online free-viewpoint video on a range of metrics. Given 2D videos of the same scene captured from different angles, it typically takes under five seconds of training time to render free-viewpoint videos at around 350 frames per second.
This combination of speed and visual quality can support media broadcasts of concerts and sports games by offering immersive virtual reality experiences or instant replays of key moments in a competition.
In warehouse settings, robot operators could use QUEEN to better gauge depth when maneuvering physical objects. And in a videoconferencing application - such as the 3D videoconferencing demo shown at SIGGRAPH and NVIDIA GTC - it could help presenters demonstrate tasks like cooking or origami while letting viewers pick the visual angle that best supports their learning.
The code for QUEEN will soon be released as open source and shared on the project page.
QUEEN is one of over 50 NVIDIA-authored NeurIPS posters and papers that feature groundbreaking AI research with potential applications in fields including simulation, robotics and healthcare. Among them is a best paper runner-up, titled Guiding a Diffusion Model With a Bad Version of Itself - one of just five papers recognized among the over 4,500 accepted this year.
Generative Adversarial Nets, the paper that first introduced GAN models, won the NeurIPS 2024 Test of Time Award. Cited more than 85,000 times, the paper was coauthored by Bing Xu, distinguished engineer at NVIDIA. Hear more from its lead author, Ian Goodfellow, research scientist at DeepMind, on the AI Podcast:
The AI Podcast Ep. 25: Google's Ian Goodfellow on How an Argument in a Bar Led to Generative Adversarial Networks
Learn more about NVIDIA Research at NeurIPS.
See the latest work from NVIDIA Research, which has hundreds of scientists and engineers worldwide, with teams focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.
Academic researchers working on large language models, simulation and modeling, edge AI and more can apply to the NVIDIA Academic Grant Program.
See notice regarding software product information.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
13/11/2026
The UIM E1 Series wraps its third season this week with the Bombay Sapphire E1 Miami GP, Nov. 11-14, on Biscayne Bay at the Miami Seaplane Base on Watson Island...
13/11/2026
In-venue and creative video staffers at the professional and collegiate level ha...
13/11/2026
Singer-songwriter Benson Boone will headline entertainment for the Discover NHL Winter Classic between the Utah Mammoth and Colorado Avalanche at Rice-Eccles St...
13/11/2026
Blackmagic Design has announced the Blackmagic Ambisonic Microphone, a 7th order ambisonic microphone array that captures spatial audio for immersive video prod...
13/11/2026
SMPTE has announced that The Studio B&H and Riedel Communications will provide the video infrastructure and communications technology for the Solutions Hub at t...
13/11/2026
The Hollywood Professional Association (HPA) has announced the theme and program focus for the 2027 HPA Tech Retreat, which takes place Feb. 14-18, 2027, in La ...
13/11/2026
Football Manager 27 Touch is available on Apple Arcade starting today. The game ...
13/11/2026
MotoGP has announced a two-year media rights agreement with ESPN covering the Un...
13/11/2026
Apple Sports, the free iPhone app, has added the Ultimate Fighting Championship (UFC) to its lineup of supported sports, in time for Saturday's UFC 334. The...
13/11/2026
The NBA and Fanatics Studios, the joint venture between Fanatics and entertainme...
13/11/2026
ESPN held its annual EDGE Innovation Conference last night and offered up an ove...
13/11/2026
Regional sports network gives Ithaca College, SUNY Cortland students a platform to thrive
One of the biggest games in Division III college football will be pla...
13/11/2026
Studer-inspired 500-series PWM compressor unveiled
The HLabs range already includes a couple of Studer-inspired offerings in the form of the PRE169 and EQ16...
13/11/2026
Download Now!
To view the white paper, please enter the following details:
--
First Name* First Name
Company*
Country* Please select your countryA...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Blue Lucy, the workflow orchestration and integration platform for broadcasters, rights owners and localisation companies, today announced an integration with c...
13/11/2026
Krotos announces Black Friday offers on Studio Pro and Studio Max
Brie Clayton November 12, 2026
0 Comments
First-year annual plans start at US$195.99...
13/11/2026
Blackmagic Design Announces Blackmagic Ambisonic Microphone
Brie Clayton November 12, 2026
0 Comments
New 7th order ambisonic microphone with 108 MEMS...
13/11/2026
Meet the Berklee Student Setting Planetary Motion to Music With his compositional platform SoniKosmos, Javier Roqueni aims to make the universe the ultimate i...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , er...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , er...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
13/11/2026
Amagi, the media industry cloud company, today announced a partnership with Golden Times Media Technology (stock code 920021.BJ), a Beijing-based media and tech...
13/11/2026
CGI (TSX: GIB.A) (NYSE: GIB), one of the largest independent IT and business consulting services firms in the world, together with its Malaysian partner, Cedar ...
13/11/2026
SMPTE, the home for media professionals, technologists and engineers, today announced that The Studio B&H and Riedel Communications will provide the video infra...
13/11/2026
iWedia, a global leader in connected TV software solutions, will present its software-first approach to DRAM optimization at Android TV APAC Summit 2026 in Bang...
13/11/2026
Beginning today, November 13, Apple HomeView is available for purchase at Apple Store locations, on the Apple Store online, and in the Apple Store app in select...
13/11/2026
From pediatric research to smart hospitals, AI is helping care teams spend less time on routine tasks and more time with patients.
Health systems and startups...
12/11/2026
Industry leaders examine how Dynamic Media Facility, Media eXchange Layer, and c...
12/11/2026
The junior Tar Heel is embracing the technical and creative challenges of live production while emerging as a leader and mentor in the control room...
12/11/2026
Gracenote, Nielsen's content intelligence business, has released its fifth a...
12/11/2026
RTL Deutschland will bring the NHL to free-to-air TV in Germany starting Nov. 30...
12/11/2026
Clear-Com supported On Air 2026, a 24-hour student-led broadcast connecting 45 u...
12/11/2026
uniqFEED has opened a Remote Operations Center (ROC) in Madrid, Spain. It is already operational, delivering virtual advertising for LaLiga matches. The ROC is ...
12/11/2026
Disney, ESPN and the NBA will present Dunk the Halls, the third real-time animated NBA game using Sony's Beyond Sports data visualization technology, on Chr...
12/11/2026
Van Wagner has launched Van Wagner RELAY, a digital platform for official athlet...
12/11/2026
Deal brings together two of the industry's most established graphics brands and significantly expands Vizrt's footprint in U.S. sports and regional broa...
12/11/2026
Euro TV Production GmbH of Austria has upgraded its outside broadcast vehicle with Lawo technology. The mc 36 Classic console in service since 2019 was replaced...
12/11/2026
Tagboard has launched Spark Resize, an AI-assisted tool now in beta that takes an existing graphic and produces a new version for a different aspect ratio or sc...
Content streaming and engagement are entering a new dimension with QUEEN, an AI model by NVIDIA Research and the University of Maryland that makes it possible to stream free-viewpoint video, which lets viewers experience a 3D scene from any angle.
