
Content streaming and engagement are entering a new dimension with QUEEN, an AI model by NVIDIA Research and the University of Maryland that makes it possible to stream free-viewpoint video, which lets viewers experience a 3D scene from any angle.
QUEEN could be used to build immersive streaming applications that teach skills like cooking, put sports fans on the field to watch their favorite teams play from any angle, or bring an extra level of depth to video conferencing in the workplace. It could also be used in industrial environments to help teleoperate robots in a warehouse or a manufacturing plant.
The model will be presented at NeurIPS, the annual conference for AI research that begins Tuesday, Dec. 10, in Vancouver.
To stream free-viewpoint videos in near real time, we must simultaneously reconstruct and compress the 3D scene, said Shalini De Mello, director of research and a distinguished research scientist at NVIDIA. QUEEN balances factors including compression rate, visual quality, encoding time and rendering time to create an optimized pipeline that sets a new standard for visual quality and streamability.
Reduce, Reuse and Recycle for Efficient Streaming Free-viewpoint videos are typically created using video footage captured from different camera angles, like a multicamera film studio setup, a set of security cameras in a warehouse or a system of videoconferencing cameras in an office.
Prior AI methods for generating free-viewpoint videos either took too much memory for livestreaming or sacrificed visual quality for smaller file sizes. QUEEN balances both to deliver high-quality visuals - even in dynamic scenes featuring sparks, flames or furry animals - that can be easily transmitted from a host server to a client's device. It also renders visuals faster than previous methods, supporting streaming use cases.
In most real-world environments, many elements of a scene stay static. In a video, that means a large share of pixels don't change from one frame to another. To save computation time, QUEEN tracks and reuses renders of these static regions - focusing instead on reconstructing the content that changes over time.
Using an NVIDIA Tensor Core GPU, the researchers evaluated QUEEN's performance on several benchmarks and found the model outperformed state-of-the-art methods for online free-viewpoint video on a range of metrics. Given 2D videos of the same scene captured from different angles, it typically takes under five seconds of training time to render free-viewpoint videos at around 350 frames per second.
This combination of speed and visual quality can support media broadcasts of concerts and sports games by offering immersive virtual reality experiences or instant replays of key moments in a competition.
In warehouse settings, robot operators could use QUEEN to better gauge depth when maneuvering physical objects. And in a videoconferencing application - such as the 3D videoconferencing demo shown at SIGGRAPH and NVIDIA GTC - it could help presenters demonstrate tasks like cooking or origami while letting viewers pick the visual angle that best supports their learning.
The code for QUEEN will soon be released as open source and shared on the project page.
QUEEN is one of over 50 NVIDIA-authored NeurIPS posters and papers that feature groundbreaking AI research with potential applications in fields including simulation, robotics and healthcare. Among them is a best paper runner-up, titled Guiding a Diffusion Model With a Bad Version of Itself - one of just five papers recognized among the over 4,500 accepted this year.
Generative Adversarial Nets, the paper that first introduced GAN models, won the NeurIPS 2024 Test of Time Award. Cited more than 85,000 times, the paper was coauthored by Bing Xu, distinguished engineer at NVIDIA. Hear more from its lead author, Ian Goodfellow, research scientist at DeepMind, on the AI Podcast:
The AI Podcast Ep. 25: Google's Ian Goodfellow on How an Argument in a Bar Led to Generative Adversarial Networks
Learn more about NVIDIA Research at NeurIPS.
See the latest work from NVIDIA Research, which has hundreds of scientists and engineers worldwide, with teams focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.
Academic researchers working on large language models, simulation and modeling, edge AI and more can apply to the NVIDIA Academic Grant Program.
See notice regarding software product information.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/09/2026
Lightware is expanding its LARA (Lightware Advanced Room Automation) platform with the GUDE Power Distribution Units (PDU) driver module. GUDE devices can now b...
04/09/2026
Lightware shares the second release in its ESG communications series. Following the first piece, which focused on the companys environmental performance, this r...
04/09/2026
Big Blue Marble supported a leading European public service broadcaster in delivering the Soccer World Cup 2026 streams, supplying more than 10 Tbps of bandwidt...
04/09/2026
Outstanding achievement in engineering, science and technology
Hitomi Broadcast has been recognised with an Engineering, Science & Technology Emmy from the Ac...
04/09/2026
California, Hollywood at a Crossroads
Andy Marken September 3, 2026
0 Comments
I'm doing everything alone. Crossing roads alone. Seeing the Eiffe...
04/09/2026
Berklee Announces Winners of 2026 SongwritersdB Songwriting Contest Students Luca Alexandru, Annika Wild, and Johanna James received the top prizes for origin...
04/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/09/2026
RT ANNOUNCE NEW DRAMA THE MURDER OF MR. MOONLIGHT
BASED ON THE DISAPPEARANCE...
03/09/2026
Yospace will be at IBC from 11th - 14th September (booth 5.C78). For more inform...
03/09/2026
NEP Group has announced leadership changes at Creative Technology Group. Founder and CEO Graham Andrews will step away from day-to-day leadership at the end of ...
03/09/2026
The NFL and TikTok have announced a multi-year partnership renewal expanding the league's presence on the platform. NFL posts on TikTok increased more than ...
03/09/2026
Grass Valley has appointed Ronny Van Geel as Vice President, Cameras. Van Geel began his career in 1999 at Philips DVS and moved into product management in 2005...
03/09/2026
Telestream has announced Vantage Super Resolution, an AI-powered upscaling capability for the Vantage media workflow platform, developed in collaboration with N...
03/09/2026
Solid State Logic (SSL) will debut the TCA Tour at IBC2026 (Hall 8, Stand 8.B45), a portable production system built from System T components in a desktop frame...
03/09/2026
DPA Microphones has announced the 6388 CORE Headset Microphone, making its debut at IBC2026 (Hall 8, Stand D60). The supercardioid headset features a physicall...
03/09/2026
Qvest will present a multi-vendor Dynamic Media Facility (DMF) showcase at IBC2026 (Booth 10.C24), demonstrating how the EBU's DMF architecture can be imple...
03/09/2026
Imagine Communications will showcase the Magellan PanelFlex control surface and other control portfolio updates at IBC2026 (Stand 1.B73, RAI Amsterdam, Septembe...
03/09/2026
Bridge Technologies will announce at IBC2026 (Booth 1.A71) that its VB440 production probe is now available as a software-based solution, deployable on commerci...
03/09/2026
Audio-Technica has unveiled the BP350ST MS (mid-side) microphone in two versions: the camera-mount BP350ST-UL and the boundary or gooseneck-mountable BP350ST-UB...
03/09/2026
Telos Alliance will introduce updates to the J nger Audio flexAI audio processin...
03/09/2026
DUNKMAN, the professional dunk league founded by Shaquille O'Neal in partner...
03/09/2026
Orlando's WRDQ TV 27, WFTV Channel 9 previously named over-the-air broadcast...
03/09/2026
EverPass Media has announced a multi-year commercial distribution agreement with Netflix, expanding on an existing partnership that has delivered NFL Christmas ...
03/09/2026
Yahoo Sports and Apple TV have announced a partnership to deliver alternate broadcasts of select Friday Night Baseball and MLS games on the Yahoo Sports website...
03/09/2026
With another academic year underway, college athletic departments across the cou...
03/09/2026
Facility becomes permanent home of NFL Live on Sept. 9; centerpiece studio for c...
03/09/2026
Create musical synth bass sounds from scratch
Anthony Marinelli Music have recently announced the launch of Bass DNA, a new online course that teaches music...
03/09/2026
Most powerful Tonex hardware to date
IK Multimedia's latest release introduces their most advanced Tonex hardware unit to date, combining their AI Machi...
03/09/2026
Upmix & downmix software upgraded
Penteo Audio Plugins have announced that the latest version of their powerful upmix and downmix software is now available....
03/09/2026
With the start of the new apprenticeship year, young talent has once again begun their professional careers at SGL Carbon's sites in Germany. Recently, the ...
03/09/2026
With the industry increasingly embracing hybrid, software-defined production models, Calrec will preview a series of Argo and ImPulseV software enhancements at ...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/09/2026
Recognizing the ever-increasing demands for media throughput in challenging environments like post-production, two leaders in network technology have worked tog...
03/09/2026
EVS, a leader in live video technology and production infrastructure solutions, and Matrox Video, a leader in software-defined media technology and IP-based vid...
03/09/2026
AJA Debuts OG-FiDO-4 openGear Four-Channel SDI/Fiber Conversion Cards
Brie Clayton September 3, 2026
0 Comments
New product family expands 12G-SDI and...
03/09/2026
Nice Touch launches V2 of its AI edit assistant for professional editors
Brie Clayton September 3, 2026
0 Comments
Built by two editors with twelve ye...
03/09/2026
Accedo, a global provider of video streaming software and services, and MediaKind, a world-leading video technology company and the number one independent provi...
03/09/2026
International research from Big Blue Marble puts a consumer churn number against a technical failure, and sets out what streaming providers must fix before 2030...
03/09/2026
High-performance points of presence power video streaming delivery to large audiences across Latin America with guaranteed quality of experience
Broadpeak , a ...
03/09/2026
Dalet, a leading technology and service provider for media-rich organizations, will unveil the next evolution of Dalia at IBC2026 (7.A43), its media-aware agent...