
Content streaming and engagement are entering a new dimension with QUEEN, an AI model by NVIDIA Research and the University of Maryland that makes it possible to stream free-viewpoint video, which lets viewers experience a 3D scene from any angle.
QUEEN could be used to build immersive streaming applications that teach skills like cooking, put sports fans on the field to watch their favorite teams play from any angle, or bring an extra level of depth to video conferencing in the workplace. It could also be used in industrial environments to help teleoperate robots in a warehouse or a manufacturing plant.
The model will be presented at NeurIPS, the annual conference for AI research that begins Tuesday, Dec. 10, in Vancouver.
To stream free-viewpoint videos in near real time, we must simultaneously reconstruct and compress the 3D scene, said Shalini De Mello, director of research and a distinguished research scientist at NVIDIA. QUEEN balances factors including compression rate, visual quality, encoding time and rendering time to create an optimized pipeline that sets a new standard for visual quality and streamability.
Reduce, Reuse and Recycle for Efficient Streaming Free-viewpoint videos are typically created using video footage captured from different camera angles, like a multicamera film studio setup, a set of security cameras in a warehouse or a system of videoconferencing cameras in an office.
Prior AI methods for generating free-viewpoint videos either took too much memory for livestreaming or sacrificed visual quality for smaller file sizes. QUEEN balances both to deliver high-quality visuals - even in dynamic scenes featuring sparks, flames or furry animals - that can be easily transmitted from a host server to a client's device. It also renders visuals faster than previous methods, supporting streaming use cases.
In most real-world environments, many elements of a scene stay static. In a video, that means a large share of pixels don't change from one frame to another. To save computation time, QUEEN tracks and reuses renders of these static regions - focusing instead on reconstructing the content that changes over time.
Using an NVIDIA Tensor Core GPU, the researchers evaluated QUEEN's performance on several benchmarks and found the model outperformed state-of-the-art methods for online free-viewpoint video on a range of metrics. Given 2D videos of the same scene captured from different angles, it typically takes under five seconds of training time to render free-viewpoint videos at around 350 frames per second.
This combination of speed and visual quality can support media broadcasts of concerts and sports games by offering immersive virtual reality experiences or instant replays of key moments in a competition.
In warehouse settings, robot operators could use QUEEN to better gauge depth when maneuvering physical objects. And in a videoconferencing application - such as the 3D videoconferencing demo shown at SIGGRAPH and NVIDIA GTC - it could help presenters demonstrate tasks like cooking or origami while letting viewers pick the visual angle that best supports their learning.
The code for QUEEN will soon be released as open source and shared on the project page.
QUEEN is one of over 50 NVIDIA-authored NeurIPS posters and papers that feature groundbreaking AI research with potential applications in fields including simulation, robotics and healthcare. Among them is a best paper runner-up, titled Guiding a Diffusion Model With a Bad Version of Itself - one of just five papers recognized among the over 4,500 accepted this year.
Generative Adversarial Nets, the paper that first introduced GAN models, won the NeurIPS 2024 Test of Time Award. Cited more than 85,000 times, the paper was coauthored by Bing Xu, distinguished engineer at NVIDIA. Hear more from its lead author, Ian Goodfellow, research scientist at DeepMind, on the AI Podcast:
The AI Podcast Ep. 25: Google's Ian Goodfellow on How an Argument in a Bar Led to Generative Adversarial Networks
Learn more about NVIDIA Research at NeurIPS.
See the latest work from NVIDIA Research, which has hundreds of scientists and engineers worldwide, with teams focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.
Academic researchers working on large language models, simulation and modeling, edge AI and more can apply to the NVIDIA Academic Grant Program.
See notice regarding software product information.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
07/07/2026
Applications are now open for the Validation Booster, a six-month acceleration programme supporting journalists, media organisations, content creators, media in...
07/07/2026
The Universal Talent Identifier (HAND) has issued HAND IDs to 50 top prospects ahead of the 2026 MLB Draft, including consensus top-ranked players Roch Cholowsk...
07/07/2026
Cisco and the McLaren Mastercard Formula 1 Team have announced an extension of their partnership, expanding the scope from networking and communications to incl...
07/07/2026
LucidLink has announced a workflow integration with Adobe Frame.io that gives production teams unified access to Frame.io projects alongside supporting producti...
07/07/2026
Chyron has announced the appointment of Ahmed Magd as Sales Director for the Middle East. Magd joins from previous senior commercial roles at Riedel Communicati...
07/07/2026
Leader Electronics of Europe has announced that its PHABRIX Qx rasterizer and Le...
07/07/2026
eCLUTCH has announced it has acquired broadcast rights to the Esports World Cup 2026 for EMEA and MENA, covering the Middle East, Africa, the Balkans, Sweden, a...
07/07/2026
Riedel Communications and ARRI supported ORF's host broadcast of the Eurovis...
07/07/2026
ARRI has announced the ALEXA 35 Live Xtreme, a camera that adds high frame rate ...
07/07/2026
LiveU will exhibit at IBC 2026 (Stand 7.C24), showcasing the LU900Q Intelligent Production Unit, expanded Sony collaboration, and additions to its open IP-video...
07/07/2026
Telemundo's coverage is packaged with Multiview, Catch Up with Key Plays, in...
07/07/2026
The Australian Rugby League Commission (ARLC) has announced a seven-year media r...
07/07/2026
Produced onsite at The Star in Frisco, TX, live coverage of the two-day event airs on ESPN platforms
The season for Football Media Days is here, and the Big 12...
07/07/2026
Rogers Communications Inc. has announced an agreement to acquire the remaining 25% ownership stake in Maple Leaf Sports and Entertainment (MLSE) from Kilmer Spo...
07/07/2026
End-to-end broadcast solution includes fleet of mobile units, Cloud Control, and...
07/07/2026
Sports-production-tech experts sound off on the state of the industry...
07/07/2026
New mixer aimed squarely at instrument-level sources
Heritage Audio have clearly been busy recently, as alongside their all-new Tubestrip, they've also ...
07/07/2026
New channel strip combines popular hardware designs
The latest arrival to the Heritage Audio line-up brings together a selection of their praised hardware d...
07/07/2026
eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({ content_source:......
07/07/2026
New Kessler 10 data now available in Nielsen CMI gives brands a deeper view of h...
07/07/2026
New Nielsen Ad Intel and CMV data reveals rising competition in Australia's ...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Riedel Communications and ARRI played a key role in one of the most technically ambitious Eurovision Song Contest productions to date. For the first time in the...
07/07/2026
Berklee Valencia Honors Eric Whitacre at the 2026 Commencement Ceremony The Grammy-winning composer was celebrated alongside this years cohort of masters degr...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
The appointment will drive growth and innovation across the Media Business Unit
Fincons Group, a leading IT business consultancy and systems integrator company...
07/07/2026
International broadcast and streaming expert Big Blue Marble has announced a multi-month Proof of Concept (PoC) for Media over QUIC (MoQ) during a major interna...
07/07/2026
Test & measurement innovator, Leader Electronics of Europe, is proud to announce that two of its leading products, the PHABRIX Qx rasterizer and LeaderPhabrix L...
07/07/2026
Clear-Com announced that Ross Production Services (RPS), a leading provider of mobile production solutions for live events and broadcasts, has standardized on ...
07/07/2026
LiveU redefined live newsgathering with the invention of cellular bonding. At IBC2026, Stand 7.C24, the company will show how that foundation is evolving into t...
07/07/2026
LucidLink, a leader in cloud-based file services for content creators, today announced an enhanced workflow integration between LucidLink and Adobe Frame.io tha...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/07/2026
New appointments to Screen Australia board 7 July 2026
The Albanese Labor Government has made new appointments to the Screen Australia board.
Dr Tania Chamber...
07/07/2026
The right team. The right time.' How The Killings at Parrish Station stuck t...
07/07/2026
Berklee College of Music Alum and Faculty Perform at Boston Public Library's...
07/07/2026
UKTV commission with BritBox and Sony Pictures Television and inspired by Deborah Cadbury's book, Chocolate Wars
The six-part series is produced by Fable P...
07/07/2026
Tuesday 7 July 2026
Record audiences tune into the ICC Women's T20 World Cup 2026 on Sky Sports
The ICC Women's T20 World Cup 2026 was the most watche...
07/07/2026
Tuesday 7 July 2026
Official trailer released for Fightland, a revenge drama fr...
07/07/2026
Wuppertal July 7, 2026
Riedel and ARRI Bring Cinematic Live Production to the Eurovision Song ContestRiedel Communications and ARRI played a key role in one o...