Blending real and virtual elements on set
11/02/2015
Originally published by The Broadcast Bridge
Gerhard Lang, chief engineering officer, Vizrt explains: When we're talking about augmented reality we are, for the most part, referencing tracked virtual elements. They're arranged three-dimensionally and attached to a live video signal. These are outputted as a ready-composite of live video and graphics. Virtual reality is an older term that refers to live motion in a three dimensional world. This doesn't necessarily have any live video in it.
The term virtual studio (or virtual set), refers to a three-dimensional space where elements - like a presenter - are keyed on top. Camera motion inside the virtual studio is created using tracking data from a live studio camera creating a realistic environment. The tracking data from the camera and the rendering of the graphics are composited and outputted as a final image. You can still put keyed graphics (for example, AR graphics) on top of this image so you can have additional graphics as part of the final image.
Lang says AR graphics are more complicated to work with compared to normal real-time 3D graphics. Users and operators need to understand not only the physics of what is going on but also the potential and limitations of the technology, he says.
The human eye is highly sensitive to irregularities in any video content and picks them up immediately, he explains. This ruins the illusion. To get AR right, we must minimise any errors introduced by technology. One of the most important elements is studio camera tracking, which reports the position and the viewing direction of every studio camera, as well as the zoom and focus of each lens. This tracking data must be accurate to the smallest detail and delivered with as little delay as possible with constant attention paid to timing behaviour.
An AR virtual camera must match its real camera as exactly as possible, including things like image deformation when using a specific wide-angle lens and depth of field. The position and rotation of the camera are, of course, local. The camera's coordinate space must match the one used to build the scene. Put simply, an AR element which is meant to sit on a real table wouldn't be there if the coordinate system's data doesn't match.
The virtual elements must fit naturally with the real set with regards to colour, contrast, light and shadows - even if the element is something which does not exist in real life, outlines Lang. It's distracting to the human eye if the shadows cast by virtual 3D objects are different to the ones cast by the real objects in the space, or if an object appears a lot brighter than the real-life studio lights would allow.
What are some of the best approaches to designing with AR in mind?Lang: Focus on getting the idea right. Content producers need to ensure they know that the AR they're putting into their work will enhance the content and make the production stand out. And involve everyone. Producers, designers and virtual set wizards should all have a say in the process.
Secondly, limit your idea to something your team is able to produce. Make sure that your AR objects, texture videos and so on will run in real time on your Viz Engine. Think carefully through the production set-up so you know what cameras and shots will be needed during production and how you'll cut between them live. Stick with real live measurements and avoid scaling. If possible have a mock-up of the real environment available inside Viz Artist to check how your elements will work without the need to have a camera and tracking available for planning.
When does AR / VR not work?Lang: Everything in this area now works so well that there aren't really any limits to the use of AR anymore. With highly portable tracking systems, like ncam, AR is now possible everywhere. If the hardware has already been assembled and calibrated, on-site setup time is really short. AR enhancements can now be used in a live broadcast within a few minutes. AR systems from Vizrt consisting of a laptop and Thunderbolt expansion box are lightweight and easy to transport anywhere. Because there are no more limitations, technically-speaking, the question producers need to ask themselves is whether AR makes sense and is appropriate for their particular project.
How do you best prepare talent to work with AR/VR?Lang: It's fair to say that some talent is better than others when working with AR or VR. Those who are better can visualise for themselves how things will look. They often have a better timing which helps when understanding how to work with virtual elements. Having enough monitors to show the final picture makes it easier for everyone, not just the talent. But most importantly, there should be plenty of rehearsal and time for talent to study plans of the content. Moving with an AR element or within a VR environment should be as natural as moving with real things in a real environment.
Vizrt virtual set at Sky Sports
How important is lens calibration?Lang: It is crucial for getting virtual and real elements to match up at all zoom levels. It's a job that just has to be done and at Vizrt we have made some nice tools to simplify this process. These tools also cut down the time needed for calibrating lenses. At Vizrt we keep the lens and camera calibration separate. That means that once you've calibrated a lens, its data can be easily recalled when being mounted on a different camera body. A little bit of fine tuning and you are
LINK: | http://www.vizrt.com/news/newsgrid/44001/Blending_real_and_virtual_ele... |
See more stories from vizrt |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
16/05/2024
Power Provokes Vital Questions About the Role of Police
PARK CITY, UTAH - JANUARY 18: Director Yance Ford introduces the Power premiere at Library Center Theatre. (Photo by Chad Salvador/Shutterstock for Sundance F...
16/05/2024
Mental Health Matters: Embrace Self-Care With These Audiobook and Podcast Listens
May is Mental Health Awareness month, a time to recognize those living with ment...
16/05/2024
ABC and SBS bring digital radio (DAB+) services to the Gold Coast
ABC and SBS bring digital radio (DAB ) services to the Gold Coast 15 May, 2024 Media releases ABC and SBS today announced that audiences on the Gold Coast ...
16/05/2024
Clear-Com Communication Solutions Enhance Oregon State University's PRAx Building
eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...
16/05/2024
6 New Clearmountain's Domain Presets
Share this article width=32 height=32 data-lazy-src=https://apogeedigital.com/app/themes/juniper-theme/src/img/icons/facebook-purple.svg /> width=32 height=...
16/05/2024
Updated: Netflix to Stream Live Christmas Day NFL Games
In a major example of how large streaming services are successfully battling for lucrative sports rights, Netflix has announced that it has secured global right...
16/05/2024
ESPN, Fox, WBD Unveil Name for New Sports Streaming Service
LOS ANGELES & NEW YORK More than three months after announcing plans to launch a major new sports streaming service, ESPN, Fox Corp. and Warner Bros. Discovery,...
16/05/2024
CTV/Streaming Top of Mind for Marketers in 2024
In a survey conducted late last year, marketing companies said their main focus for increasing advertising would be CTV/streaming. Generative AI, which was not ...
16/05/2024
CPB Issues More Grants for Next-Generation Warning
Three public radio and one public television outlet are the latest recipients of funding from the Corporation for Public Broadcasting to upgrade their emergency...
16/05/2024
A year in live HDR production
John Mailhot, senior vice president, product management at Imagine Communications, gives an insight into how HDR is impacting live production today, as well as ...
16/05/2024
Sony Pictures Post Production Services Completes Expansion of its Sound Facilities
Sony Pictures Post Production Services Completes Expansion of its Sound Faciliti...
16/05/2024
Telestream Taps Power of AI to Manage, Define Media Workflows
TV Tech recently sat down with Telestream CEO Rhonda Bassett-Spiers and CTO Simon Clark to discuss how the M&E industry is adapting the cloud, IP and AI to enha...
16/05/2024
1185 Films launches one stop shop' for industry talent
1185 Talent will include directors, editors, colourists, VFX artists and photographers By Jenny Priestley Published: May 16, 2024 1185 Talent will include...
16/05/2024
Updated: Presteigne Broadcast Hire enters administration
The administrators said they are focused on trading the business as a going concern while they assess options, including exploring the possibility of a sale By...
16/05/2024
AWS CEO Adam Selipsky steps down
Selipsky will leave the company on 3rd June, and will be succeeded by Matt Garman, a senior vice president who has overseen sales and marketing at AWS By Jenny...
16/05/2024
Sony Pictures Post Production Services Completes Sound Facilities Expansion
Sony Pictures Post Production Services has announced that the final phase of a multi-year effort to expand and modernize its motion picture and television sound...
16/05/2024
CNN, ABC to Televise Two Presidential Debates
President Joe Biden and Former President Donald Trump have agreed to participate in two televised debates, the first of which will be hosted by CNN on June 17....
16/05/2024
Disney+ Debuts First Concert Film to Stream with IMAX Enhanced Sound
BURBANK, Calif. and NEW YORK Disney+, IMAX Corporation, and DTS, a wholly-owned subsidiary of Xperi, have announced that Queen Rock Montreal is now streaming ...
16/05/2024
WBD Unveils Advanced Advertising Features for Max
NEW YORK During its upfronts presentation, Warner Bros. Discovery announced a suite of new advanced ad products available on Max, that include shoppable ads, ad...
16/05/2024
iZotope announces RX 11 audio software, introducing new machine learning for tackling noise, and new tools for music streaming
iZotope announces RX 11 audio software, introducing new machine learning for tac...
16/05/2024
Check Out the Cine Gear Expo June 7 9, 2024 at Warner Bros. Studios in Burbank, CA
Check Out the Cine Gear Expo June 7 9, 2024 at Warner Bros. Studios in Burbank...
16/05/2024
Nouvel Photo & Film Captures Cinematic Wedding Films with Blackmagic Cinema Camera 6K
Nouvel Photo & Film Captures Cinematic Wedding Films with Blackmagic Cinema Came...
16/05/2024
Replacing Surfaces Like a Pro with Mocha AE Planar Tracking
Replacing Surfaces Like a Pro with Mocha AE Planar Tracking Levi Borland May 15, 2024 0 Comments Welcome to our comprehensive tutorial on planar track...
16/05/2024
CBC's CPL Teams Salamanders, Yard Gnomes One Week Away From Season Opener
The world of baseball at Capitol Broadcasting is about to expand like fireworks on a Friday night. The Holly Springs Salamanders will open their season at home...
16/05/2024
Blockgraph Adopted by tvbeat, Enabling Audience-Based Planning
Blockgraph said that its identity and data collaboration platform has been adopted by tvbeat, a provider of campaign planning technology for media companies....
16/05/2024
VEON 1Q24 Trading Update: Accelerating USD Growth, Strong Operational Execution
16 May 2024 VEON 1Q24 Trading Update: Accelerating USD Growth, Strong Operational Execution Amsterdam, 16 May 2024 07:00AM CEST - VEON Q1 2024 Highlights Q1...
16/05/2024
Tribeca X Announces 2024 Speaker Lineup for Expanded Two-Day Event; Official Selections for Tribeca X Award
May 16th, 2024 Press Materials Available Here TRIBECA X ANNOUNCES 2024 SPEAKER...
16/05/2024
SVG Sit-Down: Beam Dynamics CEO Kaszycki on Maximizing Physical-Asset Usage, Tracking
SVG Sit-Down: Beam Dynamics CEO Kaszycki on Maximizing Physical-Asset Usage, Tra...
16/05/2024
SVG New Sponsor Spotlight: Aggreko's Don Gray on Advocating for Sustainability in Live Sports Production
SVG New Sponsor Spotlight: Aggreko's Don Gray on Advocating for Sustainabili...
16/05/2024
At PGA Championship, CBS Sports Bolsters Aerial, Bunker Camera Lineup; Takes Viewers Deeper with Optimotion Swing Tech
At PGA Championship, CBS Sports Bolsters Aerial, Bunker Camera Lineup; Takes Vie...
16/05/2024
Rohde & Schwarz shows the future of terrestrial broadcasting at CABSAT 2024
Rohde & Schwarz shows the future of terrestrial broadcasting at CABSAT 2024 Real innovations in services and sustainability Meet us at CABSAT 2024 in Dubai...
16/05/2024
Hierarchy' Teaser Unveils Jooshin High School's Elite World, Starring Next Generation of K-Teen Stars
Back to All News Hierarchy' Teaser Unveils Jooshin High School's Elite...
16/05/2024
Steamy Doctor Climax' Trailer Teases the Questions That Stirred Up 70s Thailand
Back to All News Steamy Doctor Climax' Trailer Teases the Questions That S...
16/05/2024
Getting moving for Mental Health Awareness Week
Posted on 16 May 2024 We're huge advocates of positively improving our mental wellbeing so going green for Mental Health Awareness Week was an obvious ward...
16/05/2024
Venu Sports Introduced as Name for Forthcoming Sports Streaming Service
Venu Sports Introduced as Name for Forthcoming Sports Streaming Service Details of ESPN, Fox and Warner Bros. Discovery Joint Venture Take Shape with Brand an...
16/05/2024
Irma Stern's 1930 portrait Cape Girl with Fruit headlines SA art auction
Irma Stern's 1930 portrait Cape Girl with Fruit headlines SA art auctionAn Irma Stern painting from 1930 leads Strauss & Co's diverse catalogue of moder...
16/05/2024
Prison Journalism: The role my mother played in my life
Prison Journalism: The role my mother played in my lifeWesley Leong was incarcerated at the age of 15 in 1996 at Pollsmoor Prison. He is currently part of Resto...
16/05/2024
Actress Sibongokuhle Cele joins Muvhango'
Actress Sibongokuhle Cele joins Muvhango'Model and former Durban Gen' actress Sibongokuhle Cele has secured a role in SABC2's soapie Muvhango'...
16/05/2024
Tonight on Skeem Saam: Carol sees Ntswaki getting up to no good with Toby
Tonight on Skeem Saam: Carol sees Ntswaki getting up to no good with TobyDon't miss Thursday, 16 May's riveting episode of South African soapie Skeem Sa...
16/05/2024
Fight for Honor in Men of War II' on GFN Thursday
Whether looking for new adventures, epic storylines or games to play with a friend, GeForce NOW members are covered. Start off with the much-anticipated sequel...
16/05/2024
Skeem Saam: Wednesday's episode, 15 May 2024 [video]
Skeem Saam: Wednesday's episode, 15 May 2024 [video]Missed an episode of Skeem Saam? No problem! Watch the latest episode of your favourite South African so...
16/05/2024
Monet painting fetches R640 million at New York auction
Monet painting fetches R640 million at New York auctionA Monet has sold for nearly R640 million at auction, Sotheby's said, marking a solid start to New Yor...
16/05/2024
Cruinni na ng 2024 - a day of free creative activity for young people
RT is supporting Cruinni na n g with a nationwide campaign on television, radio, online and digital. For a full listing of all activities and information on h...
15/05/2024
UKTV Ventures partners with The Farewell Guide in 1million airtime-for-equity deal
12th March 2024 - Award-winning broadcaster UKTV has today announced it has inve...
15/05/2024
Give Me the Backstory: Get to Know Jianjie Lin, the Writer-Director Behind Brief History of a Family
By Bailey Pennick One of the most exciting things about the Sundance Film Festi...
15/05/2024
De viktigaste insikterna frn Spotifys rliga Loud & Clear-rapport
Med sin rliga royaltyrapport Loud & Clear kar Spotify transparensen i musikindustrin, s att artister, l tskrivare, fans och hela musikbranschen f r en st rre...
15/05/2024
Chegaram as ltimas estatsticas: Artistas brasileiros geraram mais receita e alcanaram mais fs no Spotify - basta perguntar Anitta
Nos ltimos 10 a 15 anos, a mudan a para o streaming de udio transformou comple...
15/05/2024
The Latest Stats Are In: Brazilian Artists Generated More Revenue and Reached More Fans on Spotify-Just Ask Anitta
Over the last 10-15 years, the move toward streaming audio has completely transf...