Getting up close and personal with a digital Liam Payne

TVBEurope talks to the team at The Round, an augmented reality app which will feature an exclusive performance by Liam Payne ahead of Sundays EE British Academy Film Awards

By Jenny Priestley

Published: April 8, 2021

TVBEurope talks to the team at The Round, an augmented reality app which will feature an exclusive performance by Liam Payne ahead of Sunday's EE British Academy Film Awards

target=_blank title=Share on LinkedIn class=share-linkedin> TVBEurope talks to the team at The Round, an augmented reality app which will feature an exclusive performance by Liam Payne ahead of Sunday's EE British Academy Film Awards

https://www.tvbeurope.com/production-post/getting-up-close-and-personal-with-a-digital-liam-payne title=Share via Email class=share-email>

On Sunday night Liam Payne will open the British Academy Film Awards alongside his digital doppelganger.

But first, fans will be able to see Liam's avatar in a world-first real-time augmented reality mobile experience, created using EE's 5G network. Liam's performance will be recorded live and beamed to fans in the UK on their mobile phones wherever they are via AR app The Round.

Pip Brignall and Thomas Winsor, founders of The Round, developed the app as a way to enable live event producers and creatives to build and distribute their productions using augmented reality, and they describe it as a venue for live holographic performances.

Our background is in theatre, so our in' for immersive tech was very much from a theatre perspective, and we started to explore how virtual reality and augmented reality could be used to enhance that live performative storytelling format, Brignall tells TVBEurope. When we first started working on The Round it really came out of Covid shutting down all the theatres. We realised that a lot of the obstacles that we were facing could be solved by creating this one platform. We wanted to explore ways that dramatic storytelling could evolve into the audience's personal space. They're inviting the drama to them rather than going to it.

Pip Brignall (left) and Thomas Winsor (photo courtesy Anand Vara at V A Photography) The duo were taking part in Digital Catapult's 5G Accelerator Programme, experimenting on how they could utilise that technology to improve the platform and develop it further when the marketing team at EE found them. 5G is a huge part of EE's immediate and future roadmap and they always try to do a big exciting campaign around the BAFTAs and they were looking for something that would be a world first. They got in touch and asked us what would be possible, explains Brignall.

It was quite funny. They saw the video of our prototype and emailed saying they wanted to do that, and we said well that's what we're already doing', adds Winsor.

EE brought Payne on board, and the project developed from there. The AR version of the singer will be kept under wraps until 6.45pm on Sunday evening, but the app is available to download now with an AR holding animation that viewers can use to help them familiarise themselves with the process. Payne will perform exclusively for The Round at 6.45pm on Sunday before taking to the stage alongside his avatar to open the BAFTAs.

The avatar itself has already been created using photogrammetry. We captured Liam's image from every single angle you could imagine in a huge rig, and took lots of photos of his face and then we've turned that into the avatar, explains Winsor.

The images were then turned into a mesh, which was converted into an avatar and touched up in Maya. We're using really innovative facial capture technology as well so when we were doing the photogrammetry shoot we got Liam to recreate the face shapes that he makes when he's singing, says Brignall. When we were creating his avatar we had ready-made facial positions that we know are similar to what he does, so we then pre-programmed these blend shapes which were from an iPhone just strapped to his head. It tells the avatar that Liam is now doing blend shape number six and the face moves into blend shape number six. It has a series of blend shapes depending on what he's doing. That is all transmitted in real time so that once we've rigged the avatar it moves and behaves in the ways that Liam himself does, his digital face is programmed to make the same facial positions. When it's in real time that information makes the avatar come to life.

A slightly enhanced version of the avatar is being used in the TV performance. There are some limitations at this point as to what level of fidelity you can put an AR avatar into a mobile phone because they have a hard ceiling on what they can process. So we've created an improved' version of the avatar that people will see on the TV broadcast, says Winsor.

Part of the process has also involved Payne pulling on the dreaded mo-cap suit. But Winsor and Brignall say he's been up for anything. We had a very early session with him when we went to our motion capture studio and we got him in a mo-cap suit, and had a play and started loosely blocking it out. At that point we captured some test data so that we could run that through our app, and we did the photogrammetry session in the afternoon, adds Winsor.

We had another session about three weeks later where we had our first version of the avatar so he got to see what it was all about and move his avatar around. In that session we tested everything was working. We did a dry run the week before last in the studio we're using to stream the performance from, and then we've got him for one more day.

So what does Payne make of his digital doppelganger? He seems to love it, says Winsor. It's a strange experience, someone controlling their own avatar in augmented reality. In the studio th

LINK:	https://www.tvbeurope.com/production-post/getting-up-close-and-persona...
	See more stories from tvb

Most recent headlines