Why it's crucial to optimize the timing of signals in IP remote production posted: 20/04/2021 by Andy Rayner
IP remote production has gradually begun to revolutionize the economics and logistics of production by transforming the transmission of feeds across the globe, and this trend has only been accelerated by the impact of the Covid-19 pandemic. With an increasing number of production companies now able to utilize more cost-effective bandwidth over high-performance IP-based networks, fewer resources such as equipment and staff are needed on-site. Instead, resources in central facilities are fully utilized and higher efficiency is achieved.
It's all about timing
Timing is everything for live coverage. Popular events need to be shown in as close to real-time as possible to deliver the immediacy that makes them so compelling to viewers. This means its crucial to keep latency at its lowest possible level throughout the whole process, from lens to lens. But latency is most important in the actual production workflows, as people need make key decisions and communicate together remotely in real-time.
Ensuring the synchronization of video and audio signals is also key for timing. In a traditional outside broadcast production, this is comparatively easy to assure, as local timing is used on-site, and ready-produced composite feeds (combining synchronized video and audio, e.g. using SMPTE ST 2022-6) are returned to the central location. However, with remote production, timing needs to be reconciled between multiple, sometimes very distant, sites.
Dealing with the video compression juggling act
Higher transmission capacity is now possible due to decreased bandwidth costs, but challenges can still be presented by the increased number of feeds and the expanding quality of signals transmitted. 4K and HDR signals usually require video compression to ensure bandwidth requirements are kept to a minimum and costs kept low. Compression does however impact on latency.
H.265 and H.264 and other traditional codecs used for distribution and some contribution introduce high levels of latency that are too large for live remote production. JPEG 2000, while not resulting in as high a compression ratio, offers much lower latency, but that can still be too significant for remote production. However, a new candidate for video encoding, JPEG XS, is now gaining traction in remote production by enabling decent compression capabilities (similar to JPEG 2000) with a latency of a tiny fraction of a frame. This ensures optimum latency is maintained while not compromising on video quality. An early example of deployment was for the esport final of the League of Legends in Paris, which was remote-produced by Riot Games 9,000km away from the event at its base in Los Angeles!
The importance of signal synchronization
A long-established part of live production has been signal synchronization, and a local timing reference is distributed by equipment during an onsite project. IP remote production presents its own challenges as it involves multiple geographically diverse locations, namely the venue(s), a central production location and sometimes regional locations (for commentators or vox-pop for example). These locations need to be frequency-aligned through a GNSS reference and also phase-aligned, enabling any transit delays to be compensated for via appropriate buffering in the IP media edge at the location of central production.
SMPTE ST 2110 is able to define absolute time through the RTP timestamp, which is referenced to PTP. In contrast, SDI and AES3 can only implicitly convey relative time. Each essence therefore has a timestamp on the point of capture, so alignment can take place at any stage of the workflow with total certainty.
It remains the case however that many pieces of equipment do not honor the maintenance of origination timing with RTP timestamps, and production companies need to take this into account. However, a recent revision of SMPTE ST 2110-10 may be about to change all that, as this encourages the practice of maintaining origination timing through the production chain, and it is hoped that more manufacturers will adopt this approach moving forward.
Aligning time-domains
It used to be the case that live production involved the handling of signals in the same time-domain for production teams, with no noticeable delay between the camera, microphone and production gallery captures. Now, new technologies such as augmented reality has shown that live production chains can deal with delays and different time domains.
In terms of timing optimization, this means that solutions are now embracing delay as part of the balance in reducing transit bandwidth costs. By running the remote gallery production in the central location on proxy images (with reduced definition) in a delayed time domain, the full-resolution processing can remain onsite. The time-offset vision and audio controls from the gallery can then be applied retrospectively to buffered versions of the full resolution signals back at the origination site. Timing compensation can then account for transport and processing delays. Adopting this method removes the need for all full-resolution flows to be transported from the site to the central facilities, resulting in bandwidth and cost savings.
Taking the bigger picture into account
Careful consideration of several key factors can ultimately enable production companies to optimize all aspects of timing their signals to meet the required levels of operational efficiency in IP remote production. This is crucial to match the unique demands of live events. IP remote production can also be a smaller part of a bigger production process and presents a unique solution while lockdowns and social distancing measures remain across different countries. Distributed production allows studios,
