James Shepherd, CTO, InSync Technology, takes a look at how frame rate conversion can help broadcasters meet the ever-growing demand for high-quality contentBy Contributor
Published: September 22, 2022
James Shepherd, CTO, InSync Technology, takes a look at how frame rate conversion can help broadcasters meet the ever-growing demand for high-quality content
target=_blank title=Share on LinkedIn class=share-linkedin>
Meeting the demand for perfect high-quality content is something all broadcasters have at the front of their minds, but nowhere more so than during live events, where anything less than the best fan experience is critical. Even the slightest jerky motion, pulsing, flickering, ghosting or other visible defects is unacceptable to fans and advertisers alike. Motion compensated frame rate conversion analyses adjacent frames, detects, and calculates the movement of objects within the scene, creating new frames with appropriate object placement and display timing while converting content from one frame rate to another.
The main use cases for frame rate/standards conversion are linear channels with ongoing demand, assets that need to be distributed to different markets in different formats, and events-based content such as live sports. While ongoing services are not always financially viable in cloud environments, it is in the latter case where we see the greatest degree of demand with significant spikes around major events, such as the Olympics or major golf and tennis tournaments. Due to its highly specialist nature, this demand also comes all at once.
For many areas of the broadcast chain, while the migration to cloud is the ultimate destination, we will not be there for a few years yet. The huge spikes in demand are what make motion compensated standards conversion for premium live sports an exception. Traditional hardware-based solutions are not well suited to these high-volume peaks; adding capacity not only means a huge cost outlay, but in most live environments, where space is at a premium, also requires additional footprint.
When we are talking multiple, concurrent sporting events, the issues are further compounded. As broadcasters meet demand for new services and drive to give sports fans the best possible experience, with multiple camera angles, etc., demand for frame rate conversion spirals even higher and hardware is just unable to keep up.
The cloud answers these demands very neatly, delivering a degree of agility and scalability that hardware simply cannot match. Cloud services eliminate the need for costly capex-based infrastructures and allow broadcasters the flexibility of paying only for what they actually use, as and when needed. Additionally, almost unlimited CPU processing power allows processing to be scaled rapidly and cost-efficiently.
Meeting pain points While customers are keen to sweat their investments as effectively as possible, their hardware is becoming more difficult to maintain. Added to the need for flexibility, the availability of high-quality motion compensated standards conversion in the cloud has come at a timely point, just as many existing hardware assets approach end of life. Broadcasters are now looking at how these assets will be replaced, and with what.
In parallel, with so much of the IP transport and delivery workflow now residing in the cloud, it is an enormous operational inconvenience to bring those IP streams back to an on-premise facility, where hardware decoders take them back to baseband SDI for conversion before being reencoded and pushed back to the cloud. This adds latency which in a live environment must be kept to the bare minimum; the last thing audiences and broadcasters want is for a critical piece of action to appear on social media first. Any additional encoding/decoding cycle also results in a quality impact for the end user.
These pain points were driving the industry to look for answers even before the pandemic, which has only taught us the vital importance of having flexible capability built into systems. We are also able to benefit from the fact that so much of the IP and cloud workflows have already been established. Until recently, motion compensated frame rate conversion has been the missing piece; we expect it will be plainer sailing for customers to make the transition here than in other operational areas.
Anything to do with motion compensated standards conversion has traditionally been the preserve of a handful of specialists and so has been waiting its turn to get focus from the industry at large.
A vital niche The main challenge is ensuring the teams that will be deploying the solution work with it; they need to go through the proof of concept (POC), and confirm they are happy that the necessary controls, monitoring, and backup provision are of an equivalent quality and stability to the previous hardware solutions before deployment.
Of all the processing that has moved to the cloud, motion compensated frame standards conversion remains one of the most demanding and unique, exceeding even the most advanced codecs in terms of complexity. As a result, a big part of our job as a technology partner is building awareness around the growth in complexity that server-based technology can address and solve. InSync brought the FrameFormer standards conversion software engine to market and demonstrated live IP motion compensated standards conversion using CPU-only technology in 2019.
It might have been one of the last on the list to migrate to the cloud, but motion compensated frame standards conversion is by no means least the technology is available and is working now. We have shown that we deliver a quality result across the full breadth of customers' content property, providing complex technology in a plug-and-play format. We have seen an incredible response from broadcasters to ou
