SVG Sit-Down: AWS's Regina Rossi on the Company's New Vertical-Video-Cropping Tool

AWS Elemental Inference could help sports broadcasters learn to love AI

This week,AWS unveiled AWS Elemental Inference, an AI-powered cropping tool that intelligently transforms 16:9 landscape broadcasts into vertical 9:16 formats optimized for social and mobile platforms. The service tracks subjects and keeps key action visible, maintaining broadcast quality while automatically reformatting content for mobile viewing.

The goal of Inference is to aid sports broadcasters (and other media companies) in streaming on vertical-video-based social platforms - TikTok, Instagram Reels, YouTube Shorts - and creating vertical versions of their 16:9 live broadcasts for use on their own streaming platforms. FOX Sports and NBCUniversal are among the organizations already testing the technology.

To get a better idea of what this means for the industry, SVG sat down with AWS Media Services Head of Product Regina Rossi to discuss what makes Inference different from other automated vertical-video-conversion tools, how it serves both near-time highlight and livestreaming needs, and why major broadcasters should be embracing 9:16 video instead of fearing it.

The hottest issue problem facing live sports is the 9:16 live conversion. How does AWS Elemental Inference help address this challenge?

We're launching a fully managed AI service that is helping customers meet their viewers where they are today. The 9:16 format has taken over as one of the most widely viewed types of video content. How can broadcasters who traditionally produce 16:9 content adapt to this [trend] in a cost-effective way? Because sports content is the most valuable content broadcasters have, they have made that a priority. When there are viral moments or important plays during that live event, they want to make sure it's viewed by fans on the platform [of their choice].

That's exactly what we're focused on [with AWS Elemental Inference]. Through this set of live metadata, [broadcasters] can practically transform and enhance their video for those viewers. We're able to provide this set of metadata and as low a latency as possible so that their live content can be made available to their fans in 9:16 as soon as possible.

In addition to creating instantly creating 9:16 highlights for TikTok and other social platforms, can this be used for 9:16 livestreaming?

We have some customers - including FOX Sports, NBCUniversal, and Peacock - who are interested in doing both. Yes, they are creating clips for TikTok and [other 9:16-native social-media platforms]; they also want to be able to livestream in that format as well. [Say] there are multiple events taking place and you want to be able to scroll through on your phone to view those live events, you can now do so in that 9:16 format.

You mention FOX Sports and NBCU. What role did these users play in development of AWS Elemental Inference?

As an example, FOX Sports identified that nearly 90% of all their digital content was being consumed vertically. This was a clear demand for their content. However, one of the things that was always a challenge is the number of people it took to create those vertical [clips], considering the number of live events they have.

Over the past 18 months, we collaborated with them to understand their specific requirements, the latency they needed, and the challenges in their architecture. We also had to do be able have a certain level of accuracy so that, from an editorial perspective, it made sense. They also still wanted to have the final say as to what gets [published] on their app; they did not want that to be automated. We worked with them to build out those requirements, and it ended up helping us develop this and bring it to market.

There are plenty of tools already on the market for automated 9:16 vertical-video conversion, but latency is often a non-starter for use in live sports production. How does AWS Elemental Inference combat this issue?

One of the key things we do is run the Inference video understanding in parallel with the encoding. What I mean by that is, we take the encoded content - that could be from something like Elemental MediaLive for live or MediaConvert for VOD content - and run it through Inference.

AWS's Reginal Rossi: I think of AWS Elemental Inference as a tool giving intelligence to the video broadcaster. As it's going through Inference, we have about a 6- to 10-second delay where we're understanding the context of what's happening in the scene. We're able to recognize a key moment that should be clipped and the right points in terms of saliency mapping to crop within the video.

We pass only the useful metadata back to the encoder, and that encoder takes into account the XY coordinates to make that 9:16 video format. The encoder can provide both the horizontal format and the vertical format back to the customer with very minimal latency.

Another issue with some of the existing automated 9:16 vertical-video-conversion tools has been the ability to adapt to camera cuts in live broadcasts. How does AWS Elemental Inference deal with that?

That was one of the key things we focused on. We went beyond just looking at things like object detection. As an example, in a basketball game, you are going to have cuts that have players and basketball, but you're also going to have cuts of a specific athlete, in which case there's no basketball. How do you recognize when that cut happens and what saliency matters? We do more than just object-detection. We use a form of action-saliency mapping that recognizes the athlete as well as the object.

Though not as relevant for sports, there is speaker detection with news content. We are able to understand that two individuals are speaking in the video so that we can go between them or move from one to the other.

Another element of a model we use is

LINK:	https://www.sportsvideo.org/2026/02/26/svg-sit-down-awss-regina-rossi-...
	See more stories from svg

More from Sports Video Group