Live From the US Open: IBM Watson Media Opens Up AI Possibilities for USTA IBM launches suite of solutions for Cognitive Highlights, Closed Captioning, and more By Jason Dachman, Chief Editor Saturday, September 9, 2017 - 9:57 pm
Print This Story
Story Highlights
IBM has long been a major tech player for the USTA at the US Open, providing backend technology ranging from the US Open app to the SlamTracker point-by-point analysis tool. This year, however, IBM has taken it up a notch, using the Open as its launch platform for IBM Watson Media, a suite of AI-powered solutions on the IBM Cloud that analyze images, video, language, sentiment, and tone. Throughout the tournament, the USTA has used IBM Watson Media's Cognitive Highlights solution to rapidly create and share highlight videos of matches across multiple platforms.
This allows the USTA to produce more highlights at scale, explains John Kent, program manager, sports and entertainment partnerships, IBM. If you asked someone to [produce] highlights for every player of every match on the seven [TV] courts and then push them out to the mobile and social immediately, it would be a monumental task. That's what Watson does as an advisor. There's still a great editorial team and video team here that do a wonderful job creating highlights, but this allows them to produce that content at scale.
Cognitive Highlights Service Makes US Open Debut
Developed at IBM Research with IBM iX, Cognitive Highlights identifies a match's most important moments by analyzing the statistical tennis data, sounds from the crowd, and the reactions of a player, using both action and facial-expression recognition. The system ranks the shots from the seven TV courts at the USTA Billie Jean King National Tennis Center (NTC) and auto-curates the highlights, simplifying video production and ultimately positioning the USTA team to scale and accelerate creation of cognitive-highlight packages.
The Cognitive Highlights platform identifies important moments in a match by analyzing data, crowd sounds, and player reactions.
IBM first piloted the Cognitive Highlights platform at The Masters golf tournament in April and then supplied it as part of the content-production system at Wimbledon two months later. According to IBM, although many editors may fear that AI technology could replace them, the users at these events and the Open embraced Cognitive Highlights once they saw its capabilities first-hand.
As soon as [the edit team] gets their hands on the technology, they [see] it as a tool that can help them be more effective, says Dave Kulczar, senior offering manager, Watson Media. Bringing insights into their existing editorial tools is a big focus for us as well because it allows them to take information and clip it, cut it, repackage it, or put together in a reel. Those are the things that you're never going to automate; those are things that editors are going to do forever. We're just making their job easier.
The finished highlight videos are available in four ways. First, the USTA posts a Highlight of the Day, as ranked by Watson, on its Facebook page each day. Second, fans that favorite players on the US Open apps automatically receive real-time push notifications about those players' highlights (and iOS 10 users can play the highlights within the lock screen). Third, video highlights are available on player bio pages across all USTA digital platforms. And, fourth, highlights are available onsite in the players lounge and in the fan-facing IBM Watson Experience at the NTC.
The USTA is always looking for opportunities to enhance that fan experience, which is where the concept of cognitive highlights come into play, says Kent. On the seven courts being monitored, we are identifying those individual shots that we would call highlight-worthy, and we are coming up with an excitement level based on the crowd cheering, player gestures, and, for the first time here, facial expressions. So we're able to truly understand the emotion of that player and factor that in, as well as the high fives and other gestures.
IBM Watson Media Goes Live With New Products
Although IBM has been piloting Watson-powered AI tools at sports events for several years, the launch of IBM Watson Media signals the company's productization of these solutions.
With IBM Watson Media, we are packaging up real, scalable products that we can build upon and customize for individual media and entertainment clients, says Kulczar. The technology being used here at the US Open is now part of the IBM Watson Media portfolio. We plan on scaling that technology for other environments, other customers, and other sports.
In addition to Cognitive Highlights, IBM Watson Media offers a variety of other AI solutions: Content Search and Discovery, Intelligent Closed Captioning, Recommendation Uplift, and Spotlighting.
Editors are finding that, instead of automating them out of a job, AI solutions like Cognitive Highlights are just another tool.
The Content Search and Discovery solution creates refined, detailed searches of content down to the scene level, based on rich metadata identifying items and context. The idea, says Kulczar, is to annotate both live and VOD content and be able to annotate that with extremely deep metadata in order to more easily search and discover the most relevant highlight clips.
Automated speech-to-text for closed captioning is also key to the Watson Media offering, transcribing audio and improving by learning topic-specific terminology. In addition to Cognitive Highlights, the USTA is using this Intelligent Closed Captioning technology at the Open this year.
[Watson] is listening to the audio and transcribing for closed captioning, Kulczar explains. For each sentence, Watson gives you a confidence level in terms of its a
