-- A new consortium of media technology professionals is taking on an exciting project certain to change the way content is stored and accessed. ReCAP (Real-time Content Analysis and Processing) is a joint project that teams broadcast video software company ToolsOnAir with British media technology solutions provider New Media Research (NMR); nablet, a German provider of streaming, codec, muxing, and transcoding solutions; and Austrian nonprofit Joanneum Research (JRS), which focuses on applied research. The ReCAP project, which is partially funded by Horizon 2020, a European Union fund that supports making technology commercially affordable, will implement new technology to enable automated content analysis of video and audio content.
Neil Anderson, CEO of NMR
As NMR CEO Neil Anderson explains, We got together for the ReCAP project because many customers with live video workflows or existing archived video rarely have enough time to watch, analyze, and make decisions about it. ReCAP will provide a range of automatic content-analysis services, using Computer Learning and Artificial Intelligence (AI) to form an affordable, scalable, and flexible platform designed to enhance metadata-driven media workflows and realize the value of content archives.
For example, in a live television or streaming environment, rather than having staff watch multiple feeds for signal loss and audio and picture quality or invest in expensive proprietary hardware, the ReCAP software product will enable a real-time technical analysis to look out for problems and flag them. Perhaps even more powerful, ReCAP will enable analysis of existing video archives. Maybe youre a music TV channel, with years of historical footage, and you dont know whats in the archive because it was all digitized from tape, with minimal cost, time and effort Anderson comments. The company that digitized it took the metadata on the tape label and attached that to the file. But thats all youve got.
For example, say when David Bowie died and you wanted to find all the footage of Bowie so it could be repurposed for broadcast or licensed for news and documentary makers. Today, it would be very hard to find, Anderson observes, because although the video tapes are digitized and may be stored in a Media Asset Management (MAM) system, there is no time stamped metadata to find at what point in a program David Bowie appears, when hes mentioned in an interview, if his name appears in a graphic, or when one of his songs is played. Without descriptive, time-based analysis, you cant find and aggregate any historical content into a marketable David Bowie collection.
There are countless other examples. We have a house of worship customer who has footage going back 80 years. Its digitized, so its one step better than aging videotapes in the attic but none of it has time-stamped metadata - they would love to have speech-to-text conversion, to enhance their search, automatic subtitle creation and also automatic language detection. ReCAP will be a great solution for archive enrichment.
Known for powerful and intuitive applications for Mac and Linux, ToolsOnAir develops crucial products for broadcasting. The companys new onCore software combines with its mediaCore media engine to enable Linux-based broadcast professionals to experience ToolsOnAirs intuitive workflow and seamless integration of ingest, play-out, storage, and real-time graphics and to deploy solutions locally or in the Cloud.
One of the global investment banks is a big ToolsOnAir customer, and they have a lot of live ingest, notes Anderson. They are also required to store content because theyre under financial compliance. So they have an archive but they cant easily find historical content of their CEO. ReCAP would make their lives easier.
This bank also uses ToolsOnAirs just:live to put graphics over their videos. For example, theyll have a graphic with the name of an analyst who talked in an interview, Anderson explains. Again, this is a compliance issue. They need to be able to put up a slate to say, John Smith is on this call to talk about market analysis. ReCAP could do Optical Character Recognition (OCR) on the slate, find the speakers name, and make that a time-stamped piece of metadata that could be put into a MAM system so users could easily find the footage in their archive.
ToolsOnAir is currently developing a media-processing engine for ReCAP that will work with onCore. OnCore will be the front-end configuration and workflow management tool, confirms Anderson. The new media processing engine will store the algorithms that we will spin up on servers to process the content in real time. It also can be used to batch process algorithms for archived content, get the results as time-stamped metadata, and store that in a MAM.
NMR will integrate the toolsets into a solution that can work with a MAM. ToolsOnAir built their new platform on Linux, using GPU accelerated open-source multimedia framework technology, Anderson recounts. JRS and nablet are making their technology available to run on the GPU. NMR has a lot of expertise on MAM systems that run on Linux. Together, we will build an integrated solution and make this available to the global market as a product, with ongoing development and support.
No comparable commercial technology currently exists. In the enterprise, companies like Google, Facebook, Amazon, and IBM have APIs that do amazing visual analysis, speech-to-text conversion, and so on, Anderson explains. But they run exclusively on the Cloud. A lot of companies wont put content into the Cloud because it may be sensitive; or theyre under compliance or financial regulation; or perhaps they are a Hollywood studio and would never store high-value content in the Cloud. So people want to run these services on whichever platform they choose.
Not coincidentally, NMR is expert
