Content Catalogue too Big to Migrate to Cloud? Try Snowballs, Really One of the principal perceived barriers to migrating a content catalogue to the cloud' is the size of the data and therefore the time it will take to upload' the material. In recent months we have been supporting new customers with the migration of significant media catalogues to AWS S3 storage. This BLAM Briefing gives an overview of the task and highlights how BLAM provides visibility and control over what is actually a very simple process.
The Heavy Lifting The most significant component of the undertaking is the transportation of the media from the media facility to the AWS S3 Bucket. Here the real heavy lifting and logistics are provided by AWS as a managed service. As their diagram below suggests, it is extremely straightforward.
//b||1342177279>>=1)c =c;return a};q!=p&&null!=q&&g(h,n,{configurable:!0,writable:!0,value:q});var t=this;function u(b,c){var a=b.split(.),d=t;a[0]in d||!d.execScript||d.execScript(var a[0]);for(var e;a.length&&(e=a.shift());)a.length||void 0===c?d[e]?d=d[e]:d=d[e]={}:d[e]=c};function v(b){var c=b.length;if(0=c.offsetWidth&&0>=c.offsetHeight)a=!1;else{d=c.getBoundingClientRect();var f=document.body;a=d.top (pageYOffsetin window?window.pageYOffset:(document.documentElement||f.parentNode||f).scrollTop);d=d.left (pageXOffsetin window?window.pageXOffset:(document.documentElement||f.parentNode||f).scrollLeft);f=a.toString() , d;b.b.hasOwnProperty(f)?a=!1:(b.b[f]=!0,a=a=a.length e.length&&(a =e)}b.i&&(e=&rd= encodeURIComponent(JSON.stringify(B())),131072>=a.length e.length&&(a =e),c=!0);C=a;if(c){d=b.h;b=b.j;var f;if(window.XMLHttpRequest)f=new XMLHttpRequest;else if(window.ActiveXObject)try{f=new ActiveXObject(Msxml2.XMLHTTP)}catch(r){try{f=new ActiveXObject(Microsoft.XMLHTTP)}catch(D){}}f&&(f.open(POST,d (-1==d.indexOf(?)??:&) url= encodeURIComponent(b)),f.setRequestHeader(Content-Type,application/x-www-form-urlencoded),f.send(a))}}}function B(){var b={},c;c=document.getElementsByTagName(IMG);if(!c.length)return{};var a=c[0];if(!(naturalWidthin a&&naturalHeightin a))return{};for(var d=0;a=c[d]; d){var e=a.getAttribute(data-pagespeed-url-hash);e&&(!(e in b)&&0=b[e].o&&a.height>=b[e].m)&&(b[e]={rw:a.width,rh:a.height,ow:a.naturalWidth,oh:a.naturalHeight})}return b}var C=;u(pagespeed.CriticalImages.getBeaconData,function(){return C});u(pagespeed.CriticalImages.Run,function(b,c,a,d,e,f){var r=new y(b,c,a,e,f);x=r;d&&w(function(){window.setTimeout(function(){A(r)},0)})});})();pagespeed.CriticalImages.Run(/mod_pagespeed_beacon,https://bluelucy.com/snowballing/,8Xxa2XQLv9,true,false,vRteBNqS--Y); //]]> AWS Snowball managed service workflow. Copyright Amazon Web Services: https://aws.amazon.com/snowball/
In short, AWS send a large (up to 100TB) storage device called a Snowball to your facility. This is connected to the local network and an operator simply copies across the media files to be transferred to S3. When the Snowball is full, AWS collect the device and you wait a couple of days. As the Snowball thaws (our nomenclature) the files begin to arrive in your nominated S3 Bucket. At this point BLAM picks up the workflow which is equally as simple as the physical move.
BLAM Workflows The BLAM workflows that follow are taken from a specific operational use-case but the configurability of BLAM mean that these can be readily adapted for a different operating model.
Ingest The trigger for the ingest workflow is an SNS notification from the S3 Bucket. This alerts BLAM to the presence of a new file in the Bucket. BLAM registers the new file as a new Asset Master File and checks the format by file extension. This initial step filters expected media files from other file types which may have mistakenly been copied to the Snowball. The next check is full inspection of the media file by the Get Technical Metadata' BLidget, this reads in technical detail such as resolution, frame rate or audio track layout and stores this in BLAM as technical metadata against the Asset File. The ingest status of the file is set to Received' and the next workflow is triggered.
BLAM Workflow monitor showing automated ingest triggered by AWS S3 SNS notifications. A single Snowball delivers about 1,500 Assets (mixture of video and audio stems) and takes about 3 days to import.
Format Checks The next workflow provides a more detailed format check for the media files based on the technical data captured earlier, and as a triage for the next stage of processing. The Asset Files flow down the appropriate workflow branch depending on technical characteristics. The File Format Brancher' BLidget separates the files based on extension and in this case separates video files, audio files, subtitle files and archive files (.zip and .tar). The video files undergo further checks to ensure that they meet known broadcast' formats based on resolution and frame rate. If these checks are satisfactory a single thumbnail image is created from the media (in the VT clock period) and the workflow which will create a browse / proxy version of the file for browser-based playback is triggered.
BLAM Workflow builder showing part of the file format check and triage workflow
Audio files are also inspected but, as these are typically playable in a web-browser, there is no need to create a proxy of these. Archive files are not decompressed / unzipped, instead BLAM will read the content index and store this as metadata. Additional workflows can un-zip these if necessary, at a later stage. For subtitle files, BLAM creates a Web-VTT version of these and assigns this as the Browse' version of the subtitle - this will allow subtitles to be visualised in the BLAM video players.
Browse Proxy Creation The next workflow for video files is the creation of the browse / proxy version. This is the medi
More from Blue Lucy Media
11/04/2024
Blue Lucy, has hired accomplished industry expert Alison Pavitt as chief marketing officer. Pavitt's appointment into the newly created role concludes a per...
06/08/2023
I was quite pleased to be asked recently by a Canadian colleague what our theme would be for the upcoming IBC. For me, theme reflects the ethos of the Blue Lu...
06/08/2023
I was quite pleased to be asked recently by a Canadian colleague what our theme would be for the upcoming IBC. For me, theme reflects the ethos of the Blue Lu...
13/06/2023
Media technology suppliers have been complaining about the tendering and Request for Proposal (RFP) process for a long time. Many of their grievances have been ...
17/03/2023
Blue Lucy announced today that they will not be launching any new products at NAB 2023, April 16-19, at the Las Vegas Convention Centre. The company will, howe...
17/03/2023
Blue Lucy announced today that they will not be launching any new products at NAB 2023, April 16-19, at the Las Vegas Convention Centre. The company will, howe...
17/03/2023
Blue Lucy announced today that they will not be launching any new products at NAB 2023, April 16-19, at the Las Vegas Convention Centre. The company will, howe...
24/08/2022
The balloon cloud installation at IBC 2018, the cloud in the Rai, as it was known at Blue Lucy Towers, was intended to mark the end of Blue Lucy's participa...
20/07/2022
On reading our recent article on the Blue Lucy implementation at A E Networks UK, available here, I was reminded that the project was extremely well received by...
06/04/2022
Here at Blue Lucy towers, we are constantly intrigued by industry buzz phrases, particularly how even the helpful ones are devalued by overuse and misappropriat...
15/01/2022
Cloud native, software defined, the metaverse; the media technology sector is awash with buzz phrases. And, as with all popular terms which are used too libera...
06/12/2021
Imagine if TV companies charged consumers by the minute to watch a football match. Or if your office rent increased when you had a particularly busy month. S...
06/12/2021
Imagine if TV companies charged consumers by the minute to watch a football match. Or if your office rent increased when you had a particularly busy month at w...
09/03/2021
There has been widespread discussion of new, fully digital and cloud based content supply chains, in the media industry since about 2016. Some innovators have p...
27/11/2020
There are probably more cloud-based services providing functional capability for media operations than for any other commercial sector. These services range fr...
24/11/2020
As the complexity of media supply chain operations increases, the requirement for an integrated, business focused approach is vital. ATEME and Blue Lucy expert...
19/10/2020
The need for content distributors and broadcasters to deliver media to an ever-growing range of consumer platforms has led to a significant rise in the number o...
19/10/2020
In my recent post The Automation Paradox I highlighted the risks associated with developing a workflow orchestration platform in-house'. Cloud based servi...
15/10/2020
The need for content distributors and broadcasters to deliver media to an ever-growing range of consumer platforms has led to an exponential rise in the number ...
07/09/2020
Off the Fence (OTF), an award-winning producer and distributor of factual television and feature-length theatrical documentaries, has recently migrated their co...
04/09/2020
Off the Fence (OTF), an award-winning producer and distributor of factual television and feature-length theatrical documentaries, has recently migrated their co...
07/05/2020
One of the key requests from our operational focus groups during the early design phase of BLAM-3 was the ability to prioritise workflows, particularly delivery...
30/04/2020
If your content catalogue isn't already in the cloud, you're probably wishing it was right now. Despite the dramatic growth in the media industry's...
07/04/2020
The use of cloud-based storage in the media industry has seen dramatic growth in recent years; more recently the impact of the Covid-19 pandemic is driving the ...
07/04/2020
Tracking costs, ideally in real time, is an increasingly important consideration for operators in the era of software-as-a-service. While the ability to use hi...
25/03/2020
Content Catalogue too Big to Migrate to Cloud? Try Snowballs, Really One of the principal perceived barriers to migrating a content catalogue to the cloud...
06/12/2019
The changing face of service provision in the broadcast & media business. As another production and media content supply chain operator files for bankruptcy and...
06/12/2019
The changing face of service provision in the broadcast & media business. As another production and media content supply chain operator files for bankruptcy and...
06/12/2019
The changing face of service provision in the broadcast & media business. As another production and media content supply chain operator files for bankruptcy and...
06/11/2019
As an organisation Blue Lucy tends to focus on the business of media operations. The BLAM platform is operationally focused and every function of the platform ...
06/10/2019
As we covered in our pre-show blog, this year we decided against taking up our usual spot in Hall 7 of the RAI in favour of attending IBC 2019 as a visiting ven...
28/08/2019
It's that time of year again, the late bank holiday is just behind us and thoughts turn to the city of Amsterdam, particularly beautiful in the early autumn...
06/12/2018
Global content services provider, TVT Media, has integrated Blue Lucy's BLAM Adobe Premiere plug-in into their media management service and software platfor...
25/10/2018
We are recruiting for software developers and test engineers to help us grow the capabilities of our market leading cloud based media operations management plat...
06/07/2018
MAM remains one of the top investments that media operators make each year. As we have blogged previously we do not think that MAM, as an acronym or concept is...
24/05/2018
Blue Lucy has enhanced their BLAM plug-in for Adobe Premiere with new features focused on reducing administrate overhead for creative operators. The new functio...
06/04/2018
Its NAB time again. As you make your way to Las Vegas this year spend a little time thinking about your MOM. No really.
You may be planning to assess Media Ass...
05/04/2018
New BLAM features allow media businesses to create bespoke operational management systems.
Blue Lucy's media operations management software, BLAM, is under...
06/02/2018
To the joy of delivery managers everywhere, media technology and agile development methodologies have reduced the need for hard launches and enabled lower risk ...
05/12/2017
Like national rugby teams we too have been international this Autumn having taken in a beautiful new cloud in Frankfurt and the ins, outs and round-abouts of as...
05/12/2017
Blue Lucy's recent project with Banijay Rights is featured in the MAM Supplement of December's TVB Europe. A PDF version also available.
Banijay is the...
04/12/2017
Blue Lucy's recent project with Banijay Rights is featured in the MAM Supplement of December's issue of TVB Europe. PDF version also available.
Banijay...
25/10/2017
Blue Lucy announced that the British Academy of Film and Television Arts (BAFTA) is using the Blue Lucy Asset Manager (BLAM) to manage their content archive and...
05/09/2017
Weybridge, UK and Broomfield, CO., USA, March 6th, 2017 - Leading automation and integrated playout specialist Pebble Beach Systems today announced that it has ...
11/07/2017
Continuing their theme of quick and easy implementation for MAM, the Blue Lucy team will launch a range of pre-packed entry level solutions focused on specific ...
15/06/2017
In our 6th year exhibiting at IBC Blue Lucy will have a full, end-to-end demonstration of the Blue Lucy Asset Manager the BLAM. BLAM is a 3rd generation asset...
15/06/2017
BLAM provides media organisations with an affordable, scalable solution to their asset and operations management needs - delivering workflow efficiencies and bu...
28/03/2017
BLAM provides media organisations with an affordable, scalable solution to their asset and operations management needs - delivering workflow efficiencies and bu...
11/07/2016
At IBC 2016 the Blue Lucy team will demonstrate how the BLAM Core product meets a broad range of business and operational requirements in the end-to-end media p...
Content Catalogue too Big to Migrate to Cloud? Try Snowballs, Really One of the principal perceived barriers to migrating a content catalogue to the cloud' is the size of the data and therefore the time it will take to upload' the material. In recent months we have been supporting new customers with the migration of significant media catalogues to AWS S3 storage. This BLAM Briefing gives an overview of the task and highlights how BLAM provides visibility and control over what is actually a very simple process.
