Using MatrixStore for Netflix Production Asset Compatibility

-- Using MatrixStore for Netflix Production Asset Compatibility

In 2018 Netflix purchased $12bn of content. It is predicted that figure rose to $15bn in 2019 and that spending will hit $17.8bn in 2020. 85% of that spend is estimated to be on original content. This dwarfs the budgets of individual national broadcasters (e.g., the BBC has a total budget of £3.7m but only a portion of that is spent on new content).

It is therefore no surprise that Netflix has quickly become a technology leader on how production should be made not just for itself but also in setting standards that other organisations have also signed up to as both sensible and practical.

One question we get asked is Can MatrixStore help us adhere to Netflix Data Management requirements?

And, the relevant Netflix article reads like Object Matrix marketing!:

Production Assets: Data Management

The verification and back-up of data, including original camera files (OCF) and audio, throughout a workflow is crucial. Data loss can mean losing significant investment both in planning and shooting. Our requirements are designed to help minimize costly incidents during production.

Below are Netflix's critical Backup Rule requirements:

Netflix Backup Rule Requirement MatrixStore

Hold at least 3 copies of all Original Camera Footage (OCF) and audio at all times and on 2 different types of media. MatrixStore holds 2 copies with RAID6 and the original camera media could also be held to keep the data on another media type .

Also, see below.

Maintain at least 1 backup offsite Replication can quickly make an offsite backup. Other options are to use MatrixStore's hybrid capabilities to make a copy into a cloud location this can also cover the requirement to hold 3 copies of OCF.

RAID must be connected to computer by a hardware RAID controller. Software based RAID should not be used MatrixStore uses Hardware RAID. Other systems (especially other object storage systems) tend to use less reliable software RAID.

MatrixStore of course goes even further with automated data recovery if, e.g., one location goes down as well as multiple other data authenticity and data security functions.

-- Checksum Verification

Netflix also, sensibly, requires checksum verification of content. A checksum can guarantee that the content hasn't been accidentally or maliciously changed during storage or transmission. For instance Netflix write that:

Checksums are incredibly valuable for detecting the slightest change in a file, which can occur at any point during transfer or storage. For example, if a movie file is corrupted during an upload from a hard drive to a server, the checksum of the copied file would differ from the checksum of the original movie file even if a single bit is off. This comparison is referred to as checksum verification.

Netflix supports both MD5 and xxHash64BE checksums, and you'll be unsurprised to hear that the stronger of these, MD5, which is also cryptographic, is supported by MatrixStore at all levels of data storage and transmission.

Netflix Checksum Best Practice MatrixStore

Generate checksums for original camera footage (OCF) and store as a manifest This is a stage prior to storing in MatrixStore

Calculate checksum during transmission to MatrixStore and during storage MatrixStore vaults can be set to perform this in MD5

Compare stored data to checksum in OCF manifest files A simple script can be provided to perform this checking. The advantage with MatrixStore is that the MD5 is literally compared to the read back copies of data thus showing that the content hasn't just been transferred and stored correctly but also reads back correctly at all MatrixStore storage locations.

Checksum manifest, such as MHL, accompanies the files through all transitions MatrixStore can keep the (verified) MHL file with the content at all stages.

-- On Set Media Re-purpose

Finally, Netflix recommends on set media re-purpose. Or as Netflix state:

Original Camera Media may only be reformatted after both:

Visual QC of all footage has been performed against camera reports/script notes, with all footage accounted for and signed-off on.

OCFs have been copied and reside in a minimum of three storage mediums (LTO and/or RAID1, 5, 6 or 10).

On set RAID1, 5, 6 or 10 may only be reformatted after both:

Visual QC of all footage has been accounted for per camera reports and script notes and signed off by editorial.

Netflix Best Practice MatrixStore

Visual QC This can be carried out through MatrixStore Vision browsing and content viewing. Of course, comparing to reports/scripts is a manual action until AI can take over!

OCFs have been copied or reside in a minimum of three storage mediums As discussed, MatrixStore is providing two of those locations

-- Netflix Folder Structure Compliance

Netflix also requires that production assets conform to a set of standards. This requires that a folder structure containing the following elements is created:

Show Title

YYYYMMDD (Shoot Date)

Camera Media

Camera roll folders (e.g A001C001) copied into the Camera Media folder must be an exact representation of the camera card contents and contain valid checksums.

Sound Media

Sound roll folders (e.g S001) copied into the Sound Media folder must be an exact representation of the sound card contents and contain valid checksums.

DropSpot - an ingest tool for MatrixStore can help with this. By simply creating a metadata form in DropSpot you can set it to auto dump content into a pre-defined folder structure that turns the metadata captured (or entered by the user) into the fields that define that structure.

This can avoid commonly made mistakes such as an ingestor putting content into a who-knows-where folder as well as spelling mistakes. This method is similar to ones that we have used for

LINK:	https://object-matrix.com/using-matrixstore-for-netflix-production-ass...
	See more stories from matrixstore

Most recent headlines