S3 Object Storage
At Object Matrix, we provide hybrid storage for media workflows, with the great benefit of making the solution manageable from a single pane of glass . Customers can therefore easily manage their data and where that data moves, whether that data is on-premise, or on the cloud.
A big part of our cloud integration is working with AWS's (Amazon's) S3 Object Storage which has led us to understand at a deeper technical level just what you can and can't do.
This of course is a changing environment, for instance AWS has recently unified different storage classes under the same umbrella: S3 Object Storage.
In this blog, I will share some of our experiences dealing with S3 Object Storage classes and their APIs.
Data or Object Management
The unification of the different storage classes has brought several advantages to our applications and services. The most important to us are:
Facilitates the movement of data between tiers
The new APIs allow direct access to the data to and from the different tiers
However, one of the things we noticed is that what moves seamlessly between tiers is the data, not the metadata. Apparently, Glacier classes do not support any additional metadata for the archives except for an optional archive description.
This means that the metadata has to be kept either on the other AWS tiers or on the client side.
We would guess that there must be an intrinsic limitation in managing metadata by Glacier due to the backend technology but that means this is an important aspect to consider if you want to manage the full lifecycle of an object: Where to keep the metadata safely? And, how to keep the metadata associated with the content, e.g., even if the content is moved to another technology?
At Object Matrix, we consider objects in all scenarios and fundamentally believe that the metadata should always be kept with an object regardless of where the object is moved to. We've seen solutions that, e.g., keep the metadata in a database. That disables the transportability of the object (separately from the database) and can result in a form of vendor lock-in.
Metadata
Some AWS S3 storage APIs support user and application metadata. However, there are some limitations around metadata in AWS S3 that are very relevant in media workflows:
It's not possible to manipulate metadata entries independently
S3 objects metadata is limited roughly to 2KB in size
(reference: https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html)
These limitations may not be important for generic applications but in media workflows, metadata can be large and it's as important as the data or more.
We have found, maybe due to those limitations, that many companies use S3 for data-only movements, no metadata is stored with the objects. Basically, they use an object API to access just the data.
At Object Matrix, we are strong users of S3 APIs at both the client and backend sides. However, the way that the S3 metadata storage in that area works means that we see as advantageous to keep metadata in a linked sidecar file rather than in the object itself. This has the added benefit that the same sidecar file can be stored on just about any technology: LTO, another cloud provider with its on metadata storage idiosyncrasies, etc. Also, the sidecar file can hold an almost limitless amount of metadata.
Metadata Specification
Another aspect that came to our attention whilst using AWS S3 APIs between our products and third parties is that having an API only is only half of the story when it comes to building rich integrations and media workflows.
Basically, it would be really helpful for all of the tools if there existed a media file metadata specification for object storage. The specification could define fields around projects, relative paths, media types, etc. There are specifications within, e.g., IMF that could be drawn upon. But how to bring those up to a higher level and in a format that aids applications to collaborate at an object storage level?
So in summary, we've had to go about this ourselves and have defined a high-level metadata specification, or, as we term it internally: a Metadata API. All our applications use that specification, be it via filesystem access, S3 access, FTP, DropSpot or Vision. It allows our applications to collaborate seamlessly. And where we integrate with 3rd parties, we pull their metadata into our standardised format. We'd love to see a specification body (hint hint DPP / SMPTE) look at whether a wider specification would be useful.
Conclusion
Amazon S3 Object Storage is evolving quickly to make simple to use cloud storage. With our tools, we combine different tiers. But whilst it has object metadata commands in its API we find that for transportability, avoiding metadata limitations within an API (either in AWS or any other platform), non-proprietary access and no vendor lock-in keeping metadata in a sidecar file is key.
And, with object APIs, keeping metadata is only half the story for application interoperability. The other half of the story is keeping the metadata in a format that can be easily understood and shared. We need defining metadata specifications if we want to achieve portable integrations for our customers.
About Object Matrix
Object Matrix is the award winning software company that pioneered object storage and the modernisation of media archives. Our on-prem and hybrid cloud storage solutions bring operational and financial benefits to our customers by securely managing content at every stage of its lifecycle; from ingest and nearline to archive and distribution. Deployed where you need it, our technology is non-proprietary, integrates into existing workflows, enabling you to work locally and share globally.
Our flexible approach, coupled with our focus on the media industry, me
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
07/10/2026
Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
19/08/2026
As the cost of IP nears price parity with traditional SDI, Reynolds argues that ...
19/08/2026
A private 5G bubble was chosen instead of traditional broadcast RF for the race ...
19/08/2026
(L-R) Rafael Manuel and Isabel Sicat attend the Filipi ana premiere during the 2026 Sundance Film Festival at The Ray Theatre on January 23, 2026, in Park Cit...
19/08/2026
Studio owners invited to have their say on rates cuts
The Music Producers Guild (MPG) are inviting UK-based studios to take part in an in-depth survey that ...
19/08/2026
Vintage-inspired desktop interface revealed
Harrison's latest hardware offering sees them head into the world of audio interfaces, delivering a 10-in/12...
19/08/2026
Granular effect remains locked to scale and tempo
Described as your dream granular effect , Minimal Audio's latest creation is capable of turning any s...
19/08/2026
First polyphonic SWAM instrument incoming
Audio Modeling's latest physical-modelling orchestral instrument is just around the corner, and is now availab...
19/08/2026
World Cup Games Generate 84 Billion Minutes Across FOX, Fox Sports 1 and NBCU...
19/08/2026
July brought typical peak-holiday drops in viewership, driven primarily by viewers stepping away from screens and a visible decline in reach across both traditi...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Henry joins BCE as the company accelerates its commercial strategy and expands its international presence across the media, broadcast and technology markets. He...
19/08/2026
iWedia, a leading provider of software solutions for connected TV devices, today announced a major milestone in Android platform optimization with the successfu...
19/08/2026
NTP Technology announces that ST 2110-30 audio-over-IP connectivity with AES67 and RAVENNA capabilities is now available for the brand's world-class audio i...
19/08/2026
Net Insight has been selected to provide its Nimbra platform by the service provider Big Blue Marble, formerly ORS Group, as the core media transport solution f...
19/08/2026
LiveU has supported Coventry City FC and several other top-flight UK football clubs through one of their most demanding live production challenges of the year, ...
19/08/2026
Nature Seekers Documentary Shot with Blackmagic URSA Cine 12K LF
Brie Clayton August 18, 2026
0 Comments
Filming endangered turtles required camera...
19/08/2026
Neyrinck Brings D-Control Consoles Back to Life on Apple Silicon with D-Control ...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
19/08/2026
Riedel Communications today announced that Edith Cowan University (ECU) in Perth, Australia, implemented a comprehensive communications infrastructure that brin...
19/08/2026
QuickLink, a leading provider of award-winning video production and remote guest contribution solutions, will spotlight a major leap forward in remote guest con...
19/08/2026
Integrated platform melds media asset management, playout automation and newsroom workflows
EMAM, PlayBox Neo and SNEWS have officially launched Nuvex, a conne...
19/08/2026
Screen Australia releases the Private Investment Toolkit to strengthen screen in...
19/08/2026
Do Aussies truly value local content? Unpacking Screen Currency 2026 with Deirdr...
19/08/2026
There's an early Christmas gift for Bookish fans, as UKTV confirms the re-commission of the hit U&alibi drama, created by and starring Mark Gatiss. Viewers ...
19/08/2026
Wednesday 19 August 2026
Sky gives sports fans the chance to watch multiple liv...
19/08/2026
On Saturday September 5 Katie Taylor will step into the ring for the fight of her life against Flora Pili in front of 80,000 fans in Croke Park. A legendary fig...
18/08/2026
SportStream 2026, a free three-day virtual event, will take place September 15-17. One registration provides access to all three days of live programming and on...
18/08/2026
BeckTV will introduce BeckFlow to the European market at IBC2026, demonstrated on the Providius stand (10.F53).
BeckFlow is a web-based schematic documentation...
18/08/2026
Boland Communications will introduce a SMPTE ST 2110 input option for its 4K monitors at IBC2026, along with a new 55-inch super-narrow bezel display for video ...
18/08/2026
Dolby Laboratories and the Seattle Seahawks have announced a partnership integrating Dolby OptiView into the Seahawks' live streaming coverage, making the S...
18/08/2026
For BMG Executive Producer of Sports Production, Graham Taylor, producing live s...
18/08/2026
TMRW Sports has appointed Rufus Hack as President of Golf, a newly created role in which he will lead the company's golf businesses including TGL presented ...
18/08/2026
SMT will provide race technology for FOX Sports' coverage of the Freedom 250 Grand Prix of Washington, D.C. on August 23, when the NTT INDYCAR SERIES races ...
18/08/2026
The live, YouTube-native roundtables combine fan voices, credentialed reporters,...
18/08/2026
Behind The Mic provides a roundup of recent news regarding on-air talent, includ...
18/08/2026
The industry legend sounds off on replacing the Heat's iconic Medusa score...
18/08/2026
William F. Rasmussen, the sports fan-turned broadcaster and entrepreneur whose out-of-the-box idea of a 24-hour cable sports network, ESPN, changed forever the ...
18/08/2026
DAZN Group has completed its acquisition of ViewLift, a provider of streaming and digital solutions for sports content owners. The acquisition was first announc...
18/08/2026
The American Association of Professional Baseball (AAPB) has added SI TV, Sports Illustrated's 24/7 streaming channel, as a broadcast partner for the remain...
18/08/2026
The Esports Foundation has announced that the inaugural Esports Nations Cup (ENC), originally scheduled for November 2026 in Riyadh, will be postponed to Novemb...
18/08/2026
Diversified has announced a new Global Managed Services Practice, consolidating the company's managed services capabilities into one organization. Rob Mello...
18/08/2026
Appear has completed a proof of concept with Thoroughbred Racing Productions (TRP), demonstrating multi-camera remote live production over bonded 5G cellular an...
S3 Object Storage 
