S3 Object Storage
At Object Matrix, we provide hybrid storage for media workflows, with the great benefit of making the solution manageable from a single pane of glass . Customers can therefore easily manage their data and where that data moves, whether that data is on-premise, or on the cloud.
A big part of our cloud integration is working with AWS's (Amazon's) S3 Object Storage which has led us to understand at a deeper technical level just what you can and can't do.
This of course is a changing environment, for instance AWS has recently unified different storage classes under the same umbrella: S3 Object Storage.
In this blog, I will share some of our experiences dealing with S3 Object Storage classes and their APIs.
Data or Object Management
The unification of the different storage classes has brought several advantages to our applications and services. The most important to us are:
Facilitates the movement of data between tiers
The new APIs allow direct access to the data to and from the different tiers
However, one of the things we noticed is that what moves seamlessly between tiers is the data, not the metadata. Apparently, Glacier classes do not support any additional metadata for the archives except for an optional archive description.
This means that the metadata has to be kept either on the other AWS tiers or on the client side.
We would guess that there must be an intrinsic limitation in managing metadata by Glacier due to the backend technology but that means this is an important aspect to consider if you want to manage the full lifecycle of an object: Where to keep the metadata safely? And, how to keep the metadata associated with the content, e.g., even if the content is moved to another technology?
At Object Matrix, we consider objects in all scenarios and fundamentally believe that the metadata should always be kept with an object regardless of where the object is moved to. We've seen solutions that, e.g., keep the metadata in a database. That disables the transportability of the object (separately from the database) and can result in a form of vendor lock-in.
Metadata
Some AWS S3 storage APIs support user and application metadata. However, there are some limitations around metadata in AWS S3 that are very relevant in media workflows:
It's not possible to manipulate metadata entries independently
S3 objects metadata is limited roughly to 2KB in size
(reference: https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html)
These limitations may not be important for generic applications but in media workflows, metadata can be large and it's as important as the data or more.
We have found, maybe due to those limitations, that many companies use S3 for data-only movements, no metadata is stored with the objects. Basically, they use an object API to access just the data.
At Object Matrix, we are strong users of S3 APIs at both the client and backend sides. However, the way that the S3 metadata storage in that area works means that we see as advantageous to keep metadata in a linked sidecar file rather than in the object itself. This has the added benefit that the same sidecar file can be stored on just about any technology: LTO, another cloud provider with its on metadata storage idiosyncrasies, etc. Also, the sidecar file can hold an almost limitless amount of metadata.
Metadata Specification
Another aspect that came to our attention whilst using AWS S3 APIs between our products and third parties is that having an API only is only half of the story when it comes to building rich integrations and media workflows.
Basically, it would be really helpful for all of the tools if there existed a media file metadata specification for object storage. The specification could define fields around projects, relative paths, media types, etc. There are specifications within, e.g., IMF that could be drawn upon. But how to bring those up to a higher level and in a format that aids applications to collaborate at an object storage level?
So in summary, we've had to go about this ourselves and have defined a high-level metadata specification, or, as we term it internally: a Metadata API. All our applications use that specification, be it via filesystem access, S3 access, FTP, DropSpot or Vision. It allows our applications to collaborate seamlessly. And where we integrate with 3rd parties, we pull their metadata into our standardised format. We'd love to see a specification body (hint hint DPP / SMPTE) look at whether a wider specification would be useful.
Conclusion
Amazon S3 Object Storage is evolving quickly to make simple to use cloud storage. With our tools, we combine different tiers. But whilst it has object metadata commands in its API we find that for transportability, avoiding metadata limitations within an API (either in AWS or any other platform), non-proprietary access and no vendor lock-in keeping metadata in a sidecar file is key.
And, with object APIs, keeping metadata is only half the story for application interoperability. The other half of the story is keeping the metadata in a format that can be easily understood and shared. We need defining metadata specifications if we want to achieve portable integrations for our customers.
About Object Matrix
Object Matrix is the award winning software company that pioneered object storage and the modernisation of media archives. Our on-prem and hybrid cloud storage solutions bring operational and financial benefits to our customers by securely managing content at every stage of its lifecycle; from ingest and nearline to archive and distribution. Deployed where you need it, our technology is non-proprietary, integrates into existing workflows, enabling you to work locally and share globally.
Our flexible approach, coupled with our focus on the media industry, me
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
When Canada qualified for the FIFA World Cup 2026, OneSoccer knew this wasn'...
04/07/2026
Celebrates company's 80th anniversary
Rhodes have recently revealed that they will be producing a limited run of electric pianos in celebration of their...
04/07/2026
Blackmagic Design Cameras Empower Youth Broadcasting Program
Brie Clayton July 3, 2026
0 Comments
Blackmagic Pocket Cinema Camera 6K Pro and Blackmagi...
04/07/2026
iZotope Joins Boris FX
Boris Yamnitsky July 3, 2026
0 Comments
iZotope, the team behind RX and Ozone, is joining Boris FX. A letter from founder Boris...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
03/07/2026
New bundle & three Single Packs
Continuing to expand their already sizeable orchestral collection, VSL's latest release introduces three new Single Pack...
03/07/2026
Venue installs new ATC-equipped control room
LSO St Luke's, a unique music venue that also serves as the home of the London Symphony Orchestra, have rec...
03/07/2026
Caden Pearson appointed new Commissioning Editor for NITV
3 July, 2026
Media releases
National Indigenous Television (NITV) has strengthened its commitment...
03/07/2026
1 February 2023
SHARE Facebook Twitter Linkedin Email
Munich, Germany, 1st February 2023: Cinegy GmbH, the premier provider of software technology for digit...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
03/07/2026
Death Star
Andy Marken July 2, 2026
0 Comments
Look, I can't get involved. I've got work to do. It's not that I like the Empire; I hate i...
03/07/2026
Berklee's New Visual Identity: Honoring Our History, Building for What's...
03/07/2026
Scripps Research scientists awarded $2M to advance global disease surveillance Two Gates Foundation grants will expand wastewater surveillance and AI-driven dis...
03/07/2026
Joan Pulupa joins Scripps Research faculty to study the organization of DNA in brain cells and its links to neurodegeneration Using smell-sensing neurons and ad...
02/07/2026
Entering her senior year, this hometown girl is paving a career in live sports production gaining experience in replay and audio and as a TD
In the live-sports...
02/07/2026
In-venue and creative video staffers at the professional and collegiate level ha...
02/07/2026
BLAST, a competitive entertainment company focused on esports, has announced more than $133 million in revenue for 2025, representing more than 40% year-over-ye...
02/07/2026
Riedel Communications has announced official SKAARHOJ panel support for SimplyLive production workflows, enabled through the SimplyLive 2.1 release. The integra...
02/07/2026
The Fire Rescue Service of the Czech Republic has deployed LiveU video-over-bond...
02/07/2026
Gravity Media USA has announced the appointment of Brittney Boston as Head of Business Development, effective July 1, 2026. Based in Nashville, Tennessee, Bosto...
02/07/2026
TwelveLabs, a video intelligence company, has announced $100 million in Series B funding co-led by NEA and NAVER Ventures, with participation from Amazon, Radic...
02/07/2026
The Pro Padel League (PPL) has announced a broadcast partnership with USA Sports that will air five PPL championship matches on CNBC during the 2026 season, the...
02/07/2026
LiveLike, a digital fan engagement platform, has announced eight confirmed FIFA ...
02/07/2026
Cobalt Digital has received Future's Best of Show Award, presented by AV Technology at InfoComm 2026, for its blueCORE family of standalone signal processor...
02/07/2026
Synamedia has announced the appointment of Dr. Tzvi Gerstl as Chief Executive Officer. Paul Segre, who has served as CEO for the past six years, will transition...
02/07/2026
The Esports Foundation (EF) and Sony Group Corporation have announced an expanded collaboration for the Esports World Cup 2026 (EWC), taking place in Paris, Fra...
02/07/2026
Zee Entertainment Enterprises Ltd. ( Z') has announced exclusive broadcast and digital rights for the Bundesliga in India for five years, beginning with the...
02/07/2026
An effort uniting News, Sports, Local, and Telemundo, the 50+-camera live produc...
02/07/2026
Zoey Deutch, John Slattery, Ken Marino, Miles Gutierrez-Riley, and Ben Wang appe...
02/07/2026
Stammering, stuttering, strangulated tones
The Crow Hill Company's latest creation promises to be the most original sound set they've produced to d...
02/07/2026
A new era in unmixing and spectral editing
The latest version of Steinberg's spectral audio-editing software has just arrived, building on the strength...
02/07/2026
Aims to simplify additive synthesis
Sine Machine is the debut launch from Melatonin, a Vienna-based developer who have spent the past six years creating wha...
02/07/2026
Products to remain fully active & supported
Following the news of Native Instruments joining the inMusic brand line-up, Academy and Emmy Award-winning visua...
02/07/2026
What you missed!
Last weekend, Saturday 27 June 2026, saw the debut of Sound On Sounds new GearExpo UK event, the largest dedicated pro-audio event to take ...
02/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/07/2026
Following the successful launch of its inaugural APAC Mentoring Programme last month, the Rise AV APAC Regional Council will bring the conversation around mento...
02/07/2026
Blackmagic PYXIS 6K Used to Shoot Director Takahisa Zeze's Cry Out
Brie Clayton July 2, 2026
0 Comments
Highly mobile camera supports tense and de...
02/07/2026
Broadcast Solutions acquires BFE, expanding its lead in European broadcast, medi...
02/07/2026
Berklee Alum and Faculty Perform at Boston Public Library's 250th Anniversar...
02/07/2026
Broadcast Solutions GmbH, a leading systems integrator and provider of innovative solutions for the broadcast media industry, is acquiring BFE Studio und Medien...
02/07/2026
Cinegy GmbH, the premier provider of software-defined television technology, has extended the ingest facility at leading Brazilian sports company LiveMode, work...
02/07/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/07/2026
Standalone processors acknowledged for the innovation and value they bring to Pro AV
Cobalt Digital, a leading designer and manufacturer of signal processing ...
02/07/2026
Synamedia announced today the appointment of Dr Tzvi Gerstl as Chief Executive Officer. Paul Segre, who has served as CEO for the past six years, will transitio...
S3 Object Storage 
