-- S3: The API Nobody Loves but Everybody Uses
At Object Matrix, we are big users of Amazon S3 API and with so much talk about organisations offering S3' interfaces I thought I would share our experiences.
We have been using S3 API for quite some time in two areas:
Development of applications accessing S3 buckets either at Amazon or at systems providing S3 interface.
We developed an S3 interface to our product MatrixStore to facilitate integrations with partners and existing applications using S3.
In each area, we obtained a better knowledge of what it really means dealing with S3.
Client Applications
The first thing we noticed about S3 is that it does not provide great granularity of commands in certain areas but, we understand why since it must be tied to the storage behind the API and the type of service Amazon offers.
Another thing we noticed is that the implementation behind S3 is not very strict with the API definition. That is great for developers, i.e. you can make a few mistakes calling the API with wrong parameters and the system still works. However, that makes it difficult if you are trying to develop a backend that simulates the behaviour of an S3 server, as I describe later.
I must admit Amazon did very well launching an API and infrastructure so easy to use that it attracted developers and made the API a de-facto standard. These days, everybody talks about S3.
However, as an external company working with APIs and partners, we are kind of forced to use S3 even knowing it's not ideal. It does not answer everybody's needs and it didn't come from any standardisation body.
It was interesting to see how the API evolved. It started as a REST API defined by Amazon but it has evolved to have multiple wrappers in different programming languages. That made S3 even easier to use but at the same time, more difficult to support as a backend.
Of course, being a REST API has its own benefits and issues, e.g., the speed of a REST API is never going to be great compared to optimised network protocols something that is important when transferring large media files.
S3 Backend
Implementing an S3 backend has been a challenge for several reasons. Many vendors claim to be S3 compliant (there is no such thing) but in reality we are all S3ish.
Since Amazon's implementation is very relaxed with the API definition, we found that several applications were using the API in an inconsistent and sometimes erroneous way, but they worked well with Amazon's S3 implementation.
Therefore, we had to adapt our implementation of the interface to accept those errors' since Amazon is the reference so how to explain to a developer he is using S3 in the wrong way if it works with Amazon!
The different programming language wrappers around S3 made it easier for developers to use but even more difficult for the implementation of S3 since some of those wrappers call S3 in not by-the-book ways which are hidden to developers.
Another aspect of S3 that complicates its implementation is that there are no official tests to certify the implementation. We found some frameworks but due to the nature of Amazon's implementation, even those frameworks have many failures using Amazon so it's very difficult to use them as a reference although they definitely help and we thank the people making those tests available.
Conclusion
Having S3 so easy to program against and to use has helped many developers and companies to use it in multiple areas; it has become a de-facto standard. That is one of the reasons for the success of Amazon Web Services.
We are and will continue using S3 as an API for application development and backend implementation but it wasn't as straightforward to simulate the backend as we had originally hoped for.
We think it would be for good if Amazon leads S3 standardisation for official test frameworks to appear. Everybody could share a common basis for development, products will increase their quality and after all, the more people using S3, the better for Amazon.
About Object Matrix
Object Matrix is an award winning UK based software company that pioneered Digital Content Governance (DCG), object storage and the modernisation of digital video workflows. Our media focused private and hybrid cloud solutions are tightly integrated into file based and IP workflows and bring economic and operational benefit to all of our customers. Our flagship product, MatrixStore, is used by the world's largest organisations that create and distribute video content, including NBC, TV Globo, MSG-N, the BBC & BT.
www.object-matrix.com
Most recent headlines
09/11/2025
Dalet today announced a transformative leap forward for media operations: Agentic Artificial Intelligence (AI) that unifies the Dalet ecosystem under one natura...
23/10/2025
A deadline is looming for a new carriage deal between Verizon's Fios TV and Nexstar, with both Verizon and the pay TV-backed American Television Alliance bl...
23/10/2025
NEW YORK During a high-profile session at NAB Show New York, new survey data was shared that revealed significant public concern over artificial intelligence (A...
23/10/2025
BELLEVUE, Wash. and NEW YORK Fox Weather has tapped T-Mobile has as its preferred communications provider and announced that all Fox Weather reporters are being...
23/10/2025
PBS and Amazon Web Services have announced the network is using generative AI to provide enhanced search results to viewers on the PBS App and PBS LearningMedia...
23/10/2025
Series three of the award winning, hit comedy entertainment series The 2 Johnnies Late Night Lock In is back on your screens, celebrating the very best of all t...
23/10/2025
Performances by Michael Flatley, Andy Irvine, Cuckoo's Nest, Foster and Allen and more
Friday 24 October, 8pm on RT One and RT Player
Fleadh Cheoil re...
23/10/2025
The nights grow longer and the shadows get bolder with Vampire The Masquerade: B...
22/10/2025
Prime Video Inks Deal To Present NFL Black Friday Game Worldwide By SVG Staff
Wednesday, October 22, 2025 - 10:06 am
Print This Story | Subscribe
Story ...
22/10/2025
NBA Tip-Off: ESPN Goes 1080p HDR End-to-End, Flipping HDR Switch on REMI and REM...
22/10/2025
FloSports Empowers Division II, III Athletic Departments With Turnkey Production...
22/10/2025
Wall Street Video Summit Debuts, Bringing Together 200 Financial Enterprise Vide...
22/10/2025
Dueling Pianos: International Chopin Piano Competition Is as Competitive as a Ba...
22/10/2025
In 1995, a young Colombian artist released an album that would change Latin pop ...
22/10/2025
Over the past few months, a photovoltaic system has been installed on a three-he...
22/10/2025
The Orion spacecraft for NASA's Artemis II mission is stacked on the Space Launch System (SLS) rocket in High Bay 3 of the Vehicle Assembly Building at Kenn...
22/10/2025
L3Harris' Hybrid SATCOM is resilient by design, offering path diversity that eliminates vulnerabilities by routing data across the best available networks i...
22/10/2025
WASHINGTON, D.C. Organizers of NAB Show New York said they are expecting more than 12,000 registered attendees from about 100 countries along with 260 exhibitor...
22/10/2025
WASHINGTON, D.C The organizers of The 2025 NAB Show New York have announced that they are expecting more than 12,000 registered attendees from about 100 countr...
22/10/2025
Masque Sound, a leading theatrical sound reinforcement, installation and design company, supplied an extensive gear package of professional-grade equipment for ...
22/10/2025
Lightware, a global leader in signal management and AV connectivity solutions, is seeing strong market momentum for the UCX-3x3-TPX-RX20, a compact transmitter-...
22/10/2025
MELVILLE, N.Y. Chyron has released PAINT 10.2, the latest update for its telestration platform, adding support for SMPTE ST 2110 IP workflows, expanding brandin...
22/10/2025
WASHINGTON Run3TV today said NBCUniversal is joining as an investor in the ATSC 3.0 Framework Authority, which develops the Run3TV NextGen TV application platfo...
22/10/2025
ATLANTA swXtch.io will feature two new networking solutions extending the company's reach across more cloud and on-prem workflows at NAB Show New York, set ...
22/10/2025
The Warner Bros. Discoverys HBO Max streaming services has increased prices for all its streaming tiers effectively immediately for new customers. Existing cust...
22/10/2025
LOS ANGELES OpenDrives has signed a new distribution partnership deal with Versatile Distribution Services (VDS) to strengthen its channel and streamline how it...
22/10/2025
WASHINGTON, D.C The organizers of The 2025 NAB Show New York have announced that they are expecting more than 12,000 registered attendees from about 100 countr...
22/10/2025
Samora Pinderhughes Brings Immersive Sound to Berklee's Signature Series The artist and composer, who's worked with Herbie Hancock, Robert Glasper, Co...
22/10/2025
BMI Day at Berklee Celebrates Composer Fil Eisler and Awards Scholarship to Stud...
22/10/2025
October 22nd, 2025 TRIBECA ANNOUNCES STAR-STUDDED LINEUP OF MEMBERSHIP EVENTS F...
22/10/2025
Rohde & Schwarz and TRUMPF cooperate in drone defense Rohde & Schwarz and TRUMPF partner to deliver a comprehensive drone defense solution combining Rohde & S...
22/10/2025
European Broadcaster Upgrades To Grass Valley's NativeIP LDX 135 Cameras And...
22/10/2025
Everyone Gets a Better Deal on Verizon with New FOX One Perk The $15 FOX One streaming service perk is yet another way Verizon continues to add savings for cu...
22/10/2025
First Look Hidden Assets Series 3
Premieres on 9th November 9:30pm on RT One & RT Player
WATCH HERE Promo Link: Hidden Assets Series 3 | RT
The Crimin...
21/10/2025
NAB New York 2025: Although AES Show Is on Its Own, Audio Will Be a Major Part o...
21/10/2025
NAB New York 2025: Business of Broadcast and Media,' Future of Content'...
21/10/2025
SVG All-Stars: Ethan Folz, Senior Director, Digital Operations and Quality of Ex...
21/10/2025
NBA on NBC/Peacock: Livestream Offers Graphic Overlays, Predictive Gaming, Ancil...
21/10/2025
NBA on NBC/Peacock: At the Front Bench With Producer Frank DiGraci and Director ...
21/10/2025
NBA on NBC/Peacock: NBC Sports, NEP Build Ultra-Flexible Production Plan That Se...
21/10/2025
Indigenous storytelling has been at the heart of the work of the Sundance Instit...
21/10/2025
Top L-R: Mysterious Skin, American Dream Second Row L-R: Little Miss Sunshine, D...
21/10/2025
Last week, Spotify and Columbia Records transformed Pier 4 at the Brooklyn Army ...
21/10/2025
SBS Learn's Dharug Ngurra resource empowers classrooms to meaningfully celeb...
21/10/2025
As global operators simplify and evolve their digital platforms, NPS improvement...
21/10/2025
Critical Design Review completion is a key milestone on the path toward Wideband Global Satellite Communications certification for the network in 2026, opening ...
21/10/2025
New Australian undersea training range to implement and improve warfighting tactics, proficiency and safety; enable joint/allied training that contributes to pr...
21/10/2025
eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...
21/10/2025
September, the beginning of autumn, brought an expected revival to the TV market, largely due to the new fall TV schedules. The time spent in front of the TV sc...
-- S3: The API Nobody Loves but Everybody Uses
