2020 Storage Predictions by Object Matrix

The author of this report is Jonathan Morgan, CEO, Object Matrix Ltd.

2020 Storage Predictions: Invisible Storage

When it comes to media archives most people want to store lots, never have to manage it, and certainly never have to worry about data migration. Oh, and also to pay very little in the meantime.

So in making my 2020 storage predictions I’ve focused on that rather than NVMe, computational storage, storage class management and intent based storage management. To be quite honest, I love those technical aspects; but I don’t think it’s a day to day concern.

Articles about NVMe belong to the storage geeks whereas invisible storage is for those who just want something that works. It is for the many not the few (if they even know it), and it is the theme these 2020 storage predictions.

2020: 3 Good News Storage Trends

Prediction 1. Invisible Storage: Services
Ready-to-roll services are becoming mainstream and will develop out quickly in 2020. Suppliers who provide just one component will become for very bespoke workflows only.

Multiple services are now available to not just be able to store media assets into faceless buckets, but also open those assets up for usage. These workflows go beyond safekeeping of data, but also include workflows such as search, editing, previewing, collaboration, etc. It’s a bright future for individual post house, broadcaster or practically anyone with a media workflow: workflows including storage that just… work.

And, whilst larger broadcasters are trying to build more complex and bespoke service based workflows that include on-prem and cloud services, many other media companies are signing up to off-the-shelf service solutions made ready-to-roll for the creative industry.

Suppliers: Object Matrix, DropBox, Adobe and more

Prediction 2. Invisible Storage: Hybrid Storage
What people like about the cloud – low management – easy expansion – data services – opex pricing – plus hybrid needs to extend to on-prem as well. The good news is the adoption of hybrid is accelerating and will continue to in 2020.

The cloud often provides an invisible storage service, but, can: ease of management – ease of storage expansion – seamless data services – ease of “never having to migrate data again” – ability to automatically copy data from location to location – be extended to on-prem?

The good news is that in 2020 an increasing number of solutions are available that provide some if not all of these attributes.

For instance most HSM tools can now migrate from on-prem to cloud and back again, global namespace software can virtualise the asset’s location if the global namespace’s interfaces are used to access the assets. And, in the case of on-prem object storage, a full range of invisible storage services can be achieved right where you need to access use your assets.

In the year 2020, the year of invisible storage, we predict a great rise in creatives seeking solutions that provide cloud like simplicity but with the speed of on-prem access, the pricing of on-prem and the ability to seamlessly link into hybrid workflows.

Suppliers: HSM providers, global namespace providers, object storage providers

Prediction 3. The Slow Migration from LTO to Continue
LTO continues to hold many media archives that could simply be better searched, accessed, reused and montised if they were on faster access storage. AI can now better analyse assets. Organisations the world over are realising this and that’s why the move away from LTO is accelerating.

I’ve got nothing vs. LTO as a cheap medium to store assets that will never be looked at again but everything vs. LTO if the assets need to be read-back even semi-regularly or if the assets need to be analysed, searched, or generally kept for a long period of time (since then they will need to go through migration processes). Speaking recently to an owner of a major media archive with 1.5m film assets on tape, even being able to get that film digitised and then to run image recognition and speech-to-text is a major headache and hurdle. The last thing they want to do is to put those assets back into a system that will become outdated every few years, or into a system that makes it hard to analyse and re-analyse the assets.

2020: Three Worrying Storage Trends

Prediction 1: Cloud Service Ransomware
An increased level of noise from media companies who have made bad cloud investments due to unexpectedly high and ongoing costs and due to the difficulty of exit.

At a recent technical forum for the industry a TV broadcaster CTO lamented that cloud services are becoming both expensive to commence due to ‘set up fees’, expensive to use due to egress/download fees, and almost impossible to exit from due to restrictive contracts. In short, cloud services can quickly become ransomware, holding your assets but with non-stop fees.

Of course, a cloud service that is ‘time-share’ accessed can be an extremely efficient. Paying just for when you use a cloud service allows the service provider to split the costs, e.g., for CPU time, amongst many customers. However, a storage service is a 24/7 usage requirement and comes with a significant overhead for egress. The benefits are much more limited.

Worryingly, many large cloud providers only offer sensible pricing with restrictive contracts that effectively lock the customer into that service for years to come. In a changing world, lock-in is a very dangerous strategy.

Prediction 2: Global Namespace and Digital Asset Management Vendor Lock-in
Keeping media assets is valueless if you can’t find them. Many software vendors create very proprietary, difficult to share and difficult to migrate from databases, and thus create vendor lock-in. There appears to be little industry move towards open standards that would stop this from happening and in 2020 with even more metadata being generated this is increasing rather than going away.

If it feels like déja-vu, it’s because lock-in has never really gone away. Many software vendors put your key metadata: in particular key metadata that allows you to find your media assets, into proprietary databases that don’t migrate easily if at all. In particular, there is an irony that some global namespace software vendors and many HSM software providers claim to make everything findable through a single interface but then store the assets with cryptic ‘object IDs’. You need their databases to make any sense of the assets stored. The customer thinks they are buying openness, but in fact they are getting a lock-in time bomb.

Prediction 3: Metadata Incompatibility
Beyond being able to access metadata comes being able to understand and share that metadata. With so many apps creating so much metadata open standards are missing for how that metadata should be stored and used.

It seems everyone nowadays understands the importance of metadata. Yet, open metadata exchange between applications is almost zero; everyone is simply out there doing their own thing with very little in the way of even registration of metadata formats so that open metadata exchanges can be built. It would be great to see the DPP or SMPTE tackle this area with the backing of a good cohort from the creative industry.

I propose a metadata exchange – a place where metadata formats can be logged in a uniform manner such that applications can be automated to translate and use metadata in meaningful ways even if that metadata was created by another application. It’s unlikely to happen and in 2020 even more proprietary metadata will be created that one day will have to be migrated to better formats, if that is even possible.

To conclude, on the good side 2020 is exciting. It will be a year when even more storage services (on-prem and off-prem) will take care of the nuances of storage, making storing more assets easier, less expensive and with more powerful search and management services.

However, the industry still has some way to go to move away from ‘proprietary’, especially where metadata is concerned. Worse of all, many customers will get locked in to tools and/or cloud services that are paramount to legalised ransomware. But that can be fixed sooner rather than later and you don’t have to wait till 2021, or beyond, to demand solutions that avoid that.