Top 3 Technologies for the First 25 Years of the Century

Like every year, Coldago Research runs an end-user survey but 2025 appeared to be a bit special.

Around the end of last year, right after SC24 in Atlanta, GA, I realized that 2025 will arrive fast, representing the first 25 years of the century, a pretty unique moment. I asked myself about the technologies that impacted our industry the most and I decided to add this simple question to our survey as question #0.

The questions was

“What are your Top 3 IT Technologies and Innovations since 2000 that positively impacted Data Storage?” OR, in other words, “What are your Top 3 Data Storage Technologies and Innovations since 2000?”

and we addressed it to almost 1500 people, approximately 2 third from the US and 1 third from Europe, and by Europe I mean France, Germany and UK. We target only end-users and no vendors, nor cloud providers or system integrators have been involved as we wish to get their direct opinions and comments from the user community and collect opinion based on past or current offerings. We didn’t suggest any answers and we had to aggregate and merge results to unify diversity and group them.

The number 1 is about Media, number 2 about Access and number 3 is related to Operating Environment.

Number 1 is MEDIA with 82%.

And by media we mean Flash in multiple aspects and we all know all flavors as we use everyday so many products with flash memory in it.

Look at your phone, you imagine a phone with an HDD. And yes historically, we had some amazing devices like the HDD-based iPOD launched the 23rd of October 2001, yep still in our current century but almost 25 years ago. I invite you to realize all the iterations we had with phones, portable music players, tablets, laptops, servers… and in the data center.

SD Cards also revolutionized portable personalized devices. The SD Card association was founded in 2000 and today has 800 members. The first one sold in 2000 had 8MB whereas today’s SDUC or microSDUC cards offer 4TB. And today we find that more than 12 billion SD and microSD memory cards have been sold. The MicroSD iteration is 20 years old this year.

And Flash solves a famous triangle for storage and infrastructure people and globally enterprises: IOPS, BW and Latency at the same time from one single entity.

As SD Card’s trajectory illustrates, the other big jump is the capacity and the density thanks to multi-bits per cell, 3D nands and multiple dense layers. And one of the last advantages of HDDs has disappeared with high capacity SSDs and we all have in mind a few brands with 120+TB SSD, which is 4 times the largest HDD.

One of the most major progress made is related to the connectivity of flash media and in particular SSDs for the data center. NVMe was a big jump and numbers speak for themselves: 64k queues and 64k commands per queue.

NVMe also means a new form of storage networking, some people said SAN 3.0, but I think it is more significant than that as it invites vendors to reinvent storage architecture and deliver Any-to-Any or shared-everything models.

Number 2 is ACCESS with 74% fueled by HTTP, S3 and API.

This is very transformative with HTTP becoming a storage protocol since Amazon introduced the S3 API in 2006. HTTP exists before, nou doubt, on that, and some of us remember for sure some companies like FilePool, a content addressable storage from Belgium invented at Wave Research, that participated to the popularity of the concept. They got acquired by EMC in 2001 for $50 million and it became Centera. And the wave of object storage really took off with several episodes, I invite our readers to refer to the timeline dedicated to this domain available here.

This is amazing when we know SCSI, local or disk file systems and NFS for instance. It also triggered a big API wave. Before product integration required joining a partner program with a fee, we all remember that time, with all the business case justification. Guess what, big Internet giants changed all that and just said “Use our API for FREE” and we saw an explosion of product integrations. This also was the start of the programmable data center.

One of the values is that remote storage entities appear to be local and then resources become ubiquitous.

And even today, users asked about an S3 interface even if the storage product, by nature, is not an object storage product. And they don’t care, this internal aspect is masked for them, they don’t pay attention to the advantages of a flat namespace, being the responsibility of the vendor to make the product scalable and resilient as Object Storage promised.

We all use collaboration online products, cloud storage solutions and connect our on-premises or cloud-native apps to any S3-based services. S3 became really a de-facto standard and probably has transferred the biggest volume of data in just less than 20 years.

And to illustrate the universality of S3, you can now access tape libraries with S3 inviting users to unify namespaces end-to-end.

Number 3 is OPERATING ENVIRONMENT with 66%.

It covers Linux, Hypervisor and Open Source of course. And when we mention Open Source it is obviously associated with its license model.

Before this wave, it was expensive not to say prohibitive to access Unix source code. Startups didn’t receive VC money to pay commercial licenses for sure. The Linux wave, initiated in 1991, really took off at the end of the 90s and obviously after 2000 and it has continued and accelerated with tons of developments in system services. The Linux Kernel Organization was started in 2002.

Hypervisor in the Open System world got a boost with that, and of course I refer to all names with that in mind, proprietary or open source as well. And we see plenty of virtual and physical appliances offering various services.

This really helps the creation of containers and Kubernetes as well and also the Cloud Operating Model.

You can download (right click…) the animated gif below that summarizes this study and share globally.

Click to enlarge