Spectra Logic Top 5 Trends for 2015 Storage Industry

 Top 5 Trends for 2015 storage industry in 2015 by Matthew T. Starr, CTO, Spectra Logic

A number of factors will be driving storage advancements and investments in the next 12 – 24 months; big data, new data workflows, the cloud, virtualisation, cost, regulations requiring data to be kept for longer and new technological advancements. The last few years have seen traditional storage mechanisms begin to strain as they started to reach full capacity due to the sheer number of files and the growing file size.

For the enterprise organisation today, we are dealing with terabytes, petabytes and exabytes levels of data volumes. New storage technologies are attempting to solve the issue with petabyte scale growth, including Restful interface and other public/private cloud technologies.

The advantages of the latest innovations in the storage industry can perhaps be best observed by looking at how storage advancements are impacting those industries where data is the product driving the business and data assets are being monetised.

For 2015, we are seeing trends affecting large enterprises and vertical markets such as; supercomputing, genomics research, medicine, finance, geology, government, security and media aand entertainment in the following ways:

1. Continued archive expansion
With data volumes growing exponentially, we can expect to see the continued growth of large archives in 2015. Some are growing at over 100% year on year.

2. Shift from HSM software to ERP systems
More users will be moving to a ‘data workflow’ model for storing their data. This means data will not be stored in the traditional model of tier one, tier two and then moved from tier to tier though HSM software. Instead, we will see organisations migrating towards an ERP system for the data use and needs. So we’ll see file archiving technology shifting from a policy-based management which uses the storage devices economically to retrieve data from back storage media to a much more integrated process with business rules, which leverages business needs to suit the company’s requirements.

Data that is in use will be on the highest tier, while data not currently being used will be moved into the archive. Data archive storage will be like physical parts in a warehouse, stored and ready for use but not in production. Data that is being actively worked will be on flash or other high speed storage for faster access. 

3. ‘Planned’ data will be on demand
In the year ahead automated tiered storage will continue, where HSM dynamically moves data between various disk types and archive storage for optimum capacity. But the improvement needed here is that as the amount of data grows exponentially, data use will now be planned. Which means it will also be automatically delivered directly to a user’s work space at the time of need. 

We see this will impact day-to-day task efficiency in various industries in the following ways. Video editors in the Media & Entertainment industry can receive the video clips they need to work on for that day’s scenes the previous day. Geologists will get delivery of the day’s seismic data so they can run today’s transforms producing usable results, and in the medical industry, medical records will be delivered prior to a patient’s arrival. 

The way in which the data movement will be planned and delivered is very similar to processes in the world of manufacturing.  The term ‘Just In Time’ has been used in manufacturing for many years, where parts are delivered just in time for their use. That same idea can be used in the space of massive data sets. Data will be delivered in a just in time fashion. Object stores and other new storage technologies are some of the building blocks needed to make this possible.

4. The rise of object storage and meta data
One of the trends fuelling the growth of big data that will continue in 2015 can be seen for example in the field of genomic research, where advancements in technology is having a big impact. Scanners are gaining resolution year over year and correspondingly the size of each scan is increasing. Conversely, the cost of scanner technology is falling, resulting in the purchase of more scanners. Assuming that resolution, speed and the number of scanners all increase by two with each yearly purchase, the volume of data will increase by eight times year-on-year.
 
The same dynamics are found in broadcast media, and oil and gas exploration, where many of these companies are running into a wall when it comes to storage. These issues are slowly being addressed by new developments such as object storage, object archive storage and meta data search engines. Where most file systems begin to fall over after a few 100 million files, object storage and object archive storage are newer systems that are now capable of storing billions of files in a single instance.
 
Object storage allows the user to tag data with meta data, enabling the addition of searchable key words. As more organisations understand and get into the practice of adding descriptive data (meta data) to their storage (as opposed to file names and other legacy methods), the sooner it can truly leverage the entire might of its data. This is already happening with some big data users, the best example is within media & entertainment, where companies already catalog data, but we expect this trend to grow and spread to other verticals in 2015.
 
5. The notion of ‘forever storage’ will become more entrenched
With new regulations, extending the life and quality of data has raised the importance of archiving on the business agenda. In 2015 a greater number of organisations will have to find cost effective solutions to long term storage challenges and with the new standards for archiving we will expect to see greater demand and recognition of the importance of deep storage.