Breakdown of EMC VNX 5400 Storage Subsystem at OVH

OVH Group revealed that one of its storage bay in Paris, France, an EMC VNX 5400, was unable to boot, affecting more than 50,000 web sites of its customers.

The French firm is one of the largest European hosting provider, with 20 data centers, more than one million customers and three million hosted web sites across 138 countries and four continents. Revenue were close to €400 million in 2016/2017. The company just raised €400 million to support its development.

The company stated on its Web site that it “ensures stable and reliable product and service offerings to clients across all its brands.”

But what it encounters was a huge technical problem on January 29 at 4:30PM, as an incident appears on one of its VNX 5400 in its Paris P19 data center containing databases and being enable to boot. This system comprises 96 SSDs configured with active/active technology on several physical bays. But these databases are only backuped each day on another data center in another French site (RBX1) in Roubaix.

The hosting firm stated that the EMC technology is not the source of the incident. Octave Klaba, OVH technical director said: “Our data centers are not adapted for this type of incident. Only some rooms are especially prepared pour this type of hosting but not this bay, which explains the origin of the problem.”

OVH is working with the vendor to find a solution. Another VNX 5400 was urgently moved from Roubaix to Paris but the hosting provider does not know how long it will take to restart the bay and recuperate the data. In the morning, 15% of the databases have been recuperated in read only mode. On June 30, at midnight, all the databases were operational and OVH is currently studying the state of the storage bay before making accessible the data.

The hosting company was also subject in 2016 to the biggest attack DDoS known to date, with peaks of over 1Tb/s of traffic.