Major Updates for AWS Storage Services

For the second annual AWS storage day, Amazon unveiled several key new product iterations. It confirms the storage role within the AWS ecosystem just a few weeks before the annual Re:invent conference.

AWS storage services is a set of 12 products, all covering various aspects of data storage, movement and more globally data management. We can argue that Snowmobile, Snowball and Snowcone belong to the same Snow category or family and then reduce the number. But at the end of the day AWS storage line is just massive.

If you, as the reader, put this product line in perspective you would realize immediately that it represents one of the most comprehensive vendor product sets on the planet. Not all classic storage vendors own 3 storage products like a block, a file or an object storage respectively covered by EBS, EFS, FSx for Windows File Server and Lustre and S3. Just for file storage, having 3 products – EFS, FSx for Windows and FSx for Lustre – could show some complexity or silos but AWS offers dedicated solutions for specific use cases and environments. In fact, the provider is the same, the console is the same, the infrastructure is the same and the bill is the same… many advantages confirmed by the rapid market adoption.

And AWS can promote a storage gateway that offers block, file and tape, a backup service, a copy mechanism with DataSync and various migration/transfer capabilities. 

AWS also acquired CloudEndure in January 2019 – we heard an amount of $250 million – , a company that impressed us when we met the team in Israel in October 2017. In December 2019, AWS silently acquired ClearSky Data, we see both Ellen Rubin and Lazarus Vekiarides on the AWS employee roster. And earlier this year, AWS swallowed KMesh to strengthen its Lustre offering.

And we almost forget to list the on-premises flavor for some of these services exposed via Outposts.

Here is the list of all new features introduced for several of the 12 products listed above:

• S3 aka Simple Storage Service
  • S3 Intelligent Tiering with 2 new archival tiers (asynchronous mode)
  • S3 Bucket Owner Condition, launched September 11
  • S3 Object Ownership
  • S3 on Outposts, announced September 30, represents a major step for AWS
  • S3 Replication (object replication across buckets, 4 modes) with metrics and notifications, delete marker

• EBS aka Elastic Block Storage
  • io2 volume with higher durability and density
  • EBS direct APIs
  • Fast Snapshot Restore
  • Volume multi attach to share block volume
  • AMI Lifecycle management

• EFS aka Elastic File System – limited to NFS
  • IAM for NFS clients
  • EFS access points
  • EFS Quick Create from EC2 Launch Instance Wizard
  • Integrations with ECS, EKS, Fargate and Lambda
  • Performance improvements with 5x read IO/s increase, 2x per-client throughput and small file 
• FSx for Windows File Server – limited to SMB
  • Available in new AWS regions
  • DNS/CNAME access
  • ECS (Elastic Container Service) support
  • Protected by AWS Backup
• FSx Lustre
  • Available in new AWS regions
  • User quotas
  • Protected by AWS Backup
• Storage Gateway
  • Access-based enumeration for file gateway
  • File-level upload notification for file gateway
  • 4x cache capacity i.e 64TB in local cache for tape and volume gateway
  • Network bandwidth control for tape and volume gateway
• Backup
  • Support for FSx is added, both Windows and Lustre, and EFS restore granularity

• DataSync
  • Network bandwidth control
  • Fully automated transfers between AWS storage services
• Transfer Family
  • Flexible specific VPC security groups
• Snow Family
  • Import VM images to on site Snow device
  • Snowball support Windows OS