DDN Said it Can Process 55 Billion Objects Per Day
With Web Object Scaler system
This is a Press Release edited by StorageNewsletter.com on October 13, 2011 at 2:57 pmDataDirect Networks, Inc. (DDN) unveiled the next generation of its Web Object Scaler (WOS) system, WOS 2.0.
With this release, DDN introduces advanced accessibility and data protection features to compliment an appliance built with the fastest cloud storage system, capable of scalable performance 70 percent faster than Amazon S3 (From Amazon Web Services Official Blog, October 10, 2011) and more than 100 times faster than EMC Atmos (From EMC press release and media interviews, May 2011).
WOS 2.0 responds to growing industry requirements for geographically distributed Big Data applications, storage as a service providers, and rich content organizations seeking accessibility, management capabilities, and scalability to meet accelerating data growth.
"As more organizations move toward cloud paradigms, there is an exponential business need to globally ingest, store, process, replicate and retrieve hundreds of millions or billions of objects per hour in real time," said Alex Bouzari, CEO and cofounder, DataDirect Networks. "DDN’s WOS 2.0 is a breakthrough that will help solve the cloud and Big Data challenges faced by Global 2000 organizations across all industries, government agencies and cloud service providers. Our relentless focus on solving the challenges of the Big Data era has made DDN the top choice for organizations requiring state-of-the-art data infrastructure with cost and investment protection in mind. WOS 2.0 is an important leap forward for DDN and our globalized customer base."
Key new features in WOS 2.0 include:
- Massively scalable cloud storage appliance architecture, supporting more than 20 petabytes and hundreds of locations – all managed by a single pane of glass
- Expanded support for popular storage interfaces including NFS, Amazon S3, WebDAV and Apple iPad and iPhone clients
- System performance, featuring asynchronous and synchronous replication to deliver performance of up to 55 billion object retrievals per day and 23 billion object writes per day
- Erasure-code based, declustered data protection to lower storage costs and minimize request latency to less than 40 milliseconds response for small object writes, up to 25 times faster than competing dispersal-based storage platforms
- A complete cloud storage platform for service providers, including multi-tenancy, billing, and support for custom white-label service
WOS 2.0 introduces asynchronous replication, which enables users to capture and commit objects to storage faster than previously possible, increasing performance for big files and big data sets.
In addition to policy-directed replication, WOS 2.0 introduces ObjectAssure. This data protection feature is an erasure-code protection mechanism for hyper-scale, high-performance cloud storage. ObjectAssure provides data protection without relying on replication, enabling single-copy environments to deploy cloud storage, expand usable storage space and reduce overall storage costs. ObjectAssure was developed for environments in which multi-site collaboration and replication is not a requirement, but that do require a lower acquisition and TCO alternative than traditional, distributed approaches. With ObjectAssure erasure coding, each WOS node can withstand up to two concurrent drive failures per node without loss of data or data availability.
For service providers and enterprises requiring a flexible cloud service platform, WOS 2.0 introduces cloud storage management capabilities such as multi-tenancy, bill-back, encryption, and per-tenant reporting. WOS 2.0 enables service providers to host multiple tenants and custom branding of the smart device clients, Web addresses and email notifications for each tenant. WOS 2.0 also supports multiple billing and subscription revenue models and provides usage-based reporting for pay-for-use billing. With WOS, service providers can offer performance, better service delivery, support, cost, robustness and the ability to deliver superior SLAs to their customers compared with other offerings on the market.
WOS 2.0 is available and optimized for worldwide cloud storage service providers, content-intensive Web 2.0 organizations, geospatial and signal intelligence organizations and worldwide collaborative research partnerships.
"Libraries are challenged with providing access to ever-increasing amounts of digital information. Repository collections contain millions of digital objects that form sharable collections across multiple storage resources," said Dave Pcolar, Head of Library Systems at the University of North Carolina at Chapel Hill. "Our research interests involve the use of modular, scalable data appliances, like WOS, which grow to multiple petabytes independently of applications and underlying operating systems. The ability of WOS to automate management policies and validate trustworthiness of digital objects is key to its use in our infrastructure."
"Object storage and cloud storage in particular needs to progress beyond ‘cheap and deep’ to become a larger part of enterprise and service provider markets," said Henry Baltazar, Senior Analyst of The 451 Group. "The addition of NAS connectivity and mobile device accessibility, combined with WOS 2.0’s high performance capabilities will broaden the reach of this platform."
"DDN’s WOS technology is a significant differentiator for service providers," said Steve Lesem, President and CEO of Mezeo Software. "We have teamed with DDN to give service providers the performance and management capabilities they need to enhance their offerings and expedite their ability to make cloud storage a robust revenue generator."
"The volume of data from video, sensors and media is overwhelming the capabilities of traditional scale-out file solutions. As a result, object storage is quickly becoming the de facto standard for the cloud and big data era," said Dave Vellante, president and CEO of industry analyst firm Wikibon. "With WOS 2.0, DDN combines industry standard and widely adopted interfaces with the performance of its scale-out storage appliance to bring cloud economics to big data deployments."
With projected exponential growth in sensor data approaching and eventually exceeding exabyte levels, the US Department of Defense is continually evaluating new methods of processing this data. It recently procured a multi-petabyte, multi-site storage system to power sensor data applications and extend metadata management by utilizing the new object storage offered by WOS.
"From our flexible and low-latency APIs to the disk platter’s surface, WOS is designed from end-to-end as a true object storage system," said Jean-Luc Chatelain, Executive Vice President, Strategy and Technology for DDN. "Unlike other cloud offerings that are built upon traditional file systems, WOS is pristine in its access and data protection performance. Further, flexible object metadata combines with these new storage interfaces to squarely establish WOS as the bridge to enable organizations everywhere to address the next wave of information proliferation."
WOS 2.0:
Hyperscale Geo-Distributed Cloud Storage System
WOS is a scale-out cloud storage appliance solution. When storage needs grow to billions of files consuming petabytes of capacity, all of which must be quickly and efficiently shared between multiple data centers, traditional solutions become complex to deploy, hard to manage, difficult to scale, and overly expensive. WOS is a new object-based cloud storage system that addresses the needs of content scale-out and global distribution. WOS allows organizations to build and deploy their own storage clouds across geographically distributed sites that can scale to unprecedented levels while still being managed as a single entity. WOS provides fast access to hyperscale-sized data in the cloud from anywhere in the world, enabling globally distributed users to collaborate as part of a powerful workflow. WOS simplifies and improves how data in the cloud is stored, distributed, and administered. From a single management interface, organizations can build a global storage cloud that scales limitlessly.
Comments
Here we really speak of big data, big unstructured data with global namespace, object storage and no more conventional file systems, each object containing both data and metadata describing the object.
Recently we published an article, Amazon S3 Holds 566 Billion Objects Processing 370,000 requests per second at peak times. DDN's executive VP, strategy and technology Jean-Luc Chatelain told to StorageNewsLetter.com: "Our WOS network can reach 256 nodes. But in Spring 2012, we will reach one trillion of objects, more than Amazon." EMC claims Atmos 2 can handle up to 500 million objects a day. It's 55 billion object retrievals and 23 billion object writes per day for DDN.
The Web Object Scaler project was started at DDN three and a half years ago under the direction of chief scientist Dave Fellinger.
This WOS platform, managed from a single console, is based on a WOS node, an appliance with several GbE interfaces and thirty 3TB HDDs, memory and CPU, for a total capacity of 90TB or one billion objects, according to Chatelain. RAID technology is not used. Protection is done by a policy-directed replication or with ObjectAssure based on 'erasure coding'. Here each object is cut in logical blocks stored on several HDDs. "Overhead is 20% and there is no time lost for rebuilding a RAID. If one HDD is lost, the blocks missing are written on another HDD," precises Chatelain.
Latency is 40 milliseconds for writing small objects with HDDs, but it could be much better with SSDs. A node can be added without stopping WOS and with automatic rebuilding.
With a maximum of 256 nodes, the maximum capacity of a WOS platform is 23PB "with the possibility to read eight million of files per second."
What's also new at DDN is replication on LAN or WAN for this geo-distributed cloud storage system. User can manage a policy like: put these objects asynchronously or synchronously on a determined number of sites. Outside of WOS, DDN is able to measure the shortest time (latency) to get an object from different sites and automatically choosing another way if one of them is slower or not available.
WOS 2.0 provides access
through standard interfaces
WOS native API commands are PUT object, GET object, DELETE object and RESERVE Object ID, API languages being RESTful, PHP, Python, C++ and Java.
WOS does not use de-dupe to reduce capacity, but DDN is working on it with its own algorithms, a product supposed "to be launched in 10 to 18 months", for Chatelain.
What about the price? He emails us later this information: "A minimally configured, SATA-based high-density cluster of 3 nodes (replicated with 99% system utilization) is $675K plus service and support at our LIST PRICE. The configuration is 3 x WOS6000 enclosures, consisting of 6 x WOS nodes in total - coming in at 540TB RAW. As with everything here, we're very sensitive and responsive to MSP storage infrastructure economics - VERY IMPORTANT to note that list price discounts favor those who are ready to cut a purchase order."
Here is an update of some figures
already published on DDN last January:
Revenues and profitability
$200 million from January to October 2011, $187 million in 2010, $127 million in 2009 for fiscal years ending December 31. "Double digit profitability since 2005".
Revenues by applications
HPC and life sciences (53%), cloud, media and entertainment (25%), federal (20%), others (2%)
Number of employees
400, 40 more to be recruited before the end of 2011 and between 100 and 120 in 2012
Average system price range
$250,000 to $260,000
OEMs
Bull, Cray, Dell, Fujitsu, HP, IBM, SGI, Sony
Number of active customers
1,000
Average capacity per customer
3PB for HPC or military applications, 2PB to 3PB for broadcast, 200TB for media entertainment and post production
Competitors
EMC and NetApp