SNIA Solid-State Storage Performance Test Specification for Public Review

The Solid State Storage Initiative (SSSI) of the Storage Networking Industry Association (SNIA) announced the availability of the Solid State Storage (SSS) Performance Test Specification (PTS) (version 0.9, 48 pages)) for public review.

The SNIA SSS Performance Test Specification defines a suite of tests and test methodologies necessary to enable comparative testing of individual SSS device performance to facilitate broader market adoption of solid state storage technology within both the client and enterprise computing environments.

The public review phase is a 60-day period during which the proposed specification is publicly available and feedback is gathered across the worldwide storage industry. Upon completion of the public review phase, the SNIA Solid State Storage Technical Work Group (SSS TWG) will remove the SSS PTS from the web site, consider all submitted feedback, make modifications, and ultimately publish the version 1.0 of the ratified SSS Performance Test Specification.

"Until now, there was no industry standard test methodology or specified test environment for measuring solid state storage device performance," said Eden Kim, SNIA SSS TWG Chair. "As a result, each solid state storage manufacturer uses different measurement methodologies to derive performance specifications for their solid state storage products. This makes it difficult for purchasers of SSS to compare products from different companies. This announcement is a major step forward in enabling purchasers of solid state storage devices to compare the performance of solid state storage products from different manufacturers on an ‘apples-to-apples’ comparison."

A typical SSS device, taken ‘fresh out of the box’ and exposed to a workload, experiences a brief period of elevated performance, followed by a period of transition to an eventual performance Steady State. The SSS PTS prescribes preparation methodologies and defines ‘Steady State’ so that performance measurements can be taken in the Steady State performance region, which should be representative of the device’s performance during its normal working life. When executed in a specific hardware/software environment, the SSS PTS will provide measurements of performance that can then be fairly compared to those of other SSS products measured in the same way in the same environment.

Jim Handy, SSD analyst for Objective Analysis, added: "This new specification is a sure sign that the solid state storage market is maturing. The SNIA has done an admirable job of bringing all parties together to satisfy the needs of the SSD buyer by devising an unbiased benchmark that can be used to make a well-informed decision of which SSD to use."

The Test Environment
The SSS PTS specification itself is test hardware and software agnostic. The tests and methodologies assume nothing about the underlying test platform environment except specifying general requirements to ensure that there are no test hardware or software bottlenecks or unintended contributions to test measurements. However, while individual testers can run SSS PTS on their particular platform to compare the relative performance of SSS devices they are evaluating, in order for SSS PTS results to be compared on a true ‘apples-to-apples’ basis, across multiple testers, a consistent test platform must be used.

The Solid State Storage Performance Test Specification includes a Reference Test Platform (RTP) that describes the test hardware, software, and tools used by the Technical Work Group to do the bulk of the research and validation of the Specification. The RTP is not required to run the SSS PTS tests; it is merely an example of a platform that has been confirmed to meet the SSS PTS platform requirements. Readers are encouraged to use the Reference Test Platform and/or any of the tools specified therein to generate data as prescribed and submit it to the SNIA Solid State Storage Technical Work Group (SSS TWG).

The 56 member SSS TWG has been working on the SSS PTS since early 2009. These companies represent solid state storage industry leading SSS manufacturers, SSS controller makers, test houses, and computer system OEMs.