Benchmark storage

From Wikibon

The purpose of this how-to article is to hash out an open standard benchmarking system for storage devices. Storage devices are complicated, and this makes it hard to compare one device to another. Unlike systems and servers (which are benchmarked in a fairly straightforward way), storage devices have multiple components that contribute performance (and performance limitations) to different types of applications at different points.

Contents 1 Definitions 1.1 Storage Device 1.2 Host device 2 Anatomy of I/O 2.1 Sequential 2.2 Random 2.3 Read/Write Ratio 2.4 Protocols 2.4.1 FC 2.4.2 Network 2.4.3 Direct attached 3 Benchmarking 3.1 Storage Benchmarking History 3.1.1 SPC 3.1.1.1 Pros 3.1.1.2 Cons and Common Criticisms 3.2 Storage Benchmarking Future 3.2.1 Usefulness 3.2.2 Neutrality 3.2.3 Openness

[edit] Definitions

To start any discourse on benchmarking, a common pool of definitions is needed.

[edit] Storage Device

storage device definition

[edit] Host device

host device definition (operating system, applications)

[edit] Anatomy of I/O

Different applications perform input and output differently.

[edit] Sequential

sequential

[edit] Random

random

[edit] Read/Write Ratio

Read/Write Ratio

[edit] Protocols

List of protocols

[edit] FC

FC FICON

[edit] Network

iSCSI file protocols

[edit] Direct attached

• • note: there is a convergence between direct attached and networked protocols

SCSI ESCON SAS

[edit] Benchmarking

Benchmarking storage is complicated. In a more linear system with less complexity, benchmarks are

[edit] Storage Benchmarking History

specifically SPC

[edit] SPC

history, background, controversy

[edit] Pros

only benchmark available currently, allows companies to see if/when the device they're considering will no longer be able to saturate the back end disks

[edit] Cons and Common Criticisms

not reflective of real life, does not reflect anything but back end device performance

[edit] Storage Benchmarking Future

Proposed new open benchmarks

[edit] Usefulness

Should provide as close to real life comparison as possible, if necessary with many different workload simulators for different types of environments

[edit] Neutrality

Should be tested by multiple parties, the more the better. At least one should be either neutral or a competitor to the vendor (client or competitor)

[edit] Openness

Should allow anyone to recreate the test with the same gear to check the reproducibility of the test, should also allow anyone to create a variant or fork