You could replace any disk storage system in the data center with flash, but should you?
The range of flash storage devices available to modern data centers includes all-flash or disk arrays with flash-based caching, server-side flash, flash as part of a hyper-converged system and variations within each technology.
Pure flash players such as Pure Storage and Violin Memory have made a great deal of noise* about their arrays replacing any existing data center storage system.
Incumbents in this hardware arena -- EMC, NetApp, Hitachi Data Systems -- take a slightly different view, giving flash a part to play within an overall storage architecture, not necessarily replacing it.
The best flash storage choices depend on expected use, the price maximum and storage minimum for the data center, as well as architectural limitations and performance requirements.
Flash storage arrays
In raw storage capacity, magnetic disks still win. Although solid-state drive (SSD) densities are improving, bulk-manufactured products are still stuck around the 1 TB level. Magnetic disks of up to 6 TB are produced in bulk. SSDs are also considerably more expensive than magnetic disks. However, in a flash-based array, intelligent data management narrows the cost difference to a generally insignificant level for comparable data workloads.
Flash inherently operates faster than magnetic disk. Rather than spinning to find data bits, flash systems work on the silicon. Without latency to find and read data, flash achieves much higher IOPS than disk.
Some magnetic disk arrays compete with all-flash arrays by using a thin layer of flash storage as a data caching mechanism. Intelligent data management technology identifies "hot" (frequently accessed) data and sends it to flash, while the cooler data is stored on disks. If the data management is poor, the array will have a slow response time as it searches flash, and then disk, for a piece of information. Also, if the main performance issue is around data writes, rather than reads, then a thin layer of flash is not a good idea.
The future for magnetic disk
Eventually, disk will be the new tape -- storing near-line data archives rather than active data. In the near to mid future, however, disk isn't going to disappear. Entrenched storage vendors continue roadmaps of ever-faster spinning disks, and the technology has some characteristics that make it viable for many data centers.
Flash memory ages differently than disks. Disks fail from physical issues with heads, platters, motors or actuators, none of which exist on an SSD. Each time data is written to a flash system, it goes through a program/erase (P/E) cycle. Flash substrates can only take a certain number of P/E cycles, though vendors use various techniques, like heuristics, to improve overall array life.
The main type of flash storage, multi-level cell, fails regularly. A different means of treating the substrate led to enterprise multi-level cell (eMLC), with enhanced life and capabilities. Single-level cell flash is faster, but has a lower storage density than eMLC, and costs a lot more. Some vendors consider SLC as a potential storage tier above eMLC, but most flash arrays use eMLC as the main workhorse with possible additional flash capabilities to enhance overall performance.
Rather than investing in a flash storage array, data centers can use flash on servers. The flash drives are connected to motherboards over PCIe cards, optimizing data rates to the CPU. This boosts performance but it is dedicated to a single server, which makes resource pooling difficult. An additional option is to use server-side flash virtualization, such as PernixData FVP.
The next level is flash on DIMM, with faster interconnect capability. Flash on DIMM sits in the compute system like a standard memory chip. This technology is persistent -- data can reside in the DIMM even when it is not powered on.
Some flash-based array vendors say that storage tier management is not important to flash due to its massive performance advantage over disks. This is not the case. A hybrid system of flash-based arrays alongside server-side flash as PCIe and DIMM systems will need very careful tier management.
About the author:
Clive Longbottom is the co-founder and service director of IT research and analysis firm Quocirca, based in the U.K. Longbottom has more than 15 years of experience in the field. With a background in chemical engineering, he's worked on automation, control of hazardous substances, document management and knowledge management projects.
*Editor's note: A marketing blog by Violin Memory states that all-flash data centers have the potential benefits of new sources of revenue, lower operating costs, simplified operations and more. Similarly, Pure Storage promotes all-flash data centers as a way to accelerate business and drive new initiatives. Meanwhile, EMC advocates pragmatism around flash, and Hitachi talks about integrating flash "seamlessly" with existing storage technologies.
Explore the flash storage vendor offerings, pros and cons and different technologies in detail with a guide to SSD implementation.
No matter your path with flash and disk, don't let storage capacity go to waste in the data center. Discover locked-up capacity in your vendor mix.