The evolution of data center storage architecture
A comprehensive collection of articles, videos and more, hand-picked by our editors
Using metadata and analytics, data-aware storage offers much more than IOPS and capacity.
Data storage is usually evaluated along predictable lines: capacity, reliability and throughput. However, a new breed of storage systems claims to embed intelligence within the array to more efficiently organize and manage what's inside.
Data-aware storage, as its been called, is a general term for multiple flavors of technology that aim to make the storage process smarter, more efficient and perhaps even more secure. It is designed to replace primary storage directly and add data-aware features that are "architecturally and generally capable of handling multiple applications on a broader scale," said Jeff Kato, senior analyst at the Taneja Group, in Hopkinton, Mass.
While raw storage has decreased in price, the cost to manage storage has, in effect, been rising as storage volumes grow and as issues such as compliance become more important, Kato said. Meanwhile, a confluence of factors such as virtualization, flash storage, and more processing cycles make it technically possible to embed the necessary intelligence within the storage array to provide awareness.
Of course, data-aware storage isn't the first variation on the generic storage theme. Over the past decade, technologies like auto-tiering have helped to move less active data to lower-cost storage, Kato said. Other types of software have been fielded to automatically expunge files at the end of a defined retention period. Then there is application-aware storage, which is supposed to monitor and optimize storage to match the needs of specific applications. But all of those innovations were relatively narrow in focus and limited in what they provided.
Today's data-aware storage players have more ambitious goals -- providing insight and analytics on massive data sets, or improving compliance and security.
Seattle-based Qumulo, which makes network attached storage, for instance, "uses data-aware analytics to solve massive scale-out data-sets in the petabytes range," Kato said. So if a business is building a big data solution and needs to share that information to lots of users, Qumulo uses real-time analytics to manage data at that scale with billions of files.
Atomic Fiction, a visual effects studio based in Los Angeles, had repeatedly set up custom applications to deliver needed insights to manage and care for the enormous files used in its business -- and to help account for IT costs -- "some successful and some not", said Robert Jordan, head of systems at the firm.
With Qumulo, "we are very excited to finally see an intelligent way to access metadata that we have previously had to do in other ways," Jordan said. The development makes it easier to track costs on a per-project basis. Furthermore, the real-time nature of reporting makes it easier for both staff and users to assess storage, and to delete or reorganize.
"Honestly it's about time," Jordan said.
For Nashua, N.H.-based storage vendor DataGravity, the focus is on highly regulated customers or organizations with sensitive customer information. "They use real-time analytics to recognize if someone should not have accessed a file as well as whether a file contains any personally identifiable information," Kato said. DataGravity is more oriented toward mid-sized customers and is very successful with legal firms, he added.
But the DataGravity approach resonates with non-regulated environments too. Jim Cutrona, senior systems engineer at Encore Wire, a McKinney, Texas-based manufacturer, was in the midst of bidding traditional commodity storage when he happened on DataGravity. He said he was pleased to discover that the DataGravity product offered so many useful insights into storage and, by implication, into other aspects of IT operations without costing more than the commodity alternatives.
"I was sick of having people sell me IOPS [Input/Output Operations Per Second]. Now that I have DataGravity, I find myself in the data all the time," he said. Cutrona likes the file analytics and the performance charts that DataGravity provides and "when a user calls to complain about a missing file I can tell them exactly when they deleted it -- and ask them if they would like me to recover it for them."
Applying data awareness
Data-aware storage has piqued a lot of interest, said Scott Sinclair, storage analyst at Milford, Mass-based ESG. "What is most compelling is the ability to double click into a specific file and identify high security components in that information -- things that may have escaped the attention of compliance folks or security policies -- and this works in real time," he said.
And there is an obvious benefit to consolidating capabilities in an appliance -- as both DataGravity and Qumulo have done -- rather than through a bolted-on approach. "There are benefits and some challenges; because this is in its infancy organizations are still trying to figure out how to use it, which aspects are useful, what are is the TCO and ROI across various processes, and whether some of this is just a gimmick," Sinclair said.