Problem solve Get help with specific problems with your technologies, process and projects.

How to handle big data in enterprise IT environments

Database appliances are emerging as unlikely heroes of big data handling in enterprise data centers, not least of all because IBM acquired Netezza.

New database appliances are surprisingly integral to big data handling in a notably easy-to-implement package. Instead of typecasting Netezza, Exadata and the like as old-style appliances, consider them for a broader role in the enterprise's information architecture.

New software technologies such as columnar and in-memory databases offer fine-tuning capabilities to rival database appliances like those from Teradata and Netezza, and new hardware technologies seem to favor open and loosely coupled scale-out approaches to data. However, data virtualization software users and IBM customers are enthusiastic about database, or data warehouse, appliances such as Netezza. What's happened?

Not yesterday's appliance

I have always been a bit contrarian on appliances -- fully integrated pieces of hardware and software optimized for particular use cases -- because they tweak processors, storage infrastructure and networking hardware to achieve better scalability in the short run. This isolates the appliances from the flow of data center technology in the long run, particularly if they are not associated with a large, savvy hardware company.

My concern about appliances in the data center might be outdated, however. IBM's acquisition of Netezza removed one concern: that the chipset or other hardware was not going to keep up with the pace of innovation. At first I assumed that Netezza-cum-IBM was just another appliance, focused on particular use cases and not delivering major benefits beyond them. But IT professionals and independent consultants say Netezza suits a very broad set of use cases, and its ease of implementation is an important part of its value-add in most cases. Netezza's reputation in some quarters is just plunk it in, no tuning or app rewriting necessary, and it runs like a bat out of hell. This is the first time I've heard an appliance described in this manner, and it's an attractive way to handle big data needs.

Netezza extends to various hardware and software architectures, from the mainframe to mainframe-plus-Linux-blade "system of systems" to pure Linux/Windows scale-out. Netezza answers the question of how to handle big data with raw scalability and, according its designers, some flavor of in-memory and columnar technology. IBM Netezza is broadly applicable, easy to implement and far ahead of the performance/scalability curve, according to users and IT consultants. In fact, they recommend Netezza for big data analytics initiatives in large- and certain medium-sized companies, at least in in the short and medium term.

Appliances as tactical 'everytools'

Data warehouse appliances have lower administrative overhead, higher robustness, and better performance and scalability than other big data quick fixes, according to users that spoke at a data virtualization conference. Insert an appliance for enterprise-application handling to complex querying, then forget about it and let it work.

Appliances aren't far off from data virtualization solutions such as Composite Software and Denodo: adaptable, easy to implement and useful. With this new flexibility in information architectures and vendors committing to hardware technology updates, appliances are uncommonly future-proofed. As IT begins to commit to budgets for the next year, consider appliances to support key enterprise big data and analytics initiatives, even if the appliance vendor is not the dominant vendor in your enterprise data center.

About the author:
Wayne Kernochan is president of Infostructure Associates, an affiliate of Valley View Ventures. This document is the result of Infostructure Associates-sponsored research. Infostructure Associates believes that its findings are objective and represent the best analysis available at the time of publication.

Dig Deeper on Data center capacity planning

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.