Modern Infrastructure

Bracing yourself for big data


Manage Learn to apply best practices and optimize your operations.

Embracing chaos theory as part of enterprise IT strategy

Enterprise IT strategy has long been based on straight-line extrapolation, but the industry must learn to embrace chaos if it hopes to thrive.

When it comes to enterprise IT strategy, the basic model most shops have used to guide provisioning and operations is broken. It’s based on the ideal of straight-line extrapolation, which is belied every year by chaos theory and Moore’s Law, as well as less-obvious industry shifts and capriciousness in customer demand. But old habits die hard—especially habits so tied to our natural desire for rational, gradual change.

Both real and fictional situations are often breezily explained through chaos theory. Few are genuinely familiar with the gnarly math of complex systems, but something like the butterfly effect (“A butterfly flaps its wings in Brazil, and we get a hurricane off the coast of Florida”) captures the imagination. Chaos is now a staple of pop culture.

But people don’t really like chaos itself. We like things that are smooth, regular and symmetrical. Things that are orderly, straightforward and consistently play out in predictable ways.

But the world doesn’t work that way—at least, not for long, and not in general.

Euclid, Kepler and Newton built science on absolute ideals, but it only works as long as things can be approximated with simple, linear models. That hugely limits the situations, scale points and timeframes that can be simply modeled.

That’s where things break. It turns out that some things that should be straightforward are, in fact, impossible with simple models. And when things change—when metal wears out and snaps, when markets collapse or when customer expectations and demands suddenly shift—those are non-linear and chaotic phenomena. If you fail to systematically account for change because it’s exceptional, you’re not going to genuinely anticipate what’s ahead. Disruption is where fortunes are made or lost.

The IT-Chaos theory connection

Great, but what does this have to do with enterprise IT? Well, essentially all the interesting industry shifts of recent decades have been chaotic. The Internet. Virtualization. Smart mobile devices. Cloud. Sure, futurists and visionaries may have predicted such things would one day change everything. But saying real life will eventually be like Star Trek is the easy part. Accurately stating when it will happen, at what pace it will unfold and what specific things you need to buy, build or do to capitalize on it—you know, practical insight—is surprisingly hard.

We say something’s not ready for prime time -- then boom! It’s here, and everyone’s scrambling to figure out how to navigate. Two or three years later, everyone’s accommodated the shift—and forgotten how disrupted they were. We slip back to expecting a neat, linear future. Then we’re surprised again.

As Yogi Berra observed, “Prediction is very hard, especially about the future.” Chaos theory says it’s not just hard, but impossible. Small, seemingly random changes now can greatly change outcomes and completely reshape longer-term trends. So you can rationally extrapolate only over very short timeframes: a few months, or a few quarters. Beyond that, deltas accumulate, which completely change the shape and composition of events. Yet how often do we demand and depend on specific, quantitative predictions one to five years hence?

Current IT strategy use capacity planning, capital and project planning, and labor development techniques that assume the future will be pretty much like the past, plus a 2% to 20% annual change.

So how does one embrace chaos? The Agile development and cloud DevOps communities show the way forward.

Agile fundamentally rejects the notion that we can know what customers will want in the future, replacing it with an iterative model that constantly reexamines requirements and what to do next? Change, and a process to accommodate it, is built in.

DevOps similarly rejects the idea of static data centers and workloads, and doesn’t assume that systems are stable and reliable. Instead, it assumes that everything may change or break—so systems and other infrastructure had better be easily built and deployed in rapid, repeatable, ideally automated ways. It monitors what’s happening in real time, and seeks to provide intelligent automated responses—for example, automatically scaling apps up or down as circumstances change.

Many people pay lip service to the idea that change is the only constant, but do they really believe it? We’ve often built and invested in enterprise IT strategy in ways biased toward a static, stable world, when it demonstrably hasn’t been.

The good news is that we have a growing body of best practices for how to operate and invest in a change-dominated world. Here’s an exercise for you: Where along the continuum of embracing chaos, from lip service to genuine investment, do you fall? And is that where you want to be?

About the author
Jonathan Eunice is principal IT adviser at analyst firm Illuminata Inc.

Article 11 of 11

Dig Deeper on IT compliance and governance strategies

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Modern Infrastructure

Access to all of our back issues View All