Big Data ? Big Deal…

I’m convinced that long after the lights have been switched off on the yawningly dull public vs private debate, today’s progressive enterprise who has reasoned with, understood and accepted that there is absolutely and categorically no textbook answer to the dizzying and hard-to-demystify rhetorical questions around cloud and where one should put one’s stuff, will be sufficiently advanced to have moved from “thinking” to “doing” and the fruits of their labor, at least on the surface, should be starting to show.

Assuming, for the sake of this post, that my supposition is correct, the enterprise may now have become the custodian of an application and service portfolio of various sizes, which although perhaps considered more efficient than the previous incarnation, may still appear somewhat of a mixed bag.

If the organization’s capability to execute matches the due diligence and planning effort, such a portfolio may now include a set of consolidated applications running on virtual machines, in both on and off premise locations, along with the usual appurtenances of enabling infrastructure and tools for management, monitoring, self-service, automation, orchestration and governance.

Not a bad start. But, if the general themes that I’ve witnessed in other large organizations hold true, then as we collectively prepare to wave adios to 2011, labeling it another foundational, if not relatively disappointing year, we aren’t yet seeing great swathes of investments in the fundamental and complete re-architectures of line of business applications in preparation for deployment on public clouds (specifically IaaS or PaaS) nor in the wholesale strategic migration of line of business application migrations to SaaS… isn’t it a case of if you always do what you always did, you’re always going have what you always got ?

Well, perhaps not…could, in fact, the enterprise that has successfully achieved at least some of the above actually be sitting in the box seat, ready to take a swing at a strategy to address the latest phenomenon to hit the buzzword bingo halls of our incessantly unforgiving industry…Big Data? Quite possibly.

To me, without context, the words “Big Data” are teeth-achingly dour. I can almost hear the countless wasted words of fist clenching opining, as those with presumed dogs-in-fights work tirelessly to craft definitions (ahead of positioning products) that outline “structured vs unstructured”, “what constitutes big vs small data” and more latterly, why you can’t do without it. Sound familiar?

Add a little context, however, and it’s easy to begin to see how the enterprise that has stuck to their plan and made progress will find that the work they undertook to consolidate the portfolio becomes incredibly valuable as the unit of value shifts from virtual machines to the discovery and presentation of aggregated data contained within. As we look, and in some cases, yearn, for that value, I believe we’re moving past the stuffy concepts of traditional Business Intelligence (BI), which, to many, still comes replete with the unwelcome specter of huge complexity and matching cost.

By consolidating, which I believe is a critical step in any practical enterprise cloud experiment, you may still have monolithic applications, often in silos, but you have reduced the number of moving parts and probably have a very good handle on some key information – which applications are consuming the least storage, the most storage (i.e. generating most data), you know where they are running and equally importantly, who owns them – that’s a very good position to be in.

As the old Management 101 adage goes, you can’t manage what you can’t measure, so how could you have any realistic chance with a comprehensive Big Data strategy if you don’t know what you have?

From my experience, I would suggest that in the lion’s share of today’s traditional enterprise environments, much of the data is of the structured variety. Relational databases are still the lynchpin of many enterprise systems. I still question, for example, the industry numbers of “percent of all servers virtualized vs physical” so if anyone were to try and convince me that there are tons of enterprise applications using NoSQL, I’d raise a quizzical eyebrow or three.

Although there are undeniably more and more sources of enterprise unstructured data (from the more obvious platforms like email to the less obvious ones such as social networks), it would make sense to me for enterprises to begin with what they have in their consolidated portfolio – intelligently questioning disparate data sets to see how many nuggets of data gold are buried deep within the bowels of the organization.

The evolution of big data will certainly be interesting to follow. I’m sure we will have the usual cycle of hype accompanied by a plethora of twists on definitions from visualization to aggregation and velocity to gravity and vendors small and large, jostle for position as the space continues to grow throughout 2012 and beyond.

Recently, I was giving some thought to, and trying to boil down, a simple sentence that an enterprise might use to justify a big data strategy. I ended up with this.

“We want to use the data we have to help the company be more competitive, to win more work, and we want to use the data we generate to be able to execute that work more efficiently.”

Perhaps it’s just me, but this is a much more compelling statement than anything I’ve heard on why enterprise needs cloud. Cloud, of whichever denomination, is moving toward it’s rightful role – being the engine room for tomorrow’s data factory – in which the end product is the delivery of real business value through new and inventive ways of making sense of a whole, huge heap of 1’s and 0’s.


4 thoughts on “Big Data ? Big Deal…

  1. “From my experience, I would suggest that in the lion’s share of today’s traditional enterprise environments, much of the data is of the structured variety”

    While I cant argue your experience, it doesn’t match mine at all, most of the storage that isnt being consumed by backups seems to be in the form of emails and or filesystem data. Traditional structured Database data seems relatively flat in its storage consumption. You might argue that the vast majority of that isn’t that valuable or that its hard to extract meaningful information from it, but some of it (notably spreadsheets containing analysis) are veritable nuggets of gold in amongst the sludge

    I’d be interested in your thoughts on that too.


    • I think that’s a very fair point. I was specifically referencing line of business applications (when I made the observation around relational databases) but you’re absolutely correct, there is a huge amount of unstructured data in file shares, for example, that consumes terabytes of enterprise storage and undoubtedly contains important yet hard-to-surface information (via that data).

  2. I and also my guys were going through the excellent pointers on your web blog while instantly developed a horrible feeling I had not thanked the web blog owner for those secrets. Most of the young men had been for that reason warmed to see them and already have undoubtedly been making the most of those things. Appreciation for turning out to be quite accommodating and for picking these kinds of nice subject areas most people are really desperate to be aware of. Our own sincere regret for not saying thanks to earlier.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s