OK — Big Data doesn't have to suck. But far too often it does. And the reason is almost always the same: execution.
Over the years I've been evangelising the power of data and why measurement matters. In short, it's all about the execution. And this is where my frustration with 'Big Data' — and other vendor-pushed terms — stems from.
I've been working in the analytics space for over a decade, and in digital since the mid-nineties. During this time there have been lots of buzzwords: first web analytics, then Real-Time, then Big Data. The concepts are sound. It's just that more often than not these terms are adopted without any thought as to why they're being used or what impact they'll actually have on the business.
People try to fly before they can even crawl. Get the basics right, and then build on that foundation.
The core problem with Big Data projectsSo why pick on Big Data specifically?
Simply because it's been the current buzzword for long enough. The underlying concept — pulling your various data sources together into a coherent picture — is genuinely powerful and can bring significant insight and value. So what's the problem?
It will only work if your organisation is actually geared up for it. Most aren't. They invest heavily in the technology layer — a shiny data platform or warehouse — before sorting out the fundamentals underneath. The result is expensive, underused infrastructure and a team frustrated by data they can't trust or access.
Meet D.A.V.E.
To help organisations work out whether they're ready for a Big Data project, I use a simple readiness framework. If you can answer "Yes" to all four of the following, you're in reasonable shape. If not, think again before you invest.
Data Access
Do you know what data are available and who owns them? This sounds obvious, but data are often stored in old proprietary systems that no-one knows how to access properly. I've worked with large brands running critical databases on a computer under someone's desk.
Accurate (Meaningful) Data
I prefer the term 'Meaningful' or 'Consistent' data. 'Accurate' implies it's a true reflection of reality — it never is. Many organisations invest too much time trying to get numbers to add up, rather than focusing on whether data are consistently captured.
Value
What is the actual business value of combining these data sources? If you can't articulate a clear commercial benefit before you start, the project will struggle to justify itself once the novelty wears off and the costs mount up.
Expertise
Do you have the people — or access to the people — who can actually work with the data? Big Data platforms create Big Data problems if your team isn't equipped to interrogate them, interpret the output, and act on what they find.
Run through D.A.V.E. honestly before committing to a large data project. Complete a data audit followed by a gap analysis — understand the difference between what you want to achieve and what you're currently able to achieve — and focus your efforts on the most valuable data first. The rest will follow.