Seven steps to conquering the art of mining big data

The most effective marketing relies on the invaluable insight that data can provide. Here Mhairi McEwan sets out the "seven Es" novice marketers should follow.

The art of mining big data
The art of mining big data

Marketers can't be expected to be data experts, but need enough working knowledge to be able to work with data specialists whose numbers are swelling as recognition grows of the value of the insight that data, properly interpreted, can bring.

In this first Masterclass, we'll cover the fundamental concepts of mining big data as a checklist to see where you may need to raise your game.

To keep it practical, we'll refer to Barack Obama's 2012 US Presidential campaign as a powerful demonstration of the competitive advantage that mining complex data can deliver.

Importantly, too, there were great measures in place to secure buy-in and understanding of the sophisticated data-led approach beyond the data-crunchers, across the whole Obama campaign team.

1. End Goal Define clear objectives. What do you want to achieve?

In Obama's case it was to raise $1bn.

2. Explore Collect and familiarise yourself with existing data and start developing hypotheses.

Early on, Obama's team identified that being overwhelmed by massive, unconnected databases would seriously hamper campaign effectiveness.

3. Engineer Create an overall master database by tabling, recording, transforming, cleaning and refining.

Campaign data scientists merged data from the most important sources to create one powerful, practical, yet connected, workable system.

4. Experiment Play with a range of modelling techniques to optimise and identify which ones are best predictors of behaviour.

Obama's data chiefs ran a huge number of tests to identify how different hypotheses would play out, such as the way in which different voters would respond to different triggers and types of campaigns. This allowed them to see who would be most likely to volunteer, for example, or respond best to online, as opposed to direct-mail, appeals.

5. Engage Visualise relevant data patterns to inspire and aid understanding.

Data-crunching delivered all sorts of insights. Among these was the ability to identify the easiest targets, for example people who had underlying interest but had unsubscribed from 2008 campaign email lists - these received personal attention.

6. Evaluate Thoroughly test models to ensure they meet your goals.

A range of approaches was tested: emails with different subject lines, senders, messages etc to see which combinations would raise most funds and drive best support. When it came to visualising the campaign outcome, the data allowed Obama's team to run the election 66,000 times a night.

7. Execute Translate what you know into focused action.

Insight influenced how the campaign played out externally. It led to novel media planning and buying that moved beyond local news programming-placement. It also drove Obama to answering questions on the relatively little-known platform Reddit.

The overall impact was a fundraising result of more than $1bn. Objective secured.

Mhairi McEwan is chief executive and co-founder of Brand Learning.

The "seven Es" are adapted from the Cross Industry Standard Process for Data Mining (CRISP-DM).