Naked Marketing: The Big, Beautiful Bubble Chart – Nailing Marketing Attribution
Okay, by now you’ve discovered that the people behind this big data marketing program are all geeks. And that I might just be the geekiest of all.
This post will not disabuse you of that notion.
Because this one’s about a great new chart we’ve created that does things we’ve never seen before: the Marketing Influence Analyzer, also known as the Big Beautiful Bubble Chart.
It’s also about a new attribution model that we feel is better for B2B than the ones we’ve seen out there. But first, the graphic:
More than a pretty face
Like every great graphic, this one is more than a shiny visualization. It’s actually the pretty face of an underlying analytical framework that lets us see how our different marketing channels and campaigns (‘programs’ in Marketo terms) perform relative to each other.
The idea is to clearly visualize the answer to the question, “What works?” – and to do it in such a way that all of our marketing stakeholders – product, field and corporate marketing in this case – can be in the same discussion around the same metrics.
Why we needed to do this
Marketo already does its own version of an Opportunity Influence Analyzer but it didn’t do what we needed.
Marketo’s view is a nice way to visualize the touchpoints on any given prospect’s journey as the prospect accumulates engagement points and becomes increasingly sales-ready.
But this view is tied closely to the Marketo ‘Costs Month’ concept, so it only reflects programs where we put cost information. It is cumbersome to maintain cost for all programs in Marketo so we decided to innovate on our own. The model is also based on simplistic attribution math that gives equal weighting to every touchpoint on the buyer’s journey. We didn’t like that, we wanted the ability to either do custom weighting (e.g. first touch gets a higher attribution) or attribution based on a statistical model (which we developed/borrowed from a pharma algorithm analyzing patient survival rates in drug testing). Finally, the out-of-the-box Marketo analyzer didn’t let us answer these questions:
- Which campaigns or channels had the highest impact on opportunity creation and closure in Salesforce?
- What’s the ROI for paid campaigns such as Search, Display, and Remarketing?
- What campaigns or programs are the best at new names acquisition?
- Can we report on campaigns/channel performance by different dimensions such as time, region, themes, products, etc.?
Those are a lot of important questions to ask of one chart. But the Big Beautiful Bubble Chart rises to the challenge.
The joy of bubbles
Here’s a wide view of the Bubble Chart:
It may not look like much but after a brief tour, I think you’ll see why we love it (and why you should have one to guide your own marketing).
Here’s how it works:
- The X Axis: Campaign Members – the number of prospects included in the given campaign, program, or tactic.
- The Y Axis: Pipeline Created – the value of the opportunities (closed or won) that each campaign touched. This uses our own, multi-touch attribution model. More on this below.
- Size of the Bubbles – the degree of success for each campaign (bigger is more successful, or more actual wins).
- Color of the Bubbles – how successful each tactic is in acquiring new names (darker is better).
- The Tabs – this chart is actually a lot of charts; the tabs let you switch from the Pipeline By Channels view to, say, the Revenue Won by Channels – or either of these by Marketo program (specific campaign) instead of by channel.
- The Filters – a powerful feature that lets us slice our data any way we like. So we can look at the influence of different tactics by product or solution; by specific medium or tactic; by type of campaign, etc.
The Channel View
The above chart shows the aggregated view of all our data by channel: email, webinars, events, etc.
Looking at aggregated data like this is great for comparisons and indicators – it doesn’t have to be perfect to the third decimal point to deliver a lot of value.
In the Channel View, we can see that one channel, email, leaps out of the pack for dollar value of multi-touch pipeline created. We would expect this as we do more email than any other tactic.
But bubble size and color tell a different story. The chart shows us that web assets (gated content on our site) are critical for acquiring new names (the bubble for web assets is darker green) and are just as powerful for driving actual successes as email is (see the size of bubbles for both web assets and email).
This is where the power of a clear, multi-dimension visualization comes into play. It would be easy to over-index our budget on email if we only looked at a simple metric, like pipeline created. The added nuance of success-driving and new name acquisition tell a different story – and we see them all together in one view. Email marketing wouldn’t be much fun in the long run when running out of new names in the database …
Drilling down: by program
The Bubble Chart is the data visualization that just keeps giving. Here’s a drill down: Multi-Touch Pipeline Created by Campaigns (or Programs in the Marketo parlance):
The snapshot here appears almost random, with no discernible pattern. It’s hard to see that any program is working better than any other.
Why? Because email is swamping out every other program. We’ve got more than half a million members opted in to our email program, while, for comparison, webinars only have sixty thousand program members.
So we created another Pipeline By Channel view that excluded email:
All of a sudden, some clear winners emerge. Web assets (content) and webinars leap out of the pack.
Web assets is a winner in all metrics, pipeline created, members, successes, acquisitions and new names (Velocity, our content marketing agency, loves this view).
Drill down again: specific content
So we know that content is our killer program. But which content performs best? Click a filter and we can see:
Way off in the upper right is our Data Integration For Dummies eBook. It’s killing it on every metric. Next best are some of our third-party, research-driven content like the Gartner Magic Quadrants. But we can also see some home-grown content winners here.
An important caveat: promotion
One thing that the Big, Beautiful Bubble Chart does not yet include is a sense of how much promotion was put behind each tactic, program, or piece of content.
This is, of course, a critical input in any ROI or budget optimization discussion. For instance, the For Dummies book was featured in display ads, retargeting, social media, and on our home page. So you’d expect it to do well.
Right now, we manually compare the Bubble Chart with our promotions program to analyze ROI. But we’re looking into a way to include the promo spend in the Bubble Chart so we can control for this important variable.
How we do attribution: a new model
In Marketo’s Influence Analyzer, each touch on the way to becoming an Opportunity is given equal weighting.
Other models we’ve seen give all the credit to the last touch, the thing the prospect did just before becoming an Opportunity.
Still others use a first-touch attribution model that gives most or all of the credit to the thing the prospect did that got them into the funnel.
All three of these felt far too arbitrary for the long, complex, multi-touch buying processes that most of our customers go through. We plugged each one into the analytical framework that drives the Bubble Chart and we just didn’t feel good about what we saw.
So we sat down with our data scientist, John Teifel and came up with something better. It’s called the Blended Time-Decay and Positions Model and here’s how it works:
- The ‘Position’ part of the model gives more credit to interactions that happen just before an Opportunity is created and closed-won. So if you read an eBook that tipped your name into Opportunity creation, that eBook gets a big chunk, say 35 percent, of the opportunity credit.
- The Time Decay part gives decreasing credit to all other interactions as they go further back in time. So an email open that happened last week might get 20 points while an email open from three months ago might only get 10 points (shown as dollars in the chart below).
It looks like this:
Is our model perfect? Of course not. But we feel it represents our actual buying journeys a lot better than the equal-weight, first-touch or last-touch models out there.
A survival model
As we got really excited about a more sensible way of judging the value of the different offers and tactics, we felt compelled to take another step and climb the Everest of attribution modeling—move from correlation to causation. Just because two things are correlated doesn’t mean that one is a cause of the other.
After some research, John zeroed in on a survival model used in the pharmaceuticals industry to establish a causal relationship between drugs given and survival rates of the patients. As long as you have a large enough pool of events (e.g. patients given a different mix of drugs along with their survival data), you can use these algorithms to determine a probability for a relationship to causal and how big the lift on survival or death rate is.
When we applied this kind of model to opportunities surviving infinitely (not turning into revenue) vs. the patient ‘dying’ (ironically this is our desired event: an opportunity closed and won), we could see the offers, programs, and channels that have true and statistically-proven impact on revenue.
Here’s a quick example (we’re still working on making this Big and Beautiful)
We’ll keep refining our revenue attribution model over time but you get the idea: if you’re in B2B, you need a way to give credit to the many touchpoints that your prospects take on their way to a purchase.
Any such model will be built on assumptions. You just need to make sure that your assumptions are consciously made and that they’re exposed to whoever will be consuming the analyses driven by your model.
The power of a picture
The Big Beautiful Bubble Chart and the attribution model that drives it have become indispensable to our marketing teams:
- We’ve replaced a vague sense of what works with a clear, data-driven view.
- We’re putting more budget into things that work and less into things that don’t.
- We’re generating more opportunities without spending more money.
- We can defend our investments in our marketing programs and argue for more money from a position of strength.
And remember: the chart and model are built using the same data lake and the same data as the dashboard I talked about – the one our sales guys are so excited about (we interviewed them for post #9). The data was ready; all it took was a little massaging (using REV) and a new set of visual reports.
It’s another example of the power of treating your data as a strategic business asset instead of just the by-product of marketing. Get this right first and there’s nothing you can’t do.
Try this at home
From the outside, things like bubble charts and attribution models look like rocket science to many marketers. What I hope I’ve shown is that they’re really just common sense combined with some simple data management and visualization techniques.
You may not have the capabilities to do this right now, using only in-house resources. But the skills, capabilities, and tools are all out there, ready to be deployed. You just have to have a clear picture of who you’re serving; what data you need to bring in to your analysis; and what questions you need to answer.
Start wherever you are right now. And take that first step.
Naked Marketing, Prologue – “Finally We can Connect All the Dots”
Naked Marketing, Post 1 – “A Big Data Marketing Operations Odyssey”
Naked Marketing, Post 2 – “Who’s Who Behind Our Big Data Marketing”
Naked Marketing, Post 3 – “The 5 Foundations for Big Data Marketing”
Naked Marketing, Post 4 – “The Business Case for Big Data Marketing”
Naked Marketing, Post 5 – “The Big Data Marketing Technology Stack”
Naked Marketing, Post 6 – “Big Data Marketing Checklists for Marketo and Adobe Analytics”
Naked Marketing, Post 7 – “The Data For Big Data Marketing”
Naked Marketing, Post 8 – “The 60-Day Sprint to Our Big Data Marketing Data Lake”
Naked Marketing, Post 9 – “The Sales Leaders’ View of the Marketing Data Lake”
Naked Marketing, Post 10 – “The Account Based Marketing Dashboard”