Digital Marketing’s Massive Measurement Lies

The Marketing Measurement Card TrickDigital marketers are getting hooked on reporting bigger numbers, but in the process your marketing may actually be making a smaller and smaller impact.

Marketers are making a land grab, including in their reporting big swaths of results that were not driven by their marketing efforts. Many even count the same results twice, all to show that marketing is contributing to results. No wonder 8 in 10 CEOs don’t trust the work marketing is doing.

The problem starts when you begin measuring your programs based on the companies or individuals you reached and what they did. Unfortunately, this is the easy, and almost universal, way to measure digital marketing today.

With all of the data available today you can identify more of the people reached through your marketing than ever before. In turn, that means your reporting is more inflated than ever before. Yikes!

Here are four common culprits.

1. Branded search

If I’m looking for a computer, search for Dell, click the top organic result and buy a new laptop, is search the reason I bought a laptop from Dell? No. I was already looking for Dell, search was simply an easier way to navigate.

When you report this as a sale driven by marketing, you are taking credit for a sale even though marketing didn’t cause it. In my particular case, it would be because I have had two very long-lived Dell computers. My next computer will likely be a Dell because of that experience, not because of my navigational search.

How important is search as a navigational tool? According to Wordstream, the #1 and #2 keywords based on search volume are facebook and youtube. The top 25 includes google, hotmail, facebook login, yahoo, ebay, yahoo mail, craigslist, aol, gmail, and amazon. That’s a lot of navigational search.

Unfortunately Google makes it difficult to identify branded search traffic today, ensuring marketers will continue to credit search for what would have happened regardless.

Sorry to break it to you, but brand followers on social media have the same bias.

2. View-through Reporting

I still remember when DoubleClick first rolled out view-through reporting. In one day our reported results increased by an order of magnitude across nearly every client we were working with. What had really changed? Just the reporting methodology, the actual impact of our marketing efforts hadn’t changed one bit.

Marketers today still report on view-through, or post-impression, results. Some even still use the default 14 or 30 day reporting window many ad management providers still default to. This introduces two major issues:

  • Double counting results. How did think that view-through visitor got to your site? Many turned to Google. A few may have been on your email list. But since most B2B marketers measure view-through results in an ad management platform using cookies that cannot be read by the rest of marketing technology stack, the same activity can be independently attributed to two or three different marketing activities.
  • Counting what would have happened anyways. If you have a well-known brand and run a well-targeted campaign, you are reaching people who would buy from you without your marketing. In controlled tests, I’ve seen programs where 95% or more of view-through activity was simply counting what would have happened anyways.

And don’t forget that you are reporting on view-through activity from ads that often aren’t even viewable!

3. Marketing Influenced

Like view-through reporting, marketing influenced revenue, sales or leads counts who your marketing had the chance to reach, without any indication that your marketing made a difference in the outcome.

To your credit, the label at least indicates that your marketing wasn’t responsible for the activity, but you improve this metric by increasing your volume of marketing activity, not improving the performance of your marketing.

This is the equivalent of a supermarket buying a billboard in front of their store and then claiming the billboard was driving all of their sales. Let’s stop the ridiculousness.

4. Retargeting

Retargeting is nearly de rigueur for digital marketers today but it will almost certainly contribute to over-counting your results.

You are marketing to people who have already found their way to your site! Many are customers already!! If you managed to completely suppress this group from all of your marketing, they would probably still be far more likely to come back than the average person you reach through your programs.

If you add view-through reporting to retargeting, you can nearly guarantee that you will be double-counting.

It’s Time to Forget Attribution

For marketers that want to earn the trust of the CEO and the business, put aside attribution. Fancier counting isn’t nearly enough. It is time instead to focus on understanding the difference marketing actually makes in your business and then start improving those results.

The comments are yours.

Photo Credit: stevendepolo via Compfight cc

Get every post delivered directly to your inbox.

Your email address will not be shared or sold. I hate spam too.

  • Nick Wassenberg

    Solid post Eric. A couple points for follow up:

    - #1: Why couldn’t marketers take out those branded keywords of their analysis? There are a number of ways to avoid overstating the importance of navigational search. You make a good point, but I wonder how prevalent that actually is for marketers to report? Not saying it doesn’t happen, just not sure how many marketers take “full credit” for that effect. Part of search marketing is defending those branded keywords, so marketers do have a job to do in those cases.

    - #3: Wouldn’t having better content mean better influence? (i.e., a video that is part of a nurture sequence that helps a lead understand the UVP better). You state that “you improve this metric by increasing your volume of marketing activity, not improving the performance of your marketing.” – not sure this jives. Marketers can measure the content/method of influence not just the # influenced. I agree that the tendency is to oversimplify – that layer of detail behind volume is important.

    - I think your main point that metrics that are dubious will eventually get called out, and rightfully so. Is “who gets credit for the conversion?” the right question? We all know it is a hard question to avoid, understandably so. And as channels and tactics get more seamless for the customer harder to really pin down.

    Like I said, solid post – thanks.
    Nick Wassenberg

    • http://b2bdigital.net/ Eric Wittlake

      Thanks Nick.

      I believe the right question to ask is what difference did our marketing make for our business. It is a difficult question to answer, but we can get a lot closer than we do today. To your specific questions:

      #1: The data isn’t available to do this anymore since Google started stripping keyword data out of the referral information sites get. You can still make some extrapolations based on old conversion rates by branded and non-branded, but as that data ages those extrapolations become less and less valuable.

      #3: YES! Content does have an impact. But unfortunately the “influenced” metric I’m seeing more and more frequently is simply counting how many prospects marketing reached that ultimately purchased. It doesn’t touch on causality at all.

      To be clear, I’m not knocking the underlying marketing activity in any of these cases. It does make a difference. The problem is how we measure, and then how that measurement directs how we manage our marketing.

      Thanks for the great comment Nick!

  • Casey Carey

    Eric -

    Excellent job of identifying some of the blind spots/weaknesses common place in today’s marketing measurement approaches. “Lies” seems pretty harsh in that it would imply intentionally misleading. Unfortunately, I think many of us are just trying to do the best with the technologies and resources we have available. Of course we should always strive to do better.

    I was left a bit wanting with your conclusion. What should we be doing differently? There didn’t seem to be a clear call to action or “things to do right now”? It sounds like you are advocating incrementality and causality as primary methods of measurement? If so, I can’t argue with your
    conclusion. I do wonder though if so many are struggling with moving beyond basics, what are the chances of success with what is an even more sophisticated approach?

    A couple thoughts/things to change regarding your 4
    culprits:

    1. Branded Search – At least on the organic side, you are right in that we have lost most of the visibility into how this impacts conversions. One approach is to exclude
    non-ad events from your attribution model (last-ad event), thus dropping
    organic search, direct load, and non-paid referrals.

    2. View-Through – Impressions and views are a
    powerful data points to include in your attribution model and it gives you a much more complete view across channels and throughout the conversion path. However, if you are using them, you have to move to a multi-touch or fractional allocation attribution model. In no circumstances should your attributed results be greater than one. Again, not too hard to do and no excuse not to do it.

    3. Marketing Influenced – As a B2B marketer with a long considered purchase process, this metric is meaningless as we touch every conversion event multiple times. And yes, we are actually reporting “touched” rather than influenced. In an ideal world, we would have sufficient volume and data to create hold out tests and model true influence. But since we can’t I would advocate stop measuring this and focus on more meaningful objectives like net new leads or engagement actions.

    4. Retargeting – Unfortunately a lot of retargeting is being done very poorly, including no frequency caps, no exclusions for purchasers, and little or no segmentation. The vendors get paid on volume, so there is no incentive for you to do it smarter because the usually means doing less. There are two approaches to address your concern:
    · Divide your measurement into three buckets, new to site, returning to site, and existing customers. The key metrics, channels, and attribution models for each of these can and should be different.
    · Many retargeting platforms have built-in hold testing validate incrementality, if yours does, use it. My experience is that it is very effective on driving incremental business.

    I guess my point is, it is not attribution that is the culprit, it is our application of it and other measurement/marketing methods.

    Keep up the great work.

  • Pingback: Digital Marketing's Massive Measurement Lies | ...()

  • http://www.digitant.co Digitant

    Hi @wittlake:disqus , I would like to add one more points to this i.e: “sentiments or emotional” with services bcoz they are directly influence Brand, conversions, re-targeting & many more.

    thanks.

  • Pingback: E2M Eagle’s Eye - 13 Jun 2014 | E2M Blog()

  • Pingback: Digital Marketing’s Massive Measurement Lies – B2D Digital Marketing | The Marketing Technology Alert()

  • Pingback: Digital Marketing's Massive Measurement Lies - ...()

  • Pingback: inspiring single mother quotes()