KMart’s CMO made waves recently by saying TV, and other traditional channels, are easier to measure than social media.

Social media proponents responded, defending social media’s measurability and highlight the data social media provides. My favorite was this tweet from Jay Baer (who I have a tremendous amount of respect for and I don’t disagree with lightly).

CMO at KMART says TV is easier to track than social. I disagree. If you can't track social, you need to try harder. http://ar.gy/Uj2

— Jay Baer (@jaybaer) July 20, 2011

The problem is, most social media measurement is myopic. It focuses on direct interaction and ignores the fact it is just one part of a larger communications mix. My verdict: KMart’s CMO is right.

Using media mix models, companies like KMart assess incremental business impact, accounting for the fact that TV is one of many channels. Unfortunately, this approach doesn’t work with social media today. Here’s why:

Scale

TV has scale at KMart today and larger changes are easier to measure. From this perspective, if TV isn’t easier to measure, KMart has a major problem.

The solution, with a hint of sarcasm, is simple: make an 8-digit investment in social media. But spending $40 million on social media, starting with today’s first baby steps, would be a major challenge. Doing it effectively, without history to learn from, would be a daunting initiative for almost any CMO.

Unit of Measurement

A model requires a consistent input. For TV, the input might be GRPs by target audience and DMA. Social media does not have a similar measurement today. In fact, social media can’t get close. This is a gap social media measurement companies need to help close.

The hub and spoke social media model, with an owned media property at the center, can capture granular audience data from the hub isn’t more broadly available in social media.

Stability (see the next section) will be a challenge with hub activity, but if I were a CMO at a large company, I would at least consider testing data from a hub as a proxy for overall social media reach.

History

A model requires history to provide meaningful results. Two years is usually the minimum, five is far better. It will be another year before most companies can realistically attempt this.

Models also requires stability in the data used. Social media is changing rapidly. Even with a consistent unit of measurement, would it be consistent over time? Would a 2008 social media impression be comparable to a 2011 impression? Early attempts to include social media in a marketing mix model, even once history is available, will initially struggle because of this.

In the meantime, correlations and interactions between channels can begin to be assessed. But when looking at overall business impact, these results are the color commentary, not the definitive answers.

The Challenge

Clearly, these changes will not happen overnight. Organizations that embraced social media two years ago, on faith or based on direct measurement, have a huge head start. Many are seeing results, in anecdotes and direct activity, and making the case to continue investing.

Social media measurement continues to develop, but the continued reliance on clickstream and social monitoring generates mountains of data without getting to real incremental business impact. This approach is plagued by two systemic measurement problems:

• Undermeasurement: It ignores influence communications have beyond an individual. If someone emails a link or tells a friend, it is unaccounted for.
• Overmeasurement: It disregards the influence of any other communication activity. It doesn’t consider the possibility that someone saw a TV commercial and responded by visiting your Facebook page or saw a recruitment billboard and looked up your company on LinkedIn.

Human behavior isn’t cut and dry. Using clickstream data implicitly assumes it is.

Your Turn

What will it take for major corporations to assess the impact of social media on top line and bottom line business results? Is there an alternative to marketing mix models? Is there a way to control social media distribution in order to test within paired markets?

Or are the current tools sufficient, and as Jay’s tweet implies, I’m just not trying hard enough? Share your thoughts in the comments below or with me on Twitter.