Data-driven gone wrong: when single metrics breed unintended consequences
No, you didn’t read that headline wrong. No, this isn’t clickbait. You may find yourself asking, “Is a data-driven strategy team telling me that focusing on metrics can result in poor outcomes?”
Yes, yes we are.
Let us explain.
Our story begins on a Zoom call (don’t they all these days?) with the marketing and customer service teams of an e-commerce company we’d been working with. We were asked to help find ways to make more revenue, so one tactic we proposed was to have the customer service reps upsell customers by recommending a more expensive product than the one they were interested in. And, after asking customers a few questions about their needs, we suggested reps try to capture a greater share of their wallet by recommending complimentary products to their order. Low hanging fruit, right? A no-brainer. We’ll just be seeing ourselves out...
We were not prepared for the feedback.
Customer Service Team: “We like these ideas, but we have to keep our call times low.”
Us: “Why’s that?”
Customer Service Team: “Because we’re incentivized to keep our individual call times to a minimum. Our performance reviews factor in this metric, so we can’t spend more time on the phone.”
After duct taping our heads back together, it hit us. Because the team’s customer service reps were rewarded for keeping call times low, they were missing opportunities to:
Create a better experience for their customers by not rushing them off the phone
Deliver more revenue for the business by cross-selling and up-selling
Foster a loyal customer base by adding value beyond simply placing an order
By focusing on a single metric, reps were ignoring other, potentially more meaningful ones that could add value to customers and the business.
This got us thinking…
What other metrics, when viewed in isolation, were having potentially similar, negative unintended consequences?
Visions of marketing teams high fiving each other for getting thousands of impressions and clicks danced through our heads. But then we woke up from that dream and it turned out the company’s revenue was declining and the sales team couldn't hit their numbers.
Flashbacks of penning braggadocios emails to the executive team recounting how successful the last paid social and pay-per-click campaigns performed came to mind. According to Google Ads and Facebook, each campaign generated $1M in revenue last month….only the company’s revenue didn’t match those monster numbers.
The spiral began.
But, but...you did everything right! You implemented an omnichannel marketing strategy! You met your customers where they were! You emailed them. Ran paid social campaigns. Invested in targeted, programmatic display ads. Learned about Google Shopping ads. The works! Why is your ROI inflated?
That’s the beauty and the curse of omnichannel marketing. When your brand is everywhere it’s hard to draw a clear line of attribution to see what’s working (and what’s not). Social platforms want to optimize attribution so you keep boosting those Facebook posts. Google Ads wants you to keep bidding on those keywords. And even as Google Analytics has made improvements with their attribution models and you can set conversion windows narrowly, it’s still not perfect.
It made us wonder: as marketers, are we just lying to ourselves?
Maybe. But is there anything we can do? How do we not fall into the marketing confirmation bias that what we’re doing is working, while our business metrics don’t reflect the meteoric numbers we’re seeing in our marketing dashboards?
Should we just ignore our marketing ROI?
No. But we don’t think you should blindly accept the data, either. The amount of data we can collect these days is amazing, but it has to be paired with context, insight and well-planned objectives or it’s useless.
Well, where do you start?
Your marketing metrics are awesome, but are they reflected in your business metrics? Now more than ever, marketers are asked to be more data driven and are being held accountable for the success of the business. That means you can’t bury your head in the sand and simply brag about the impressions and clicks your campaign scored. As much as you have your finger on the pulse of your marketing metrics, you need to have an even bigger obsession with the business metrics. How’s the health of the business? How do your marketing tactics ladder up to that? How can you get plugged into the health of the business consistently so you can develop marketing strategies to actually boost revenue?
Which attribution model are you using, anyway? While attribution models deserve a blog post of their own, it’s important to note here. How are you calculating attribution? Last click (hope not)? First click (hope not)? Linear attribution? Position-based? Time-based? What makes the most sense for your business? How close can you get to an attribution model that’s a decent representation of your company’s business performance?
Are you A/B testing your marketing tactics? Performing holdback tests? The best way to determine what’s working when you’re in the thick of your omnichannel marketing program is to do the hard thing -- strategically “turn off” marketing initiatives to measure the impact of not running that campaign. The key here is to pull enough levers to derive meaningful insights, but to test in a way that doesn’t hurt the business long term. Plus, when you’re able to isolate and validate the efficacy of one marketing tactic, you can justify the budget for it to the folks on the executive team and make it harder for them to cut spend.
The bottom line? Don’t look at marketing metrics in a vacuum. Data that you can pull from tools like Google Analytics can help marketers make adjustments to their omnichannel marketing strategy when a vehicle isn’t performing as expected, but don’t be like that customer service team we mentioned. Pair your marketing metrics with an obsession with your company’s business metrics and a consistent testing strategy to make sure your marketing metrics don’t let you lie to yourself.