How twisted are your statistics?
Subscriber Benefit
As a subscriber you can listen to articles at work, in the car, or while you work out. Subscribe Now
Mark Twain’s take on prevarication remains as accurate today as when he penned it: “There are three kinds of lies: lies, damned lies, and statistics.”
We live in an age in which we’re overwhelmed by information, and a large share of what we consume is in the form of statistics. A key reason for that is many people find comfort in numbers, primarily because they’re definitive and absolute. Fifteen of something is fifteen. $4,768,387.14 is a specific amount of revenue. 240,000 square feet is an inarguable amount of space.
Given that, it’s no surprise we assume statistics are accurate and trustworthy. When a company tells us their solution will save us 22 percent on our energy bills, we count on it. If someone says four of five people prefer chocolate over vanilla, we don’t question it. When a Senator tells us one-fifth of a particular federal budget is being wasted, we get angry. But accepting statistics at face value can be dangerous. Statistics rarely provide accurate, consistent, or satisfying measures for the important information they attempt to convey.
That came to mind when a nonprofit’s social media vendor sent statistics to include in the group’s report to key stakeholders. The vendor claimed its social media campaign potentially captured over 10,000 million views. Applying grade-school math simplifies that number to 10 billion, which is indeed extraordinary.
But experience makes me suspicious of statistics, so I rarely accept them at face value, even when a client vouches for their accuracy. What made me wary? Start by considering that the population of our planet is about 8 billion and only about 12 million of those humans live in the state where the nonprofit operates. If the vendor’s numbers are accurate, every one of those 12 million folks saw the social media message 833 times.
You already know that’s unlikely, especially given the fact that many of those residents don’t access social media. A large percentage is babies and toddlers, for example, and so far nobody has created a social app for those age groups. (And please don’t.) If we simply assume half the residents spend too much time online, each of those residents interacted with the message 1,666 times.
What’s the problem? The numbers are simply not credible, and that’s deadly, because if the nonprofit were to present those statistics as accurate, it calls everything else they’re saying into question. Promoting bogus numbers will erode the trust they’ve worked so hard to establish with their stakeholders. Plus, there’s that troubling word “potentially,” one of the great weasel words. Hey, potentially, I’m the best-looking guy in the world!
How can a reader determine whether statistics are believable? It starts by applying some critical thinking. What exactly is being measured? What’s the case the people who are doing the measuring hope to make? Is what’s being measured clear and appropriate? In the case described here, what exactly is a “view”? Does it mean someone took the time to read it, or does it mean it just appeared somewhere in their feed that day?
Why is the statistic being measured and what are the goals? If the social media vendor says they provided 10 billion views, it looks like they’re doing a great job, so you’ll keep paying for their services. Might that lead them to fudge a little bit … or on a scale of billions? Are their clients accepting those numbers at face value?
How is it being measured? If an important business decision is being informed by statistics, you owe it to yourself and your company to examine the methodology. The company may claim 90 percent of customers love their product, but how did they find that out? How big was the sample size and did that sample reflect the average customer? Or did they perform a quick email check-in with ten people and project it across their entire business?
There’s a lot of bad math, too, especially where percentages are involved. I’ve seen companies excitedly post about business growth. They had 10 percent of the market last year, and now they’re up to 15 percent, for a whopping 5 percent growth! In cases like this, they’re actually shortchanging themselves, because a move from 10 to 15 percent actually represents 50 percent growth. An interest rate hike from 4 to 5 percent is not a 1 percent increase. It’s 1 percentage point higher, which is a 25 percent jump. (As Teen Talk Barbie reminded us, math class is tough.)
Data of all sorts can be a powerful tool for helping you make decisions. Just make sure you understand the math behind statistics, so you don’t let someone else’s ignorance or deceit lead you to make a lousy choice.
Scott Flood creates effective copy for companies and other organizations. His guide to evaluating freelance creative talent, The Smarter Strategy for Selecting Suppliers, can be downloaded at http://sfwriting.com. ©2022 Scott Flood All rights reserved.