Sunday, January 11, 2009

Numbers Can Lie

One of the first things you should learn when you take a statistics class is how numbers can be twisted around to "prove" any number of theories. In part, this is a good thing: we want to look at the available evidence in as many different ways as possible, so we aren't missing out on the actual reality hiding underneath. For example, in 1800, who could've guessed that the speed of light in a vacuum was fixed? Two decades ago, who knew that diapers and beer are bought at the same time? Etc, etc, etc.

The problem is that we have a habit of jumping to conclusions based on the way a set of numbers look, even though the underlying reality is quite different than it would appear. A good example was a report released in late December, which had people panicking about rising homicide rates among young African-Americans. Numbers are the way we look at a fundamental reality, but we should never confuse those numbers WITH reality.

This was brought to my attention, when I was watching a rerun of ABC's new series "True Beauty." The show's premise is basically this: gather up 10 hotties, and have them go through tests that measure their physical beauty...but also put them through secret tests that test their kindness. At the end, the winner should be a "true beauty," or a person that has a beautiful body as well as a big heart.
The first test they were put through was a "science" test. A plastic surgeon measured their beauty according to their various facial and bodily attributes, compared them to a set of "ideal" numbers, and gave them an overall score between 1 and 100. All scored above the 85 "good-looking threshold," though only two met the 95 "star quality" threshold.

Within such a narrow range (5 points maximum, on a scale of 1-100), can we really be confident in our results? Because this man's "beauty equation" isn't actually reality. It's a model, or rather, a depiction of what reality SHOULD be. I'm not saying beauty can't be quantified: I'm saying that his model may not be accurate to actually discern the difference between a 90 and a 95, in the same way that most of our eyes really aren't capable of distinguishing between 15/16 of an inch and a full inch.

Models aren't always capable of discerning reality perfectly. Another good example comes from Mankiw's recent article:
If you hire your neighbor for $100 to dig a hole in your backyard and then fill it up, and he hires you to do the same in his yard, the government statisticians report that things are improving. The economy has created two jobs, and the G.D.P. rises by $200. But it is unlikely that, having wasted all that time digging and filling, either of you is better off.

GDP is A measure of economic well-being, but it is by no means all-encompassing.

Really, this should be common-sense: don't trust models and statistics all the time, especially without due diligence. But we do it all the time, especially when it comes to investing. "Let's buy a house because it'll be a good investment, even though real estate values can fluctuate wildly." "Apple's stock price has been rising recently, so let's hop on to the bandwagon!"

It's hard to resist this temptation, especially when you don't know the underlying reality.

No comments: