How to Use Statistics to Predict the Future
Nate Silver is a statistician and forecaster who made his name predicting the performance of baseball players. He now produces the FiveThirtyEight blog, known for its dead-on accurate predictions in political contests.
In his book, The Signal and the Noise, Silver outlines the principles that can help us get a grip on our own lives, and offers strategies to help us focus on where we are, and chart a course for where we want to go. Here are the issues he tells us to pay attention to:
Focus on what's important. Two great inventions--the printing press and the Internet--have increased the amount of available information at an exponential rate. But they don't tell us what information is false or silly, versus what is real and relevant. Our challenge is to focus on information that makes a difference in our lives.
Many issues cloud our vision. People tend to attach too much importance to recent events that seem dramatic but are not really significant. We also focus on familiar places and nearby events, whether they affect the larger world or not. And we all have our prejudices and biases. Human beings like information that supports their own views. The way to break out of our tunnel vision is to test our views in the real world, acknowledge our mistakes, and learn from them.
Are you a hedgehog or a fox? Silver identifies two types of people. The hedgehog is a Type-A personality guided by firm beliefs. The fox is a scrappy creature more tolerant of dissenting opinion. Hedgehogs tend to be business and political leaders. They make good television guests. But they make bad predictions. Foxes are less certain about the world, and may come across as lacking self-confidence. But they make better predictions because they understand that many problems are hard to forecast, and they account for these uncertainties.
Know the difference between a prediction and a forecast. A prediction is a specific statement about when and how something will occur. A forecast is a probabilistic statement. Science cannot predict when or where an earthquake will occur. But it can forecast with confidence that at some point over the next hundred years there will be a major earthquake in California, and that California will experience more earthquakes than New Jersey. Similarly, you might make a reasonable forecast that the stock market will on average gain 7 percent annually over the next ten years. But no one can predict where the S&P will stand at the end of 2013. Only hedgehogs will try. And it will be a guess.
How to be less wrong. We all make predictions based on our knowledge and experience. The way to become more accurate is to alter our predictions as we acquire more data. Or as John Maynard Keynes famously said, "When the facts change, I change my mind." Business people, politicians, doctors, and lawyers all make predictions about products, people, and social policies. Those who don't adjust and improve their results often find themselves disgraced, or out of a job. Investors make predictions. The successful ones acknowledge when they're wrong and sell their mistakes.
The consensus is usually right. Many people make predictions about sports, politics, or the weather. Those predictions converge as people test out their theories and seek the truth by factoring in more evidence. This explains why the general consensus about the outcome of any event is typically a better prediction than any one person's view, and why the outlier prediction is usually less accurate.
Be careful. Sometimes we can't identify what information is important, and often there are things we just don't know. That's why we're surprised by some events, or fail to foresee a big event like September 11. We need to remember that rare and dramatic events will occur, even if we can't see them coming. We can't predict the next California earthquake. But we do know that a major earthquake will occur in Los Angeles about once every 40 years. We can't predict the next stock market crash. But we do know the chances of a crash are less than 5 percent when the average price-to-earnings ratio is under 15, but almost 20 percent when the average price-to-earnings ratio is over 30.
Despite all our advances in technology, future events are not becoming more predictable, because those very advances also bring us additional complexities. Still, you can get a grip on your own world by recognizing your biases, revising your views as you encounter new information, and trying to maintain a clear-eyed vision of the world.
Tom Sightings is a former publishing executive who was eased into early retirement in his mid-50s. He lives in the New York area and blogs at Sightings at 60, where he covers health, finance, retirement, and other concerns of baby boomers who realize that somehow they have grown up.
More From US News & World Report
