The Blog

Groundhog Day and Regression to the Mean

It is only when we begin to explore the realm of statistics that "average" becomes both very interesting and very powerful.
This post was published on the now-closed HuffPost Contributor platform. Contributors control their own work and posted freely to our site. If you need to flag this entry as abusive, send us an email.

If asked what the word average means, most people would probably define it as not being particularly good or bad. Others might say it means not standing out in any way. Really good or really bad pops out dramatically -- think of the lists of the year's best or worst movies -- but average never quite gets noticed. It's invisible. This changes, however, when we place average in the context of statistics and behavior. In statistics the mean is the term used to describe the average measurement for whatever is being studied. And it is only when we begin to explore the realm of statistics that "average" becomes both very interesting and very powerful.

In July of 2009 The Wall Street Journal published an article in their sports column, "The Count," about the statistical concept, regression to the mean. This concept suggests that behavior over time tends to drift back to the average. In this particular case the question being asked was whether Albert Pujols, who hit over 30 home runs during the first half of the 2009 season, was going to reach the mythic number of 60 for the full season? It would appear that reaching this goal was highly probable given his current performance. Looked at from the point of view of regression to the mean, however, the answer is, no, he wouldn't do it. Based on Pujols's home run output in previous years, his first half of the season should be considered an aberration, and the rest of that season would reflect his average performance. And that's exactly what happened. Pujols ended the season with 47.

Now, let's take regression to the mean and make it more personal. We all have a series of behavioral patterns that recur frequently. These patterns are so much a part of our lives that there are moments when we feel as if we're Bill Murray in "Groundhog Day." And it is our lifestyle patterns where regression to the mean has its most profound implications. Some of these patterns are the ones we most often complain about and attempt to change. Take for example our attempts to eat more healthfully, exercise, stop procrastinating, get more sleep, manage our finances better, improve our relationships, etc. We all know what typically happens during these motivational moments in our lives. We start out strongly and then slowly and inexorably, revert back to the old patterns that reflect our "behavioral mean."

There's an old saying in psychology that goes as follows: the best predictor of future behavior is past behavior. Clearly, there is something to the phenomenon of habitual behavior having an almost magnetic pull to it. And, this is not limited to behavior. Social psychological research has demonstrated that mood -- our "average" everyday experience -- is subject to the same tendency to regress back to the mean. So whether something really good or really bad happens in our lives, our mood drifts back to that narrow band of day-to-day experience we typically have. Even lottery winners will tell you that the state of euphoria they experience lasts only for a relatively brief period of time before their everyday mood regresses to their mean.

All is not lost, however. We all know individuals who have figured out ways to negotiate with themselves at critical micro-moments in time in order to maintain the changes they've made. What that means (no pun intended) is that they created a new mean, a new set of average behaviors that were more functional and sustained the new patterns for a sufficient amount of time for the patterns to "take." And these changes don't have to be major. Small changes generate huge returns over time. So when you boil it all down, it really is all about one key element -- doing what you know you should be doing when you don't feel like doing it. A wonderful byproduct of this incremental shift is that your day-to-day mood tends to improve as well. The old Nike tag line, Just Do It, has some serious merit.