Maybe the Fans Are Right

By David J. Berri,Martin B. Schmidt

Date: Jan 19, 2010

Return to the article

Decision-making is not often as rational as traditional economics argues. And that story has an impact on our understanding of both sports and economics.

"I must say, with all due respect, I find it very hard to see the logic behind some of the moves you have made with this fine organization. In the past 20 years, you have caused myself, and the city of New York, a good deal of distress, as we have watched you take our beloved Yankees and reduce them to a laughing stock."

George Costanza upon meeting George Steinbrenner (owner of the New York Yankees): Seinfeld, "The Opposite" (season 5, 1994)

"What the hell did you trade Jay Buhner for?! He had 30 home runs and over 100 RBIs last year. He's got a rocket for an arm. You don't know what the hell you're doin'!"

Frank Costanza (George's father) upon meeting George Steinbrenner: Seinfeld, "The Caddy" (season 7, 1996)1

Few sports fans ever meet the people who operate their beloved sports teams. Such a meeting, though, would probably inspire many fans to get in touch with their inner "Costanza." Given the opportunity, fans would love to ask:

Obviously, this is just a sample of the questions asked. And, just as obviously, we have cleaned up the language. What may not be obvious is the economic implication of these questions.

Fans often suggest that decision-makers in sports are less than perfect. Managers and coaches are not only accused of making bad choices, fans often accuse these people of making the same bad choices over and over again. Many economists, though, find such stories unbelievable. After all, traditional economics clearly teaches that decision-makers are supposed to be "rational."

What does it mean to be a "rational" decision-maker? Thorstein Veblen sarcastically argued in 1898 that economists tend to see people as "hedonistic lightning calculators."2 In more recent years, Richard Thaler and Cass Sunstein have just as sarcastically suggested that the rational decision-makers described by economists "can think like Albert Einstein, store as much memory as IBM's Big Blue, and exercise the willpower of Mahatama Gandhi."3

Both these remarks comment on the simple idea that rational decision-makers "choose efficiently the means that advance their goals."4 Let's imagine the behavior of a manager and coach that "chooses efficiently." Such a person would tend to make the correct decision given the circumstances they observe. Perhaps more importantly, as the game changes, these same coaches and managers would change their point of view and make different decisions. Therefore—and contrary to what sports fans often contend—it's not possible for coaches and managers to make the same mistake over and over again.

So who is right: fans or economists? The emerging field of behavioral economics—via a collection of laboratory experiments—seems to side with the fans. Experiments have shown that people are not quite as rational as traditional economics contends.5 Some economists have argued, though, that how people behave in a laboratory experiment is different from how they behave in the "real world."6 In the real world, people face real consequences for making mistakes, and real consequences force people to be rational.

Sporting Rationality

To settle this debate, it might help to move out of the laboratory and look at decisions in the "real world." Sports are often described as being removed from reality. Yet for the people in this particular reality, what happens in sports matters. Consequently, we can learn about the rationality—or irrationality—of human decision-making by examining the "real world" of sports. This examination, consistent with the experimental evidence, will show again and again and again (actually we will present at least 20 "agains") that decisions in sports are not completely rational.

Before we get to this examination, let's emphasize that the word "irrational" is not synonymous with the word "stupid." When we eventually argue that decision-makers in sports are "irrational," we will not be saying that people in sports are not as smart as people are in other industries or other occupations. In fact, people in sports are often better prepared for their jobs than people employed elsewhere.7 Furthermore, it seems likely that whatever "irrationalities" are observed in sports are likely to be found elsewhere.

We make this claim because at first glance decision-makers in sports perhaps more than anyone else should be "rational." There are two characteristics of the sports industry that bolster this expectation. First, despite being a relatively small industry in the American economy,8 sports receive an inordinate amount of attention from the media. After all, no other industry has an entire section of each local paper devoted to its happenings. Such coverage raises the cost of failure to the participants in sporting contests. Losing in sports, as noted earlier, is not a private affair. Sports fans both near and far witness your failure and are often not shy in expressing their disappointment. Although people do pay some attention to failures in non-sports industries, it's rare to see interested observers in other industries pay money to yell obscenities at those who fail to achieve success.

Sports are not only different in terms of attention received. In sports, success and failure would seem to be—relative to other industries—somewhat easy to understand. To illustrate, ask yourself this question: At your place of employment, who is the most productive worker? Yes, we know. It must be you. But is this something you could prove? We suspect, for many people, this would be difficult. For workers in many non-sports industries, measuring worker productivity is difficult.

Take our profession, college professors. We both think of ourselves as above average professors. But such a self-assessment may be dubious. In fact, a survey at the University of Nebraska revealed that 94% of college professors thought they were better teachers than the average at that same institution.9 We don't think this obvious delusion is unique to Nebraska. Neither of us can recall meeting a fellow professor who thought he or she was below average.

It also turns out that professors are not the only people who overestimate their abilities. Thaler and Sunstein find evidence of this phenomenon in surveys of MBA students, drivers, and new business owners,10 and this is just a partial list. They go on to note that "unrealistic optimism is a pervasive feature of human life; it characterizes most people in most social categories."11

In sports, though, there's a brake on this natural tendency. If we asked Jeff Francoeur of the Atlanta Braves how his hitting in 2008 compared to the league average, Francoeur would be hard pressed to argue he was above average. With respect to most of the standard measures of hitting performance, Francoeur was below average. Likewise, Francoeur's teammate Chipper Jones can be pretty confident that he really was an above-average hitter in 2008. Again, that's what the stats indicate.12

Because sports come with numbers, evaluating worker performance in sports would seem to be easier. Consequently, the path to success would seem—relative to what's seen in other industries—easier to navigate. Unfortunately, there are a few stumbling blocks on the path to victory.

The stumbling blocks can be separated into two broad categories. First, numbers have to be understood. Coaches and general managers can see the numbers associated with each player's performance. But how these numbers connect to wins is not always appreciated. Even if the numbers were understood, though, another stumbling block gets in the way. Understanding the past doesn't have much value if the past can't predict the future. Some numbers in sports are simply inconsistent across time. When that's the case, following the unpredictable numbers makes the path to victory hard to find.

What the numbers mean for the present and future is the foundation of our story. But before we get to that story, we need to address a fundamental objection to any sports analysis offered by academics. Specifically, is it likely that academics would be able to say anything that the "experts" employed in the sports industry don't already know?

Crunchers, "Experts," and the Wrath of Randomness

Even if you don't believe people are perfectly rational, you might still expect decision-makers in sports—where there is an abundance of information, clear objectives, and severe consequences for failure—to get it "right." After all, these people are the "experts." There is no reason to think that some college professors armed with a slide rule can do any better.

Let's respond to that by noting that neither of us owns a slide rule (or knows how to use one). We do, though, have spreadsheets and some fairly sophisticated econometric software. There are a number of examples where people armed with such tools can see things that "the experts" miss. Some of our favorite examples come from places as diverse as the wine industry,13 analysis of Supreme Court decisions,14 and the treatment of heart patients in the emergency room.15 In essence, it appears that human beings—who are not actually lightning calculators—tend to lose in a contest against actual lightning calculators.16 Such an outcome is observed whether or not the human being is an "expert."

Related to the obvious point that people are not lightning calculators is a classic finding in psychology. People in sports often claim they can simply watch a player during a game and "know" if he is good or bad. The seminal work of George Miller, though, has shown that the human mind can only track about seven items at one time.17 In sports, though, a multitude of events are happening throughout the contest. All these events not only have to be seen and noted, the impact of these factors on wins must be ascertained. To claim that you can simply watch a player and see his or her overall contribution to wins suggests that you believe your mind can do something that research suggests is difficult. Despite the limitations of personal observation, though, human beings still tend to believe the analysis based on this approach is correct. Such overconfidence can often cause people to ignore contradictory information.

Statistical analysis, though, can overcome these issues. Spreadsheets and statistical software can evaluate more games than a person can ever personally observe. These evaluations can also allow us to look past the "most dramatic factors" and identify which factors truly matter most in terms of wins. Furthermore, the analysis can also easily change as new data arrives. Perhaps most importantly, statistical models come with confidence intervals.18 In other words, statistical models can assess the quality of the prediction being made. Try getting that kind of service from a human expert!

Number crunching does more than offer better explanations than what we get from "experts." It can also tell us when there really isn't an explanation. In other words, number crunching can help us see when a process is inherently random.

Let's illustrate this last point with an oddity from the Super Bowl. As of 2009, the National Football Conference (NFC) team has won the coin toss at the Super Bowl for 12 consecutive years. Such a streak clearly indicates that the NFC has some secret that allows it to better predict coin tosses; and the American Football Conference (AFC) better do some work if it hopes to close the "coin toss predicting gap." Then again, maybe there's another possibility. Flipping a coin is a random process.19 Even if you flipped a coin 12 times in a row with the same result, the process is still random. The outcomes don't tell us anything about the skill level of the NFC teams. This point should be obvious, since predicting a coin toss is not an actual skill.

This simple story highlights an additional advantage of analyzing sports data, and another potential pitfall for decision-makers. Some numbers that we associate with an athlete represent the skills of the performer. Other numbers, though, are not about a player's skill, but instead are determined by the actions of the player's teammates (or coaching or some random process). The analysis of numbers can actually clue us in on the skills versus non-skills argument. In the absence of such analysis, though, a decision-maker can actually suffer from the "wrath of randomness." Specifically, a decision-maker can be fooled by numbers that are as reliable predictors of the future as the numbers generated by our coin-flipping game. When that happens, money can be wasted on players who are not really helping. Or on the flip side, a player with some supposedly poor numbers can be removed from the roster when in fact the player is actually helping the team win.

A Century of Mistakes in Baseball

Although the "wrath of randomness" does rear its head in the study of sports, often the numbers do tell a story. Let's start with a great story that reveals a century of mistakes in Major League Baseball (MLB).

In 1997, the Oakland A's ranked toward the bottom in Major League Baseball, in respect to both team payroll and winning percentage. The next season, Billy Beane became general manager, and part of this story stayed pretty much the same. Specifically, the lack of spending on players didn't change. What did change were the outcomes achieved by the A's. From 1999 to 2002, only the New York Yankees, a team that spent three times more on playing talent than Beane, managed to win more games in the American League. The term "more" is a bit misleading. The Yankees actually won only two more games than the A's across these four seasons.

How was this possible? It's been argued20 that the key was Beane's ability to recognize specific inefficiencies in baseball's labor market. Such inefficiencies allowed Beane to pick up talent that was both cheap and productive.21

At least, that's the story that's been told. For the empirical evidence supporting this tale, we turn to the work of Jahn Hakes and Raymond Sauer. These economists decided to investigate whether the baseball player market was, as they say, "grossly inefficient." Before we get to their answer, however, let's briefly describe an efficient labor market. A basic tenet in economics is that workers are paid in line with their expected productivity, that is, workers who are expected to be the most productive get paid the most. This suggests that baseball players who are expected to perform the best are paid the highest salaries (at least, once they become free agents). In a world where some teams are "rich" and others "poor," the best players typically end up on teams that have the ability to pay the most. In other words, we would expect the Yankees—or the "rich" team—to get the best talent, and a "poor" team like the Oakland A's should end up with the less capable players.

The key to the above reasoning is the phrase "ballplayers who are expected to be the most productive." This tells us that having money isn't enough. Teams have to be able to identify the "most productive" players. If one team can do a better job at identifying the "most productive," then that team might be able to field a very good team that's not very expensive.

To see if the Oakland A's actually followed this blueprint, Hakes and Sauer needed to connect three dots:

To cut to the chase, Hakes and Sauer found that "...hitters' salaries during this period (2000-2003) did not accurately reflect the contribution of various batting skills to winning games." Furthermore, "this inefficiency was sufficiently large enough that knowledge of its existence, and the ability to exploit it, enabled the Oakland Athletics to gain a substantial advantage over their competition."22

How did they reach this conclusion? First, data was collected on team winning percentage, team on-base percentage,23 and team slugging percentage24 for all 30 MLB teams from 1999 to 2003. They then ran a simple regression.

Okay, we get ahead of ourselves. What's a "simple regression?" Regressions25 are essentially the test tubes of economics. When a chemist seeks to understand the world, he or she steps into a laboratory and starts playing around with test tubes. These test tubes allow a chemist to conduct controlled experiments. Hakes and Sauer, though, could not conduct a controlled experiment with Major League Baseball (at least, Major League Baseball probably wouldn't let them do this). What they could do, though, is employ regression analysis. This is simply a standard technique economists employ to uncover the relationship between two variables (like player salary and on-base percentage), while statistically holding other factors constant. When properly executed, regression analysis allows one to see if the relationship between two variables exists; or more precisely, if the relationship between two variables is statistically significant.

Beyond statistical significance, we can also measure the economic significance of a relationship,26 or the size of the impact one variable has on another. Consider how on-base percentage and slugging percentage relate to team wins. Hakes and Sauer found both to be statistically significant. On-base percentage, though, had twice the impact on team wins. Such a result suggests that players should be paid more for on-base percentage. The study of salaries, though, suggested that prior to 2004, it was slugging percentage that got a hitter paid. In fact, in many of the years these authors examined, on-base percentage was not even found to have a statistically significant impact on player salaries.

After 2004, though, the story changed.27 An examination of data from 2004 to 2006 reveals that on-base percentage had a bigger impact on player salaries than slugging percentage. In other words, an inefficiency exploited by Billy Beane was eventually eliminated.28

It's important to note, though, how long this took. The National League came into existence in 1876. All of the data necessary to calculate on-base percentage was actually tracked that very first season in the 19th century. However, it was not until the 21st century—or after more than 100 years—that these numbers were understood by decision-makers in baseball. It appears that decision-makers in baseball made the same mistake in evaluating talent year after year, and this continued for a century. Such a tale suggests that maybe all those fans are on to something. Maybe coaches and general managers are capable of repeating the same mistakes.

Of course, one story from the real world of sports doesn't make a point. What we need is a multitude of stories. And that's what we provide. The stories we tell give insight into how free agents are evaluated, how teams make decisions on draft day, and even how choices are made on game day. We even present evidence that the evaluation of coaches in the National Basketball Association (NBA) is less than ideal.

All of these tales from the world of sports tell one very important story. Decision-making is not often as rational as traditional economics argues. And that story has an impact on our understanding of both sports and economics.