Probability

There has been a tremendous increase in the use of mathematical analysis to make policy, inform organizational decisions, and explain news and events.  I generally think this is a positive thing:  I understand that math gives us a powerful set of tools for understanding and processing the world.  But I also understand the limitations inherent in mathematical modeling.

All mathematical models rely on assumptions that limit their impact.  Mathematicians are typically aware of the assumptions about objects, relationships, and contexts that their models make.  Politicians, journalists, and others who invoke mathematics to make a point seem less aware.  This often leads to bold, unjustified claims based on what “the math” has told them.

An inconsequential but illustrative example of this occurred during Super Bowl 49.

At the start of the second half with the game tied at 14, Seattle drove into scoring position, and faced a 4th-and-1 at New England’s 8 yard line.  Seattle basically had two options:  kick the field goal (a high-percentage play for 3 points), or try to make a 1st-down, and ultimately a touchdown (a lower-percentage play for 7 points).  Seattle opted for the field goal and went ahead 17-14.

Lots of people on Twitter second-guessed the decision, including The Upshot’s David Leohnardt.

Two reasons to go for it there: 1) You probably make it. 2) If you don’t, the other team has bad field position. Expected value is > 3.

— David Leonhardt (@DLeonhardt) February 2, 2015

Here, David Leonhardt is applying a simple expected value argument.  “Going for it” on 4th-and-1 at the opponent’s 8-yard-line likely produces more total points in the long run than kicking field goals, which suggests that Seattle should have gone for it.  It’s not a bad argument; in fact, I used a similar analysis on the NFL’s new overtime rule.

But in order to apply this argument, it’s important to understand what assumptions this model makes.  For example, this model assumes that the amount of time remaining in the game is irrelevant.  Of course, it’s not:  it’s easy to construct a situation in which “time remaining” is the determining factor in the kicking a field goal (say, the game is tied, and only seconds remain).

This model also assumes that all points are of equal worth.  But they aren’t.  Depending on the game situation, the extra four points a touchdown gives you may be irrelevant, or of significantly less value than the three points the field goal gives you (imagine a team up by six points late in the game).

There are lots of factors this analysis does not consider.  This doesn’t mean that the expected value argument is invalid.  It just means that, like all mathematical models, what it says depends on the assumptions it makes.  And the more we use mathematical models to drive our decisions, the more important it is to be clear about the assumptions that are made and the consequences they entail.