Statistics

An interesting article in the New York Times (with a cool interactive graphic) digs down into the data of “on-time performance” for the three major NYC-area railways–LIRR, Metro North, and NJT.

http://www.nytimes.com/2010/07/27/nyregion/27ontime.html

The official figure is that 96% of trains arrive on time.  This may be hard for the average commuter to believe, but it is technically true.  What accounts for this discrepancy between perception and performance?

As the data shows, trains were far more likely to run behind schedule during peak times (rush hours) than during non-peak times.  This makes intuitive sense–more people on the trains, more trains running simultaneously, and so more potential problems and delays.  But this also means that a delay during rush hour will affect more people, and thus have a greater impact on the perception of the train’s timeliness than, say, a late-night lateness.

Furthermore, there are only a couple of “rush” hours, so the total number of rush-hour trains is less than the total number of non-rush-hour trains.  The latter are more likely to run on time, and so their aggregate impact on overall performance dominates the calculation.

To simplify, if 5 out of 10 rush hour trains are late (50%), and 5 out of 50 non-rush hour trains are late (10%), then  the overall lateness ratio is 10 out of 60, or about 17%.  But a lot more people are riding the rush hour trains, and to them, it seems like the trains are late half the time.  This is a example of the Inspection Paradox.

There are also some interesting psychological principles at work here:  for example, people will generally think more about the one time they are late than, say, the 10 times the train ran on-time.  It will be even more significant if that lateness ends up costing you something–like a good impression on a client or a boss.