Why long tail graphs are usually shown on a log scale:
Unfortunately, the illustration works only as a large graph, because graphed out on small paper gives us only two discernable lines, one on each axis.
— A practical model for analyzing long tails, by Kalevi Kilkki First Monday, volume 12, number 5 (May 2007)
The sports field graph is a clever way of showing how the fat head of a long tail distribution can be extremely higher than the long tail; this is normally not so clear on log scale graphs.
Ben Hyde draws attention to the same kind of presentation applied to a different long tail distribution: income distribution in the U.S.. If the median U.S. family income of about $40,000 is represented by a stack of $100 bills about 4 centimeters high on the 50 yard line of a football field, The height of a curve doesn’t rise visibly until it reaches the goalposts on the far end, at which point it suddently ascends higher than any nearby trees, mountains, or stratosphere. Nope, I’m not exaggerating; go look. Sometimes it takes a good visualization to bring out relevant comparisons.
This is the nature of many big, rare, risks, too. No hurricane damage for years on end, then suddenly devastation. You can’t really manage a risk unless you understand it.
-jsq
That L-curve video is great isn’t it! I wish it managed to get across the additional point about how much of the total income is in that last yard of the field; but really that’s nit picking.
Presumably one could make analogous presentations for weather, network traffic through routers, transaction traffic through retailers, etc.
The risk point is exactly right on. The rogue terrorist, the mutating germ, the innovator; these are all instances of some actor out on the long tail blowing the tires off somebodies hub(s).
Many of these things are often considered unpredictable,as in Nasimm Taleb’s idea of a black swan that John Robb has been explicating.
http://globalguerrillas.typepad.com/globalguerrillas/2006/02/al_qaedas_black.html
But are they really unpredictable? Maybe not, if you look at enough of the small but similar or related events leading up to them. After all, we now know that something very like 9/11 (al Qaeda, New York, probably airplanes) was predicted by the CIA before the event.
And yes, how much of disruptive business innovation is really completely different from what has gone before? Much of it is probably some clever and l ucky person or group tweaking earlier methods and coming up with a combination that works a lot better. That’s certainly what both Apple and Microsoft did.
-jsq
They certainly become a lot more predictable once the model becomes more accurate and a pool of talent begins to have good intuitions based on that model. The networks that underly the supply chain, the social fabric, or the internet are to varying degrees robust against these kinds of risks.
While I find Rob to be excessively pessimistic it is disconcerting how even reasonably mature systems like the interstate highway system can’t seem to defend their hubs, even from idiots.
http://www.nytimes.com/2007/06/01/nyregion/01truck.html?_r=1&oref=slogin
http://www.youtube.com/watch?v=faQLrb7-qQ8
The problem comes in when quantifying risks, while we use HARM in some places, quantifying risk for natural disasters is somewhat unpredictable because many of them happen with only some warning.
Of course there are direct warnings, such as tremors before an earthquake, and more indirect warnings, such as historical records showing there have been earthquakes here before, and contextual warnings, such as the geology hereabouts sure does look like a fault line, or there have been other earthquakes in a snaky line that includes this point, and frequency warnings, such as many points along this fault seem to have earthquakes every x years on average, and it’s been 2x years since the last one here. Risk quantification may still be a bit fuzzy, but some level of confidence for a big rare risk is probably better than none, for planning purposes.
-jsq