At the Tools Page on Mathestate.com, Dr. Bob Rimmer analyzes the price data in the elections futures market. It is very sophisticated modeling, using stable distributions to forecast the probability of winning for McCain and for Obama.
The raw data show the Palin Effect in the dramatic turn around in the futures prices with McCain trailing for many months and Obama leading. In early September there is a sudden reversal. Politics, like life, is dominated by extreme events which only a heavy tailed distribution can capture. None of the other election models are capable of capturing these extreme events. They are far too static and tame. The polls only partly capture the Palin Effect. It is real and dramatic.
(Via John Lott.)
UPDATE:
Sarah Palin is a Black Swan.
Typing the phrase “Sarah Palin is a Black Swan” made me think of the scene in The Lord of the Rings where Galadriel is transfigured as the Dark Queen she would become if she took the Ring.
But she is really more like Eowyn, a rough-hewn woman from a wild and remote province who is good with weapons and is fated to smite down the forces of evil at the moment of their seeming triumph.
Or, so I hope.
Is that lipstick?
Lipstick on a swan.
Michael Barone has a piece entitled “McCain Has the Advantage Over Obama in Post-Convention Polls” which is a good run-down. Nonetheless, the point that markets are better predictors, given that they are deep enough, is somethint most people have not yet figured out. Note also that the bookmakers, as of today anyway, have McCain slightly ahead, e.g. here and here and here. These money-on-the-table predictions are much more convincing than surveys or polls.
Speaking of Black Swans, here is an article by Nassim Nicholas Taleb on the very subject.
Look, it’s the bluebird of hope!
Congrat’s on the perfect Palin metaphor:
‘The Black Swan Song’
Black Swan Theory credit taker, Taleb has the dubious honor of having inspired Rumsfeld, to make his speech about “unknown unknowns”. (He outlined his ideas to Rumsfeld’s aides earlier.) “But I don’t want to be advertised as someone who’s too close to these people,” Taleb says today.
Rumsfeld’s “unknown unknowns” speech made perfect sense. The problem, Taleb says, was that “Rumsfeld himself didn’t understand it.” The black swan way of thinking should have prompted the defense secretary to be cautious about his capacity to predict the future in Iraq. Instead, he fell, again and again, into the prediction trap.
Is this the review you are quoting?
Taleb popularizes ideas that have been around for a long time. I doubt that Rumsfeld first heard about them from Taleb, or that our experience in Iraq would have been better if only the ignorant fool Rumsfeld (this is sarcasm) had taken Taleb’s advice to heart. I get the impression that Taleb spun his comments to humor leftist reviewers. He could just as easily have said that by invading Iraq we reduced the odds of other black-swan events such as the 9/11 attacks.
“Black swan” does not mean any unpredictable event. It means an event that is a statistical outlier — e.g., a market move whose size is five standard deviations from the mean — that occurs much more frequently than a normal distribution would predict.
The point of my post is to note that someone is arguing that the distribution of political events is statistically abnormal, and that normal predictive methods such as those used by pollsters produce misleading probabilities. Maybe McCain’s selection of Palin is a black-swan type of event. I have no clue if this is the case, but it’s an interesting idea. The prediction markets contradict Rimmer. The markets are usually right but not always.
Sarah Palin is not a Black Swan, more like a flash in the pan.
Flash in the pan: Something which disappoints by failing to deliver anything of value, despite a showy beginning.
Arthur Devany may be on to something with his “Evolutionary Fitness” but insofar as politics are concerned … he should stick to fitness.
Nassim would likely agree.