July, 2011

IARPA and Open Testing Methods

by Bruce Bueno de Mesquita
The Conversation
July 27th, 2011

I find myself overwhelmingly in agreement with Gardner and Tetlock. The IARPA undertaking may well be the right and most opportune forum for testing alternative theories, methods, and hunches about prediction. Just tell me how to participate and I am happy to do so. I have already provided Tetlock with dozens of forecasts by my [...]

Read: IARPA and Open Testing Methods

An Unpredicted Outcome of Our Exchange: Testable Hypotheses

by Dan Gardner and Philip Tetlock
The Conversation
July 26th, 2011

Each of the three commentaries on our target essay highlights obstacles to distinguishing real from pseudo-expertise in the political realm—and raises issues that we welcome the chance to address. Robin Hanson poses a puzzle: Why are thoughtful people so reluctant to embrace technologies, such as prediction markets, that have a track record of improving predictive [...]

Read: An Unpredicted Outcome of Our Exchange: Testable Hypotheses

Forecasting Tournaments Should Cast a Wider Net

by Robin Hanson
The Conversation
July 20th, 2011

I’m happy to learn that Bruce Bueno de Mesquita has made specific predictions in some of his publications, and has participated in some forecasting competitions. He deserves praise for being forthright in this way. But the specific issue was the relative accuracy of foxes and hedgehogs, and my claim was about “open competitions giving a [...]

Read: Forecasting Tournaments Should Cast a Wider Net

The Tests Are Underway

by Bruce Bueno de Mesquita
The Conversation
July 20th, 2011

Robin Hanson’s comments intrigue me. He seems to believe that I am reluctant to subject my method to comparison to alternative methods and that I have not been open to doing so in the past. Nothing could be further from the truth. He does correctly note that academic journals are an open competition but, as [...]

Read: The Tests Are Underway

Designing Fair Tests for the Hedeghogs

by Robin Hanson
The Conversation
July 20th, 2011

Over decades, Philip Tetlock painstakingly collected data on simple long-term forecasts in political economy. He showed that hedgehogs, who focus on one main analytical tool, are less accurate than foxes, who used a wide assortment of analytical tools. Since John Cochrane and Bruce Bueno De Mesquita are both prototypical hedgehogs, I was curious to see [...]

Read: Designing Fair Tests for the Hedeghogs

Fox-Hedging or Knowing: One Big Way to Know Many Things

by Bruce Bueno de Mesquita
Reaction Essay
July 18th, 2011

We should not be surprised when experts fail to predict the future, says Bruce Bueno de Mesquita. Expertise doesn’t mean good judgment; rather, expertise is an accumulation of many facts about a subject. That we commonly prefer the pronouncements of experts suggests a bias in favor of “wisdom” and against the scientific method. He argues that statistically rigorous game theory can do better by examining the beliefs and objectives of major players in a given situation, and he welcomes forecasting tournaments as a means of refining the method.

Read: Fox-Hedging or Knowing: One Big Way to Know Many Things

In Defense of the Hedgehogs

by John H. Cochrane
Reaction Essay
July 15th, 2011

John H. Cochrane offers a limited defense of the hedgehogs: Economics is full of uncertainty, because the agents within the system are aware of the theories and possible actions of the other agents. Trying to capture all of them produces a hopeless muddle. Instead, what are needed are explanations of principle and the tendencies that arise all other things being equal. This calls for a hedgehoggy worldview after all. “Especially around policy debates,” he argues, “keeping the simple picture and a few basic principles in mind is the only hope.”

Read: In Defense of the Hedgehogs

Who Cares About Forecast Accuracy?

by Robin Hanson
Reaction Essay
July 13th, 2011

Robin Hanson argues that most people aren’t interested in the accuracy of predictions because predictions often aren’t about knowing the future. They are about affiliating with an ideology or signaling one’s authority. The outcomes of predictions have nothing to do with either, of course, especially in the present. He suggests that one way to make predictions more accurate might be to lift both the social stigma and legal prohibitions against gambling. Unlike mere predictions, wagers carry real consequences for those who make them. Which, Hanson argues, they should.

Read: Who Cares About Forecast Accuracy?