TNL Conclave: Superforecasting

Presenter: Ricardo Sanchez

In 2011, I signed up to be part of an IARPA experiment run under the Aggregative Contingent Estimation (ACE) program. The goal of the project was to conduct a multi-year study to explore techniques for improving the accuracy, precision and timeliness of intelligence forecasts. Several teams were invited to participate. I ended up on Team Good Judgment, run by researchers at Wharton. Study participants were asked to make probabilistic estimates on a wide range of geopolitical topics and events in exchange for a payment in the form of Amazon gift cards. Four years later, the results were in. Team Good Judgment blew the competition out of the water and consistently outperformed the Intelligence Community’s own analysts using nothing but open source information. At that point a company was formed, Good Judgment Inc., which provides training and paid forecasts to clients from Washington, DC to Wall Street. In this talk, I will describe the early days of the tournament and what we’re doing now. I’ll also try my best to relate some of the research findings. Full disclosure: I’m still currently a paid Superforecaster with GJI.

Share your thoughts