Expert Political Judgment
How Good Is It? How Can We Know?
by Philip E. Tetlock · first published 2004
25% recommend75% neutral
"The intelligence failures surrounding the invasion of Iraq dramatically illustrate the necessity of developing standards for evaluating expert opinion. This book fills that need. Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts." "Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. He evaluates predictions from experts in different fields, comparing them to predictions by well-
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2015–2026Superforecasting
Philip E. Tetlock, Dan Gardner, Santiago Foz (argentino)
Recommended by 44 · mentioned by 7144 recommend · 71 mention44 recommend · 7144 recommend
2015–2026Psychological warfare
Paul Myron Anthony Linebarger
Recommended by 1 · mentioned by 41 recommend · 4 mention1 recommend · 41 recommend
2007–2026The Structure of Scientific Revolutions
Thomas S. Kuhn, Dennis Holland
Recommended by 96 · mentioned by 21596 recommend · 215 mention96 recommend · 21596 recommend
2011–2026Thinking, fast and slow
Daniel Kahneman
Recommended by 373 · mentioned by 879373 recommend · 879 mention373 recommend · 879373 recommend
Specifically in terms of being predictive you might be interested in reading Expert Political Judgment: How Good Is It? How Can We Know? by Philip E. Tetlock if you are not familiar with it already. He tried to empirically test experts political judgement and the main conclusion was pretty much everybody…
on “Epistemic Minor Leagues” ↗You should read Tetlock's other work about foxes and hedgehogs in Expert Political Judgment. It presents a theory of why some people are indeed better forecasters than others. It's not lucky or genius, it's more about people who are able to consider several different data points, consider historical…
on “Forecasting is a talent – luckily it can be learned” ↗…others. If you are really interested in this he has as website called "good judgement project" where you can test your ability to predict against other people on the site over time. http://www.amazon.com/Expert-Political-Judgment-Good-Know/dp... https://www.gjopen.com/
on “Keep your identity small (2009)” ↗…confident prognosticator: I am right / I will be right, except about the timing[0]. But to make more verifiable predictions I would need to start assigning probabilities to particular dates. [0] Tetlock's Expert Political Judgment: How Good Is It? How Can We Know? is still a brilliant and amusing empirical study of opinionated forecasting: https://www.amazon.com/Expert-Political-Judgment-Good-Know/d...
on “Bill Could Destroy Uber's Unsustainable Business Model” ↗…https://rationaldino.substack.com/ but which will take a while. There is a very simple litmus test to tell which are likely to fade, versus which aren't. How certain are they? https://www.amazon.com/Expert-Political-Judgment-Good-Know/d... reports on a long-term study about pundits. Basically they can be divided into two groups: 1. Hedgehogs. Have one overarching theory that they are certain of. 2. Foxes. Can pick and choose…
on “The Death of a Public Intellectual” ↗…documented history of quantitative reasoning, along with data showing historical accuracy, is the best evidence of a trustworthy source. I base those heuristics on the research in https://www.amazon.com/Expert-Political-Judgment-Good-Know/d.... Applying this heuristic to "sounds certain" results in a conclusion, that is an invitation to further research that may or may not happen.
on “The impossibility of rationally analyzing partisan news” ↗…when subject to a prediction tournament with strict rules, they would score no better than random [2]. [1]: http://www.amazon.com/Superforecasting-The-Art-Science-Predi... [2]: http://www.amazon.com/Expert-Political-Judgment-Good-Know/dp...
on “New Kid on the Blockchain” ↗…because it should be, as there is an increasing body of evidence that experts, from politics to medicine, have almost no advantage in forecasting power than the average person. https://www.amazon.com/Expert-Political-Judgment-Good-Know/d... Why should "experts" (often just pundits) have any authority when they have consistently demonstrated they deserve very little? Finally, the political slant of this article, going along…
on “The declining authority of statistics and the experts who analyse them” ↗"Expert Political Judgment", by Tetlock, has a lot to say about this. It adds data to the story about the Fox and the Hedgehog. Basically, the hedgehog knows one big thing and tries to force the world into their mental model. The…
on “How to Be an Expert in a Changing World” ↗I recently started reading Philip E. Tetlock's Expert Political Judgment[0] and Ehrlich gets namechecked pretty early in it. For me, this article proved timely. Ehrlich gets credit in my book for making concrete falsifiable predictions and acknowledging some are wrong. He…
on “The Unrealized Horrors of Population Explosion” ↗…future events. > The Good Judgment Project (GJP) is a project "harnessing the wisdom of the crowd to forecast world events". It was co-created by Philip E. Tetlock (author of Superforecasting and of Expert Political Judgment: How Good Is It? How Can We Know?), decision scientist Barbara Mellers, and Don Moore. It was a participant in the Aggregative Contingent Estimation (ACE) program of the Intelligence Advanced Research…
on “The Good Judgement Project” ↗Most people (especially pundits) are terrible at predicting the future. If you want to get better, you should read "Superforecasting" by Tetlock. [1] https://en.wikipedia.org/wiki/Expert_Political_Judgment:_How... [2] https://en.wikipedia.org/wiki/Superforecasting
on “Space: A sudden light” ↗…better than actual experts, who often have poor track records as a result of incentives (or selection) to hype and extremize.[1] Philip Tetlock has written on this topic for years. Two of his books are Expert Political Judgment and Superforecasting. [1]: https://en.wikipedia.org/wiki/The_Good_Judgment_Project Edit: So to directly answer your question, rather than AI experts, I'd prefer technology experts (AI or otherwise)…
on “When Will Machines Exceed Human Performance? Survey of ML Researchers.” ↗In context, Gove was talking about expert political judgment applied to making short term predictions: experts predicting prompt, dire consequences from Brexit. Such predictions have been the subject of academic study. One obtains gold-standard data on the accuracy…
on “How Turkey Purged Its Intellectuals” ↗…effectiveness. Specifically he used the analogy of "foxes vs hedgehogs" from Isiah Berlin's famous essay as different cognitive styles and compared predictions over along period. https://www.amazon.com/Expert-Political-Judgment-Good-Know/d... https://en.wikipedia.org/wiki/The_Hedgehog_and_the_Fox
on “Stupidity is a specific cognitive failing” ↗