Perpetual Beta

“The fox knows many things, but the hedgehog knows one big idea”—Philip E. Tetlock.

The reference to the fox and the hedgehog comes from Phil Tetlock—a professor who, since the 1980s, has consistently questioned the accuracy of high–profile predictions and demonstrated that the “stars” of analysis can be “about as accurate as a chimpanzee throwing darts at a board.” At the center of his interest lies the search for the traits that describe so–called superforecasters—people distinguished by sober judgment and accurate prediction of the future ¹.

From the prophet ...

To begin with, a small history lesson: when the Blitz began in 1940, one of the greatest pre–war fears circulating in Allied society was that “the bombing of London would break the spirit of the British.” Studies of morale and later accounts showed rather an increase in resilience, growing solidarity, and the capacity to adapt ². Similarly today: in eastern Ukraine, the “buzzing” of drones has become part of everyday life, and Kyiv is not laying down its arms as had been assumed at the beginning. On the contrary, it says loudly that “there can be no talk of giving up the occupied territories,” let alone consent to a rotten peace, and the desire to join the West is stronger than it was before the war began ³.

Poor recognition of the situation, arrogance and bravado on the part of decision–makers, as well as their advisers, in this case affect distorted estimates and inaccurate predictions about the future.

... to Tetlock

Tetlock systematically demonstrates that the best forecasters do not, in fact, have to belong to a homogeneous elite of authorities—the kind we look at through the prism of specialist identity, self-confidence, fame or, as we would say today, “followers.” In the realities of the social and political sciences, which constitute Tetlock’s field of research, this group turns out to be surprisingly diverse: from housewives, through office workers, all the way to professors of mathematics. These are usually intelligent people, above average, but rarely brilliant in the colloquial sense of the word. More important than intelligence itself turns out to be mental attitude: the superforecaster does not rely on one great idea, but assumes that the world is too complex to be reduced to a single slogan. They are not “patriots” of a single concept, which they honorably defend at all costs and they approach even their own area of interest self-critically ⁴.

Most importantly, they are “perpetual betas” and despite their accumulated body of experience, avoid the arrogant posture of the alpha, rooted in the illusory conviction that they know everything best. Instead, they honestly admit, all the time, that despite their own systematic observations, producing certain repeatable effects, they are still merely “sliding across the surface” (of the world’s complexity), which stimulates learning and the updating of personal convictions ⁵.

Futurology

Tetlock’s research and that of his team do not negate the value of expert knowledge as such.

Quite the opposite, they suggest which traits describe outstanding “futurologists” (Table 1), capable of predicting the future more effectively, at least within a short time horizon ¹. The dimension of time is in fact the greatest limitation of accurate prognosis, because the further we try to “run ahead” into the future, the more expertise begins to resemble the prophecies of Nostradamus ⁶. It turns out, then, that simple statistical models cope better with solving long–term problems than expert interpretation ⁷, which, even in the case of the best forecasters, explains only about 20% of the total variance in outcomes—significantly less than we would intuitively expect from “recognized” experts ⁴.

A critical factor distorting judgment is excessive self–confidence (overconfidence). Even the best–calibrated claims may have an actual accuracy of around 68%, although too much confidence may increase the risk of error ⁹. Therefore, the precision of conjectures is determined more by style of thinking than by the expert label itself. This leads to one of the more controversial conclusions formulated by Tetlock: markers of prestige, such as a doctorate or specialization, turn out to be secondary to the style of thinking ¹⁰. The source of the problem may also lie in the expert environment itself, which does not select outstanding forecasters on the basis of scoring the accuracy of the claims they make. It may also be characterized by resistance to teamwork, since egotistical specialists (desiring the same thing) often come into conflict with one another ¹¹.

1.	HEALTHY SCEPTCISM
An attitude consisting in maintaining cognitive distance toward one’s own object of interest. Avoiding excessive fascination with a single idea, understanding both its usefulness and its limitations, and resisting generalisation claims, that is, the belief that a given concept “explains everything.”
2.	EPISTEMIC HUMILITY
Superforecasters feel comfortable with uncertainty and avoid categorical “for sure.” They recognise that knowledge of the world is always somewhat incomplete, which is why they formulate judgments in terms of probabilities and boundary conditions rather than rhetorical certainty.
3.	ACTIVE OPEN–MINDEDNESS
A willingness to seek out information that does not confirm one’s own thesis, but may weaken or modify it—even if confronting such information involves psychological discomfort.
4.	(TITULAR) PERPETUAL BETA
Treating one’s own judgments as working hypotheses rather than declarations of identity. Excellent forecasters update their assumptions in real time, closely observing the “incoming” flow of new data.
5.	GRANULAR THINKING
The ability to break complex questions down into smaller components and translate qualitative assumptions into precise estimates of probability. Rather than thinking in “black” or “white,” superforecasters operate with scales and ranges of credibility, on the basis of which they get ahead of events that seem inevitable.
6.	COGNITIVE EFFORT
A tendency toward deep reflection, searching for valuable sources of data, comparing perspectives, and treating forecasting as a task that requires effort. A distinguishing feature is also the motivation to be accurate, as well as the sheer pleasure of solving games and puzzles.

Table 1. Framework of predictive thinking that help maintain distance, organise uncertainty and temper personal impulses toward making bold judgments—derived from the social and political sciences ¹. The next paragraph constitutes an attempt to “transfer” them to the sciences of sport and human health.

Choices for future

If rehabilitation and training can so easily be related to the metaphor of a “military polygon” of ideas, then perhaps what should distinguish a good clinician and coach is precisely the aforementioned attitude of the “fox”—resistant to trends that, together with their grand promises, come and go with each successive decade. Every skilled practitioner, at some point in their career, sees how important coherent clinical reasoning is—not only in the context of diagnosis, that is, of what was and what already is, but also of prognosis for tomorrow and prevention for a lifetime. The questions: “what is wrong with me?”, “how should this be treated?” and “when will I recover?” (if it is treatable at all) are the daily bread of medical professionals ¹².

With the objectification of athlete–monitoring processes, the importance of looking into the “crystal ball” of data, interpreting it and offering appropriate assumptions about “when can I start running after injury?”, “what tactics should be adopted for the upcoming match?” or even “what jumps or lifts should be taught in the weight room?” continues to grow ¹³. Questions directed toward areas of (unrealised) performance potential, elevated injury risk or uncertainty surrounding the diagnosis itself are examples of efforts that determine the future choices of practitioners ¹⁴. Forecasting is not only a standard of contemporary rehabilitation and sport, but also an expectation placed upon professionals to formulate increasingly vivid scenarios of the fates yet to come—offered reasonably, albeit polemically, since the modeling of events over a long time horizon—regardless of one’s own stubbornness—is only possible backward, not forward ¹⁵.

In this sense, the agility of the “fox” is indispensable, as is its ability to connect the dots and the wisdom to notice fleeting, momentary circumstances that matter and may bring about or rule out future changes.

Summary

The argument presented shifts the center of gravity from the proud expert figure to a nimble style of thinking which is capable—purely theoretically—of producing more accurate predictions with the smallest possible error. Tetlock’s body of work sobers overly bold opinions and exposes the forecasts of the “great connoisseurs” of geopolitics, climate, medicine or sport, who, entirely seriously and without a trace of humor, wave their grand doctrine in front of our noses ¹⁶.

Are there, however, exceptions to Tetlock’s rule—are there circumstances in which it pays to be the proverbial “hedgehog”? As it turns out, fundamental matters such as education, medical competence, evidence–based practice or the good of the patient are areas worth defending at all costs ¹⁷—just like freedom of speech, the rule of law, democracy or the inviolability of borders ¹⁸. Moreover, to become a superforecaster (and we already know that this is the latest trend), a certain basic range of competencies is indispensable: something between analytical and critical thinking.

Skeptics of Tetlock’s postulates argue that “thinking” alone is of little use if someone does not understand probability theory, lacks quantitative fluency, cannot diagnose correctly and completely fails to grasp the meaning of the professional context. Forecasting is therefore something that comes only after orienting oneself to “what is going on?”, because without accurate evaluation, there will be no good prediction ¹⁹. After all, how could a professor of mathematics predict the future more accurately, for example in the context of health, than a doctor—perhaps proud—but one who has learned all the nuances of his narrow practice through many years of specialisation, especially when a sporting career or the patient’s life is at stake?

Przeczytaj więcej

¹ Tetlock PE. Superforecasting: The Art and Science of Prediction. Macmillan Publishers (2016).

² Jones E et al. Civilian Morale During the Second World War: Responses to Air Raids Re–examined. Social History of Medicine (2004).

³ Raport badań społecznych opracowany przez Razumkow Centre (2024). OPEN ACCESS.

⁴ Tetlock PE. Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press (2005).

⁵ Atasanov P et al. Small Steps to Accuracy: Incremental Belief Updaters are Better Forecasters. Organizational Behavior and Human Decision Processes (2020).

⁶ Tetlock PE et al. Long–Range Subjective–Probability Forecasts of Slow–Motion Variables in World Politics: Exploring Limits on Expert Judgment. Futures & Foresight Science (2023).

⁷ Mandel D et al. A Quantitative Assessment of the Quality of Strategic Intelligence Forecasts. Defence Research and Developmend Canada (2014).

⁸ Mandel DR et al. Proceedings of the National Academy of Sciences of the United States of America (2014). OPEN ACCESS.

⁹ Mandel DR et al. Accuracy of Forecasts in Strategic Intelligence. Proceedings of the National Academy of Sciences of the United States of America OPEN ACCESS.

¹⁰ Mellers BA et al. The Psychology of Intelligence Analysis: Drivers of Prediction Accuracy in World Politics. Journal of Experimental Psychology: Applied (2015).

¹¹ Tetlock PE et al.Intelligent Management of Intelligence Agencies: Beyond Accountability Ping–Pong. American Psychology (2011).

¹² Bullock GS et al. Clinical Prediction Models in Sports Medicine: A Guide for Clinicians and Researchers. Journal of Orthopaedic and Sports Physical Therapy (2021).

¹³ Gabbett TJ et al. The Athlete Monitoring Cycle: A Practical Guide to Interpreting and Applying Training Monitoring Data. British Journal of Sports Medicine (2017). OPEN ACCESS.

¹⁴ Ruddy JD et al. Modeling the Risk of Team Sport Injuries: A Narrative Review. Frontiers of Physiology (2019). OPEN ACCESS.

¹⁵ Yung KK et al. Judgement and Decision Making in Clinical and Return-to-Sports Decision Making: A Narrative Review. Sports Medicine (2015). OPEN ACCESS.

¹⁶ Mellers B et al. Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions. Association of Psychological Science (2015).

¹⁷ Sackett DL et al. Evidence–Based Medicine: How to Practice and Teach EBM. 2nd ed. Edinburgh: Churchill Livingstone (2000).

¹⁸ United Nations. Universal Declaration of Human Rights. New York: United Nations (1948).

¹⁹ Keil FC. When and Why do Hedgehogs and Foxes Differ? Critical Review (2011). OPEN ACCESS.