Hausfather et al. — climate model projections vs. observed warming

Source class: Scientific papers
Metric: Implied transient climate response error; observed-vs-projected warming
Reported value: 14 of 17 surveyed climate models from 1970–2007 produced projections within natural-variability range of subsequent observed warming when adjusted for actual emissions
Measured: 2020-01-04

Context

Evaluation of how well climate model projections published 1970-2007 actually tracked observed global mean surface temperature in the years following. Once corrected for actual greenhouse-gas emissions (which differed from modelers' assumed emissions), most models were skillful. A landmark finding for scoring scientific model projections — directly applicable to AI-model calibration analogues.

Citation

Hausfather, Z., Drake, H. F., Abbott, T., & Schmidt, G. A. (2020). Evaluating the Performance of Past Climate Model Projections. Geophysical Research Letters 47(1), e2019GL085378.

https://doi.org/10.1029/2019GL085378

What Phase 1 launch will add

Calibration Ledger has not independently recomputed the value above. Phase 1 launch (target Q3 2027, gated on prerequisites) will add for this source class:

Independent recomputation from the original outcome data, under data-licensing agreement
Time-windowed breakdown (rolling 3-month, 12-month, lifetime)
Cross-domain calibration (does this source calibrate uniformly across topical verticals?)
Append-only timestamp anchoring of every score so retroactive revisions are visible
Per-source citation page with full Murphy decomposition (Reliability − Resolution + Uncertainty)

Other findings in the same source class

Open Science Collaboration — psychological science replication rate — Replication rate + effect-size shrinkage
Camerer et al. — social science experiment replication (Nature/Science 2010-2015) — Replication rate + median effect-size shrinkage

All other findings

Good Judgment Project Superforecasters (Human forecasters)
Metaculus community-prediction aggregate (Forecaster aggregator platform)
Manifold Markets — platform calibration (Prediction market)
GPT-4 (OpenAI) — pre-RLHF vs post-RLHF calibration (AI models)
Sell-side equity analysts — earnings forecast accuracy (Analyst firms)
Anthropic — Claude / language model self-knowledge (AI models)
Federal Reserve Survey of Professional Forecasters — GDP / inflation accuracy (Analyst firms)

All beta findings — at-a-glance + JSON + BibTeX exports
Methodology v1.1 — full Brier + Murphy + append-only framework
Operator track record — methodology applied to Paulo de Vries’s own dated forecasts
Source classes — what each of the 6 source classes will score at Phase 1
Roadmap — milestone status + Q3 2027 launch gate + kill criterion

Last verified: 2026-04-28. Cited; Calibration Ledger has not independently recomputed this finding. Independent recomputation in Phase 1 (Q3 2027). Operator: Paulo de Vries. Contact: contact@calibrationledger.com.