Bellwether
A model that notices it is getting worse, and replaces itself.
Ingested
17.17% of ingested edits already carried a revert tag when we first saw them. A floor, not a rate — the tag array is frozen at ingestion, so for a live edit it cannot yet know about a revert that has not happened.
Revert rate, matured 48h+
| Editor | Sample | Reverted | Rate | Population | Weighted rate |
|---|---|---|---|---|---|
| logged out | 145,236 | 38,329 | 26.39% | 290,472 | 26.39% |
| registered | 53,430 | 2,060 | 3.86% | 1,781,000 | 3.86% |
Every rate carries its sample size and the maturity it was measured at. A revert rate without a maturity is a lower bound, not a rate.
Logged-out edits are sampled at 50% and registered ones at 3%, so the sample is not the population. Both rates are shown — publishing only one would mean choosing which.
How good is it?
| Measurement | PR-AUC | vs logged-out | Sample | Matured at |
|---|---|---|---|---|
| Backtest (offline, backfilled) | 0.2560 | +0.1072 | 11,188 | 48h · provisional |
| Live (register, all events) | 0.3029 [0.2940, 0.3118] | 0.1362 | 75,995 · 9677 pos | 168h |
| Live (register, maturity cohort) | 0.2697 [0.2474, 0.2959] | 0.1295 | 11,177 · 1190 pos | 48h |
These are not the same measurement and the higher one is not the answer. The backtest scores a backfilled census whose labels were harvested at leisure, with folds trained on data adjacent to what they score. The live rows grade forecasts written to an append-only register before the outcome existed, on a different population, under a scoring lag. Where they disagree, the live one is true.
The two live rows are also not interchangeable. The maturity cohort is the tenth of events the labeller checks densely enough to grade at 48 hours; everything else waits for its seven-day check. The cohort figure arrives first over far fewer events.
Excluded from the live figures: 55,061 not yet matured, and 7,611 scored after their own outcome was already visible. The second exclusion is correct and is not neutral — those edits were reverted fast, so dropping them removes real positives, which is why their own base rate is published beside the count.
Against Wikimedia’s own model
Bellwether 0.2978 against Lift Wing 0.2390 on 1085 events both models scored — a margin of 0.0588.
Wikimedia runs revertrisk-language-agnostic in production against the same edits. The SRS recorded, before any model here existed, that it is expected to win — and the comparison is published whichever way it falls. It is paired on the events both models scored, which is why the Bellwether figure quoted here is not the one in the table above.
Pipeline
| Job | Last run | Outcome |
|---|---|---|
| agreement | 745 min ago | success |
| apply_reverts | 114 min ago | success |
| gapfill | 114 min ago | success |
| ingest | 121 min ago | success |
| label | 287 min ago | partial |
| label_secondary | 120 min ago | success |
| liftwing | 845 min ago | success |
| metrics | 1377 min ago | success |
| reconcile | 1033 min ago | partial |
| reproduce | 1000 min ago | partial |
| retention | 1089 min ago | success |
| score | 113 min ago | partial |
| seal | 1090 min ago | success |
| train | 35436 min ago | success |
| triggers | 891 min ago | success |
Staleness is stated rather than implied. Counts with no indication of when they last changed look identical to counts that stopped changing days ago.
These figures were read from the API at 2026-11-02 04:34 UTC and are served from a cache that refreshes in the background. The API sleeps when idle; caching is what stops that from becoming your wait.