Debates
A decision model's 0.906 score becomes 0.791 after a clean re-score
Developing 2 sources
T
Typed Decisions benchmark correction
News Oct 13, 2026
The Typed Decisions maintainers replaced a test-exposed model score with a clean re-score after a public challenge.
2 sources attached
What happened lately
Typed Decisions benchmark correction timeline
The Typed Decisions benchmark’s maintainers changed the listed score for Phocinae-Largha-150M-v1 after a public discussion identified training material derived from test-set predictions. The model’s submitter reported a new evaluation after removing that material, and the benchmark now lists the clean result. The episode illustrates why a model’s training and evaluation split matters when reading a leaderboard.