Parsimony Ranks AI Coding Agents by How Much Code They Add

A second question for coding-agent benchmarks

Parsimony asks a question that solve-rate leaderboards leave open: how much code did an AI agent add to make its fix? In an October 13 Show HN post, the creator presented an open-source site that measures patches from published coding-agent benchmarks. It counts syntax units such as statements, calls, comparisons, names and literals, then subtracts removed units from added ones. Lines are not the metric, so compressing a patch onto fewer lines does not itself improve its result. The GitHub repository was created in September; the new event is its public Show HN discussion, not a claim that the project was first released today.

What the numbers can and cannot say

The benchmark site ranks models by the mean net units added on tasks their agents solved and shows solve rates beside the size measure. Its repository provides code, method notes and measurement data for inspection. Parsimony takes pass-or-fail outcomes from the source benchmarks rather than rerunning their tests, and its published method excludes tests, documentation and generated files from the implementation footprint. A smaller patch may be easier to maintain, but size alone cannot establish readability, design quality or correctness. A Hacker News commenter raised that distinction directly, arguing that terse code can still be hard to understand. The leaderboard is therefore a useful additional lens, not a verdict on the best coding model.