All stories
Trail of Bits Patching Plugins Ecosystem Community reports

Prove the fix, then undo it: Trail of Bits ships agent patching plugins

The new patching-bugs and regression-hunt plugins turn bug reports into scoped repairs with live reproduction, independent replay and an explicit evidence trail.

By AITrending2 min read
Sources (4)

Why now Trail of Bits announced both plugins on October 21, following its review of regressions in merged Patch the Planet fixes. The launch supplies a concrete workflow beyond asking an agent to review a diff.
Trail of Bits diagram routes regression candidates through reproduction, independent replay and adjudication before reporting.
Author-published regression-hunt workflow. This diagram describes the proposed evidence pipeline; AITrending did not run the plugins or reproduce their evaluation. Source: Trail of Bits / Anish Naik and Marcelo Morales

Trail of Bits has released two agent plugins built around an unusually concrete demand: prove the bug, prove the patch, then bring the bug back by reverting the patch. Announced October 21, patching-bugs and regression-hunt package workflows developed through Patch the Planet, the security firm’s joint initiative with OpenAI.

The first plugin takes a supplied bug or falsifiable hypothesis in a local Git repository. Fresh agents reproduce and validate it, develop a scoped repair, then run the same reproduction against the baseline, patched and completely reverted trees. The expected pattern is failure, success, failure. That final arm helps distinguish an actual fix from an unrelated environment or test change. The workflow creates local evidence and a local commit; its guide says it never pushes or opens a pull request.

A separate agent must replay the evidence

Regression-hunt examines selected pull requests or commits, traces changed code to reachable behavior, reproduces suspected regressions and asks a fresh agent to repeat the proof of concept. An adjudication step applies explicit exclusions before accepting a finding. Missing live evidence leaves a result inconclusive rather than automatically confirmed.

Trail of Bits reports 25% fewer false-positive findings than a baseline Codex agent prompted to review a pull request for regressions, with a 95% confidence interval of 14% to 38%. This is its preliminary evaluation using 21 confirmed regressions from its own merged patches, not an independently reproduced general benchmark. The firm also says stricter evidence rules left some known regressions borderline, so lower noise came with a detection tradeoff.

Installation and project context matter

These are plugins for an existing agent harness, not a new model or standalone security scanner. The English marketplace guide supports Claude Code and Codex. For Codex, it documents codex plugin marketplace add trailofbits/skills, followed by installing each named plugin from that marketplace. Patching-bugs requires regression-hunt for its final check.

Users supply a working target build, appropriate execution access and run directories; the host provides agent dispatch, credentials and model access. Helper scripts require Python 3.13 or later and use uv. A project’s THREAT_MODEL.md, PATCHING_GUIDELINES.md and AGENTS.md tell the workflow what behavior is supported and what counts as a valid fix. Model costs and permissions remain those of the configured host; the announcement does not establish universal platform support or a fixed cost per repair.

AITrending read the announcement and repository guides but did not run the plugins or reproduce the evaluation.

Related reading