Fairlearn 0.15 fixes a bug that could hide model fairness gaps
The new release corrects a weighted single-member group calculation that could falsely report no demographic disparity, and adds group-level ROC plots.
A false all-clear in a fairness metric
Fairlearn 0.15.0 ships a fix for a case in which a model fairness check could incorrectly say there was no demographic disparity. The project’s release notes say a weighted group containing exactly one sample could make demographic_parity_difference return zero and demographic_parity_ratio return one regardless of the actual selection rates. In the merged fix’s worked example, the difference should be 0.25 but the earlier calculation returned zero. That is a narrow input condition, not a finding that every Fairlearn audit was wrong.
The same release adds ROC curves broken out by sensitive group and automatic plotting of bootstrap confidence intervals when they are available. It also fixes how CorrelationRemover centers multiple sensitive features. The maintainer’s pull request says an AI agent found and fixed the single-member bug; that is the contributor’s disclosure, not an independent assessment of the agent. Users upgrading must also account for new minimum versions of Python 3.11 and scikit-learn 1.6.