Shown one denied and one approved file, the model picks the denied one 86 times in 100
Shown one denied and one approved file, the model picks the denied one 86 times in 100
Denial model with race left out, trained on 2023 and 2024, tested on 2025
86 / 100pairs in the right orderA coin flip gets about 50, down to the black line.
How to read this: each square is one pair of applications. Orange squares are pairs the model put in the right order. Grey squares are pairs it got the wrong way round.
Note: AUC 0.86 (Gini 0.72) for gradient-boosted trees and AUC 0.78 (Gini 0.57) for logistic regression. Tested on the 2025 holdout, about 380,000 decisions per seed, from a 1,500,000-row sample; three random seeds agreed. Ranking well is not the same as being accurate for one applicant.
Source: public HMDA filings, 2023 to 2025. hmda-audit analysis.