← hmda-audit
Chart 4 of 6

Race was never an input, yet a probe reads it at 0.68 AUC from the allowed features

Race was never an input, yet a probe reads it at 0.68 AUC from the allowed features
Linear race-probe AUC, Black vs. White applicants, mean over 5 seeds; 0.5 is chance
0.50
Allowed input features0.678
Hidden layer 10.673
Hidden layer 20.659
Random-init layer 2 (control)0.642
Shuffled labels (control)0.499
Note: Race leaks through ordinary credit fields; the strongest proxies by one-feature AUC are property value (0.599), income (0.595) and debt-to-income band (0.594). A probe reads race a little less easily from each hidden layer than from the raw inputs, so training does not make race easier to read than it already is from the allowed features. This measures what information a probe can extract, not discrimination or intent; numbers are sample means over 5 seeds on a 500,000-row national sample, and this small network's own denial AUC is 0.838 (logistic regression 0.784, LightGBM 0.859).
Source: public HMDA filings, 2023 to 2025. hmda-audit analysis.
Patrick Taylor
Back to everything else