A change that should do nothing changed 203 of 864 answers, and the placebo changed none
A change that should do nothing changed 203 of 864 answers, and the placebo changed none
Answers changed, out of 864, with the A and B labels in their normal order
Fake swap: the same change to both pictures203 of 864
Placebo: a random signal of the same strength0 of 864
How to read this: the full width is every answer. The fake swap should leave its bar empty, and it does not. The placebo bar is empty, so there is nothing to compare the fake swap against.
Note: With the A and B labels swapped, the fake swap changed 49 answers and the placebo 5. The fake swap's changes went toward one letter 100% and 88% of the time. This is a larger model than in the other charts, so the paper does not combine the two.
Source: Taylor et al., accepted to the InterpScience workshop at NeurIPS 2026. Qwen2-VL-7B-Instruct.