The "color" signal is almost the same as the model's "pick A" habit: 0.99 out of 1
The "color" signal is almost the same as the model's "pick A" habit: 0.99 out of 1
Cosine similarity between each signal and the answer-letter direction, layer 29
0.99 / 1.00similarity between the "color" signal and the "pick A" habitThe "object" signal scores 0.99 too.
How to read this: 1.00 means the two signals point the same way inside the model, 0 means they are unrelated. A signal that really carried color should not sit this close to a letter preference.
Note: Used unchanged on a question about shape instead of color, the color signal gave the same results (correlation 0.9999). The object signal scores 0.9859 to 0.9882 across layers.
Source: Taylor et al., accepted to the InterpScience workshop at NeurIPS 2026. Qwen2.5-VL-3B-Instruct.