Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
Summary
arXiv:2504.09762 argues against anthropomorphizing intermediate tokens produced during reasoning (often labeled as 'reasoning traces' or 'thinking traces') in language models. The paper provides evidence that these traces do not faithfully represent the model's internal thoughts and warns that treating them as such can mislead researchers and misuse interpretability. It calls for avoiding this anthropomorphism and discusses implications for how we study and evaluate model reasoning, with ICML 2026 as the venue context.