LLM Neuroanatomy II: Modern LLM Hacking and hints of a Universal Language?
Summary
This post (LLM Neuroanatomy II) extending Part I, investigates whether the Repeat Your Self (RYS) relayering technique generalizes to stronger open-models like Qwen3.5-27B. It presents a three-phase view of Transformer anatomy (Encoding, Reasoning, Decoding), supports it with heatmaps and cosine-similarity analyses across languages, and shows that contiguous mid-stack blocks offer the best efficiency frontier. The work includes beam search, sparse repeats, surrogate modeling, and released RYS variants and code/models for community experimentation.