Language models are weird for the same reason human cultures are weird
Summary
An essay arguing that language models exhibit weird behaviors because they are adaptive systems learning from sparse, coarse feedback. It traces the goblin tic across model generations, explains chunky post-training and overfitting, and draws a parallel with human culture. The piece argues for scientific investigation and mechanistic interpretability to understand and potentially mitigate these quirks.