Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
Summary
Sparks of Weight-Space Programming demonstrates that frontier LLMs can read and write neural network weights directly, enabling weight-space optimization and interpretability without traditional training loops. The post covers experiments that show LLMs reading weights to identify trained tasks, fixing failures by editing weights, and even writing new weight configurations from scratch, with implications for optimization cost, auditing, and future AI tooling.