ATTN/11 - Paper Tape Is All You Need: Training a Transformer on a 1976 Minicomputer
Summary
ATTN/11 presents a compact encoder-only transformer trained on a 1976 PDP-11/34A, using fixed-point arithmetic and hand-tuned learning rates to fit within tight memory constraints. The project demonstrates how core transformer components can be implemented on vintage hardware, includes prototypes in Sheaf, and provides detailed architectural and memory-layout descriptions along with build/run instructions.