Types and Neural Networks
Summary
The post surveys methods to produce typed output from LLMs, comparing post-training type enforcement with code generation loops and pre-token type checks. It discusses learning types during training via differentiating through structure, and how to encode the rules of a programming language into training to improve scalability. It cites AlphaZero-like approaches and CHAD in context.