PSSA: a non-transformer language model written from scratch in Rust
Summary
The article introduces PSSA, a non-transformer language model implemented in Rust, featuring a recurrent state-space core, episodic memory, and plastic weights. It compares PSSA to a transformer on WikiText-103, showing faster generation and lower training loss, and discusses generalization on held-out data. It also provides repository details, run instructions, and avenues for collaboration.