δ-Mem: Efficient Online Memory for Large Language Models
Summary
The paper presents δ-Mem, a lightweight online memory mechanism that augments a frozen full-attention backbone with a compact online associative memory state. It compresses history into an 8x8 state updated by delta-rule learning and applies low-rank corrections to attention, improving performance on memory-heavy tasks without full fine-tuning. Useful for enabling longer-context reasoning in LLMs with minimal overhead.