Progress on provable inference for large language models
Summary
Anthropic introduces FlashProve, a software-only provable inference prototype using zkSNARKs to certify that model outputs come from a committed 400B MoE model. It reports offline commitments and online proofs, performance metrics (proofs ~19 MB, ~110 tokens/second on an H100), and Lean-based formal verification, while noting limitations and regulatory implications.