Reflecting to optimise
Summary
An accessible dive into optimization on the probability simplex, comparing softmax reparameterization, projected gradient descent, and mirror descent with different Bregman divergences. The post explains why gradient steps can violate simplex constraints and how projection or mirror maps preserve them, with practical notes on sparsity, high-dimensional behavior, and applications to protein binder design.