Honey, I shrunk the embeddings: Matryoshka vs. PCA
Summary
A technical blog post comparing Matryoshka Representation Learning (MRL) and PCA for shrinking embedding dimensions in vector databases, evaluated across BEIR datasets. The piece presents practical insights on when PCA can outperform MRL truncation, discusses in-domain vs out-of-domain fitting, and explores quantization as a complementary compression technique, with a GitHub repo for replication.