Why don’t machine learning research agents overfit?
Summary
Amazon Science reports that ML research agents learn compressible, structure-based strategies and often do not overfit. Through an explorer/compressor/reproducer experiment, the study shows that very short prompts (as little as 16–32 tokens) can reproduce performance on multiple datasets, suggesting memorization is limited and true structure is captured. The piece discusses Occam's razor, compression as a diagnostic for overfitting, and caveats regarding pretraining memorization and dataset freshness.