A Fast Path for Fixed-Length Lists in Parquet
Summary
Parquet's nested-list encoding is inefficient for fixed-length lists, causing overhead from def and rep level handling. The Hardwood project adds a fast path by detecting pages that contain fixed-length lists via encoded level streams, allowing bypass of the standard reconstruction. Benchmarks show substantial gains (up to ~3.9x for row reads and ~2.5x for column reads) and the feature is opt-in but slated for default in a future release.