Progressive encoding and decoding of 'repeated' protobuffer fields
Summary
The post discusses progressive encoding and decoding of repeated protobuf fields to enable streaming of large traces (e.g., Perfetto traces) without loading entire messages into memory. It covers the protobuf wire format basics (varints, LEN, embedded messages), and demonstrates progressive serialization and deserialization techniques using Rust/prost with practical examples. It also explains the challenges with nested and repeated fields and how to implement a streaming approach safely.