Générer un Excel volumineux avec Airflow sans dépassement de mémoire
Summary
The article explains why exporting very large Excel files in Airflow can cause Out Of Memory errors and presents a memory-stable streaming solution. It details reading from MySQL with a non-buffered cursor and writing with openpyxl in write-only mode, processing data in batches to keep memory usage bounded. It also notes AWS Fargate memory limits and how this streaming approach improves pipeline reliability.