Comparing Python packages for A/B test analysis (with code examples)
Summary
A practical comparison of four Python packages for A/B test analysis—tea-tasting, Pingouin, statsmodels, and SciPy—highlighting what each package does well, what requires manual glue, and how they handle common experimentation tasks. The article includes code examples and discusses power analysis, multiple testing, CUPED, and when to rely on aggregated statistics versus granular data.