Scanning for Pangram Errors
Summary
The post analyzes Pangram, an AI writing detection tool, focusing on the concern of memorization in ML detectors. It describes a hands-on test using old, non-digitized books scanned via OCR to check for false positives, finding that Pangram largely labels text as human and that any AI labels arose from OCR hallucinations. The author concludes Pangram is likely not memorizing its training data and emphasizes the importance of human review for critical uses.