Facebook's Fascination with My Robots.txt
Summary
A blogger reports unusual traffic from Facebook's crawler repeatedly requesting the robots.txt file on a self-hosted Forgejo instance, using Facebook's external hit user-agent and Meta IP ranges. The post analyzes why only robots.txt is accessed, discusses Facebook's ExternalHit purpose, and raises questions about bandwidth and global crawler activity, without presenting official statements from Facebook.