The importance of limiting syndication feed requests in some way
Summary
This piece argues that syndication feed requests should be limited and that User-Agent headers should clearly identify the client. It highlights the surge of high-volume crawlers used for data gathering (including for LLM training) and recommends blocking or throttling such traffic while ensuring transparent identification.