wxpath - declarative web crawling with XPath
Summary
wxpath is a Python-based declarative web crawler that expresses traversal and extraction in XPath. It supports asynchronous crawling, a url(...) operator for starting points, and deep crawling with ///url(...), with results streamed as they are discovered; the project includes design docs, examples, and configuration guidance.