requests-robotstxt. Support for robots.txt in a requests Session

github.com/ambv/requests-robotstxt
by @ambv

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.