gproxynet/proxyspin

proxyspin is a Python library that manages a rotating pool of proxy servers for web scraping and automation. It works across three different tools: Scrapy, Playwright, and the requests library, all sharing the same underlying pool and health tracking logic. The README explains that it was built because an older, similar tool for Scrapy has not been updated in years and only worked with Scrapy, while proxyspin is meant to be one small pool usable everywhere. It has no required dependencies beyond Python's own standard library, and it installs with a simple pip install command. The core idea is a ProxyPool object that can be built from a file, a plain list, or a URL such as a provider's export endpoint. Once built, code can pull a proxy from the pool, mark it as failed after an error, or mark it as working again. The pool supports different rotation strategies, including round robin, random, and sticky, where sticky keeps using the same proxy for a given domain or account until it stops working. Failing proxies are automatically benched for a period of time that grows the more they fail, then quietly rejoin rotation on their own. For each of the three integrations, the README gives a short code example. Scrapy users add a downloader middleware and some settings. Playwright users create a browser context using a helper function that assigns a proxy per account or session. Requests users can use a special rotating session object that gets a new proxy for each call. There is also a small command line tool, proxyspin check, that tests a list of proxies at once and can save the working ones to a file. The library also lets developers configure which HTTP response codes count as a sign that a proxy has been banned, such as 403 or 429, so the pool automatically rotates away from a blocked proxy. The project is released under the MIT license.