jhao104/proxy_pool

Analysis updated 2026-05-18

★ 23,317PythonAudience · developerComplexity · 3/5LicenseSetup · moderate

Mindmap

mindmap
  root((proxy_pool))
    What it does
      Gathers free proxies
      Verifies they work
      Serves via API
    How it works
      Scheduler collects
      Redis storage
      Web endpoints
    Use cases
      Web scraping
      Avoid rate limits
      Data collection
    Tech stack
      Python
      Redis
      Docker
    Key features
      Custom sources
      Bad proxy removal
      Random selection

mindmap root((proxy_pool)) What it does Gathers free proxies Verifies they work Serves via API How it works Scheduler collects Redis storage Web endpoints Use cases Web scraping Avoid rate limits Data collection Tech stack Python Redis Docker Key features Custom sources Bad proxy removal Random selection

Click or tap to explore — scroll the page freely

What do people build with it?

USE CASE 1

Build a web crawler that makes thousands of requests without getting blocked by rotating through different IP addresses.

USE CASE 2

Collect data from websites that rate-limit or ban repeated requests from the same source.

USE CASE 3

Test your application's behavior when accessed from different geographic locations using proxy rotation.

What is it built with?

PythonRedisDocker

How does it compare?

	jhao104/proxy_pool	deepseek-ai/deepseek-coder	vanna-ai/vanna
Stars	23,317	23,251	23,386
Language	Python	Python	Python
Setup difficulty	moderate	moderate	moderate
Complexity	3/5	3/5	3/5
Audience	developer	developer	pm founder

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · moderate Time to first run · 30min

Requires Docker to run Redis and the proxy service, API key or proxy source configuration may be needed.

Use freely for any purpose including commercial, as long as you keep the copyright notice.

In plain English

proxy_pool is a Python tool that automatically gathers and maintains a pool of free proxy IP addresses for use in web scraping. A proxy is an intermediary server that lets your program fetch web pages using a different IP address, which helps avoid being blocked by websites that restrict repeated requests from the same source. The tool works in two main parts running at once: a scheduler that continuously collects proxy addresses from several free proxy listing websites, verifies whether each one actually works, and stores the valid ones in a Redis database (a fast in-memory data store), and a small web API server that your scraper can query to get a working proxy on demand. You interact with it through simple web endpoints, ask for a random proxy, fetch all proxies, check the count, or delete a bad one. When a proxy stops working, your scraper can report it for removal and request a fresh one. The system also lets you add your own proxy sources if the built-in free ones are not reliable enough for your needs. You would use this when building a web crawler or data collection script that needs to make many requests without triggering rate limits or bans. It is written in Python and uses Redis for storage, and can be run directly or via Docker.

Copy-paste prompts

Prompt 1

How do I set up proxy_pool with Redis and start collecting proxies from free sources?

Prompt 2

Show me how to query the proxy_pool API to get a random working proxy for my web scraper.

Prompt 3

How can I add my own custom proxy sources to proxy_pool instead of using the built-in free ones?

Prompt 4

What's the best way to report a dead proxy to proxy_pool and request a fresh one in my scraper?

Frequently asked questions

What is proxy_pool?

Automatically collects, verifies, and serves working proxy IP addresses for web scraping via a simple API, storing them in Redis.

What language is proxy_pool written in?

Mainly Python. The stack also includes Python, Redis, Docker.

What license does proxy_pool use?

Use freely for any purpose including commercial, as long as you keep the copyright notice.

How hard is proxy_pool to set up?

Setup difficulty is rated moderate, with roughly 30min to a first successful run.

Who is proxy_pool for?

Mainly developer.

Open on GitHub → Explain another repo

This repo across BitVibe Labs

Scan in gitsafehub Deploy in gitdeployhub jhao104 on gitmyhub

Verify against the repo before relying on details.