explaingit

lexiforest/curl_cffi

★ 5,584PythonAudience · developerComplexity · 2/5LicenseSetup · easy

TLDR

A Python HTTP library that makes your scraper or automation script look like a real Chrome, Firefox, or Safari browser by copying their exact connection signatures, bypassing most bot-detection blocks.

Mindmap

mindmap
  root((repo))
    What it does
      Mimics browser fingerprints
      Bypasses bot detection
    Supported browsers
      Chrome Firefox Safari
    Features
      HTTP 2 and HTTP 3
      Async and sync modes
      WebSocket support
      Proxy rotation
    Setup
      Single pip install
      Linux macOS Windows
Click or tap to explore — scroll the page freely

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

Things people build with this

USE CASE 1

Scrape websites that block the standard Python requests library by mimicking a real Chrome or Firefox browser connection signature.

USE CASE 2

Make asynchronous HTTP/2 requests from Python without the standard library limitations, requests and httpx cannot do HTTP/2.

USE CASE 3

Connect to WebSocket endpoints from a Python script while appearing as a real browser to the server.

USE CASE 4

Rotate proxies on a per-request basis in a scraping pipeline while maintaining realistic browser fingerprints throughout.

Tech stack

PythonC

Getting it running

Difficulty · easy Time to first run · 5min

Install with a single pip command, no compilation step needed on Linux, macOS, or Windows.

Use freely for any purpose, including commercial use, as long as you keep the copyright notice.

In plain English

curl_cffi is a Python library for making HTTP requests that can disguise itself as a real browser. Websites use various technical signatures to detect automated scripts, and curl_cffi defeats those checks by mimicking the exact connection patterns that Chrome, Firefox, Safari, and other browsers produce. If your scraper or automation tool keeps getting blocked without a clear reason, that blocking is often based on these signatures rather than your IP address, and this library addresses that. The library wraps a modified version of curl called curl-impersonate. When you make a request with the impersonate option set to a specific browser, the library reproduces that browser's TLS handshake pattern (sometimes called a JA3 fingerprint) and its HTTP/2 negotiation behavior. These are low-level connection details that other Python HTTP libraries like requests or httpx cannot replicate because they use their own TLS stacks. Beyond fingerprint matching, curl_cffi supports modern HTTP versions including HTTP/2 (which the standard requests library does not support) and HTTP/3. It works in both synchronous and asynchronous code, supports WebSocket connections, handles proxy rotation per request, and includes built-in retry logic. Its API is intentionally similar to the requests library so existing code is easy to adapt. A command-line tool called curl-cffi ships with the library for quick manual testing. Installation is a single pip command and works on Linux, macOS, and Windows without any compilation step. The library is MIT licensed. Commercial support, additional browser fingerprints, and integrated anti-bot bypass solutions are available through a separate paid service called impersonate.pro.

Copy-paste prompts

Prompt 1
Write a Python script using curl_cffi to scrape a website that blocks the standard requests library, using the Chrome 120 impersonation profile and async mode for speed.
Prompt 2
Rewrite this requests-based Python scraper to use curl_cffi with async support so I can run multiple requests concurrently without triggering bot detection.
Prompt 3
Show me how to configure per-request proxy rotation in curl_cffi to cycle through a list of residential proxies while keeping the same browser fingerprint.
Prompt 4
Use curl_cffi to maintain a persistent session with cookies while scraping a multi-page site protected by Cloudflare bot detection.
Prompt 5
Explain what a JA3 fingerprint is and show me how curl_cffi reproduces a specific browser's fingerprint when making a request.
Open on GitHub → Explain another repo

← lexiforest on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.