Scrape websites that block regular curl by detecting non-browser TLS fingerprints
Test whether your own website correctly identifies real browsers versus automated tools
Integrate a browser-impersonating HTTP client into an existing program that uses libcurl
Run browser-fingerprint web requests inside Docker containers for automated data collection
Docker images are the easiest path, building from source requires matching the correct TLS library (BoringSSL or NSS) to the target browser variant.
curl is a widely used command-line tool for making web requests from a terminal. When curl connects to a website over HTTPS, it sends a message called a Client Hello as part of the connection setup. This message contains technical details about how curl wants to communicate, and those details differ from what a real browser like Chrome or Firefox sends. Some websites use these differences to detect that they are talking to an automated tool rather than a real human using a browser, and they may block the request or return different content as a result. curl-impersonate is a modified version of curl that makes the connection setup look identical to that of a real browser. It replicates the exact TLS and HTTP/2 handshake patterns produced by Chrome, Firefox, Edge, and Safari, including the specific encryption settings, extensions, and protocol parameters each browser uses. From the website's point of view, the connection appears to come from an actual browser. To achieve this, the project recompiles curl with different underlying TLS libraries. The Firefox version uses NSS, the same TLS library Firefox uses. The Chrome version uses BoringSSL, which is Google's TLS library. Additional changes are made to match the exact configuration parameters each browser sends. Usage is similar to regular curl. For each supported browser version, a wrapper script is provided. Running a command like curl_chrome116 followed by a URL will make the request using Chrome 116's exact network signature. The tool also works as a library that developers can integrate into programs that currently use libcurl. The supported browser list includes multiple versions of Chrome, Edge, Firefox, and Safari on Windows and macOS. Docker images are available for easier deployment. The README links to detailed technical blog posts explaining the underlying mechanics of TLS and HTTP/2 fingerprinting.
← lwthiker on gitmyhub — every repo by this author, as a profile.
Verify against the repo before relying on details.