WORK / Infrastructure
curl_reap
Python scraping with a browser-exact fingerprint.
A server sees how a client connects before it sees what the client asks for, and a request that does not look like a browser's can get a different answer. curl_reap is a Python library whose own transport sends a captured browser's Transport Layer Security (TLS) handshake byte for byte, and matches that browser's HTTP/2 behaviour. Self-healing selectors, structured extraction and a concurrent crawler sit on top, in one library.
- Language
- Python
- License
- MIT
- Part of
- Velofy