The Web Scraping Club
Subscribe
Sign in
Home
News
The Lab
Advertise on TWSC
Proxy pricing benchmark
Consulting
Archive
About
The Lab
Latest
Top
Discussions
THE LAB #112: Intercepting a Website's Internal API Calls
Fifteen tools against two live bot detectors, and only four passed the strict one.
Jul 30
•
Pierluigi Vinciguerra
THE LAB #111: The OSS browser automation stealth benchmark
Fifteen tools against two live bot detectors, and only four passed the strict one.
Jul 23
•
Pierluigi Vinciguerra
1
1
THE LAB #110: Why Shopee is so hard to scrape
Usually, when I dig into an app’s request signing, I like to end with something practical you can use, like a script, an offline signer, or at least a…
Jul 16
•
Pierluigi Vinciguerra
1
THE LAB #109: UDP, Proxies, and the WebRTC Leak
The proxy carries the UDP. The browser refuses to hand WebRTC over. We tested every combination to find where your IP actually leaks.
Jul 2
•
Pierluigi Vinciguerra
1
1
THE LAB #108: How LLMs and Manus AI are transforming bot detection
AI is making faster and cheaper to deobfuscate code. This changes how antibot solutions should be built.
Jun 26
•
Pierluigi Vinciguerra
3
THE LAB #107: Reversing Shopee's native crypto with Ghidra
Shopee hides its crypto in a native library. We read it in Ghidra and rebuild it in Python, byte for byte.
Jun 11
•
Pierluigi Vinciguerra
6
1
THE LAB #104: Bypassing AWS WAF on IMDB with Scrapling
An hands-on test on tools for TLS spoofing and Scrapling
May 14
•
Pierluigi Vinciguerra
3
1
THE LAB #103: Bypassing DataDome-Protected Websites in the Agentic Era
Fifteen browser configurations, one tough anti-bot, and only a couple made it to the cart
Apr 30
•
Pierluigi Vinciguerra
1
4
THE LAB #102: How Fast Can You Call Polymarket's APIs?
Three languages, four locations, 1,000 requests. The biggest speed gain has nothing to do with code.
Apr 16
•
Pierluigi Vinciguerra
4
THE LAB #101: Building an Internal Knowledge Base for Your Scraping Team
Every scraping team that survives long enough develops the same disease.
Apr 2
•
Pierluigi Vinciguerra
2
1
THE LAB #100: Hybrid Scraping - One Browser Login, Thousands of HTTP Requests
Building a pipeline that uses Camoufox for authentication and curl_cffi for extraction on Akamai-protected targets.
Mar 19
•
Pierluigi Vinciguerra
3
1
THE LAB #99: HTTP Caching for Web Scraping
How Conditional Requests Can Cut Your Proxy Bill, using HTTP caching.
Mar 5
•
Pierluigi Vinciguerra
2
1
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts
