Archive - The Web Scraping Club

Why LLM-Ready Scrapers Return Content in Markdown: A Deep Dive

Why do all AI-ready scraping solutions produce Markdown results? Let’s find out!

Feb 22 • Antonello Zanini

THE LAB #98: Scraping Google Search Results in 2026: Device, Location, and Identity

Google does not have one set of results. It has millions. The hard part is knowing which one you are looking at.

Feb 19 • Pierluigi Vinciguerra

How to Avoid Copyright Violations While Scraping

Discover how copyright violations can occur in web scraping and how to avoid them

Feb 15 • Federico Trotta

Google vs IPIDEA: Anatomy of a Residential Proxy Takedown

Google Took Down 16 Million Proxy IPs. Here is Why It Will Not Be Enough.

Feb 8 • Pierluigi Vinciguerra

THE LAB #97: My first week with OpenClaw

160,000 Stars in Two Months: What OpenClaw Means for Scrapers

Feb 5 • Pierluigi Vinciguerra

WebDriver vs Chrome DevTools Protocol (CDP) vs WebDriver BiDi: How We Control Browsers

Do you know how browser automation libraries actually manage to control browsers? Let’s find out!

Feb 1 • Antonello Zanini

January 2026

THE LAB #96: Scraping Nike.com with 5 open source tools

Match your tool to the protection, not the brand

Jan 29 • Pierluigi Vinciguerra

A preview of the Zyte 2026 Web Scraping Industry report

Where the industry is headed according to Zyte

Jan 25 • Pierluigi Vinciguerra

THE LAB #95: Bypassing Cloudflare in 2026

Testing Open Source Browser Automation Tools Against Real Targets

Jan 22 • Pierluigi Vinciguerra

Understanding robots.txt and its Implications

A discussion on the robots.txt file, its legal implications, and what’s to be further taken in to account when scraping

Jan 18 • Federico Trotta

AnyCrawl: Testing the LLM-Ready Web Scraping Service

Let's try AnyCrawl and see what this new kind of web scraping API solution brings to the table!

Jan 11 • Antonello Zanini

Using Python’s Async Features for High-performance Scraping

From theory to practice: why async is better for scraping at scale

Jan 4 • Federico Trotta

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts