The Web Scraping Club | Pierluigi Vinciguerra | Substack

Best Practices for Ethical Web Scraping

A discussion on the golden rules for scraping websites ethically

21 hrs ago • Federico Trotta

Build an AI Agent for Scraping and Analyzing Research Papers

Let’s build an AI agent in Python for research paper scraping and analysis.

Dec 14 • Antonello Zanini

Using NLP for Entity Extraction From Scraped Data

From theory to practice: how to extract entities from textual scraped data using NLP

Dec 7 • Federico Trotta

Faster Web Scraping with HTTP/3 Web Requests

Let’s explore why HTTP/3 can be a game-changer for automating requests in a web scraping scenario!

Nov 30 • Antonello Zanini

Using AI to Detect Patterns in Scraped Data

A practical guide on finding patterns in scraped data with advanced techniques

Nov 23 • Federico Trotta

Most Popular

THE LAB #1: Scraping data from an app

Sep 4, 2022 • Pierluigi Vinciguerra

THE LAB #3: Scraping Cloudflare protected websites

Sep 27, 2022 • Pierluigi Vinciguerra

THE LAB #73: How to Bypass Cloudflare in 2025

Jan 23 • Pierluigi Vinciguerra

Ten years of web scraping: a personal perspective about selling web data

Mar 24, 2024 • Pierluigi Vinciguerra

Platinum Partner

Discover Decodo Web Scraping API

Meet the Platinum Partner of The Web Scraping Club

Feb 6 • Pierluigi Vinciguerra

The Great Web Unblocker Benchmark - Cloudflare Edition

Testing different web unblockers against Indeed.com

Sep 22, 2024 • Pierluigi Vinciguerra

The Web Unblocker Cost Benchmark

Price comparison between the most well-known web unblockers on the market

Dec 24, 2023 • Pierluigi Vinciguerra

Web Scraping

Best Practices for Ethical Web Scraping

A discussion on the golden rules for scraping websites ethically

21 hrs ago • Federico Trotta

Build an AI Agent for Scraping and Analyzing Research Papers

Let’s build an AI agent in Python for research paper scraping and analysis.

Dec 14 • Antonello Zanini

Using NLP for Entity Extraction From Scraped Data

From theory to practice: how to extract entities from textual scraped data using NLP

Dec 7 • Federico Trotta

Faster Web Scraping with HTTP/3 Web Requests

Let’s explore why HTTP/3 can be a game-changer for automating requests in a web scraping scenario!

Nov 30 • Antonello Zanini

Using AI to Detect Patterns in Scraped Data

A practical guide on finding patterns in scraped data with advanced techniques

Nov 23 • Federico Trotta

Pydoll: WebDriver-Free Browser Automation in Python

Let’s explore Pydoll, the Python library for browser automation without WebDrivers, featuring scraping-ready features—complete with practical examples!

Nov 16 • Antonello Zanini

Gold Partners

Discover the NetNut's Website Unblocker

Get a huge 55% off for residential proxies

Limited Offer from Oxylabs: 50% Off Residential Proxies

Tailored to meet all your Static ISP, Residential and Datacenter proxy needs. 1 GB Trial here

Rayobyte is offering a 30% discount on rack rates for residential proxies by emailing sales@rayobyte.com.

Reliable APIs for Web Data Extraction

AI

Build an AI Agent for Scraping and Analyzing Research Papers

Let’s build an AI agent in Python for research paper scraping and analysis.

Dec 14 • Antonello Zanini

Using NLP for Entity Extraction From Scraped Data

From theory to practice: how to extract entities from textual scraped data using NLP

Dec 7 • Federico Trotta

Using AI to Detect Patterns in Scraped Data

A practical guide on finding patterns in scraped data with advanced techniques

Nov 23 • Federico Trotta

When Browsers Start to Think: ChatGPT Atlas, Stagehand, Cursor, and the Future of Web Scraping

How recent browser integrations with LLMs are changing the way we explore and scrape the web.

Nov 2 • Pierluigi Vinciguerra

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts