The Web Scraping Club

Home
Consulting
The Lab
Special offers for our readers
Archive
Leaderboard
About
Rethinking the web browser
How will web browsers evolve to meet the needs of AI?
 • 
Katie Hallett
The Scriptwall: Why Google is hiding its SERP content behind Javascript
What are the implications of this move for the web scraping industry?
 • 
Pierluigi Vinciguerra
The 2025 web scraping tech stack
Start this year with a tech stack for every season
 • 
Pierluigi Vinciguerra
2
THE LAB #72: Advanced logging in Playwright
RabbitMQ, screenshots and system monitoring for your Playwright scrapers
 • 
Pierluigi Vinciguerra
The Dirty Little Secret of Internet's Data
Everyone Scrapes It, But Very Few Admit It
 • 
Andrea Squatrito

December 2024

The Web Scraping Club 2024 Wrap
What happened in 2024 and what to expect for the 2025
 • 
Pierluigi Vinciguerra
2
THE LAB #71: Sending Scrapy logs to RabbitMQ
Saving the logs of your distributed scraping architecture to your database
 • 
Pierluigi Vinciguerra
How We Scraped Global Hotel Data to Track Economic Trends
Tracking Tourism and Economic Flows with Global Hotel Data Scraping
 • 
Andrea Squatrito
How to Parse JSON with Python: A Beginner-Friendly Guide
Tips and tricks for handling JSON in your scraping operations
 • 
Pierluigi Vinciguerra
2
THE LAB #70: Advanced logging in Scrapy
How to extract the most meaningful metrics from your scrapers
 • 
Pierluigi Vinciguerra
How Scraping the Web Became an Expensive Business
How the industry moved from gathering low-hanging fruits to an expensive engineering challenge.
 • 
Andrea Squatrito
2
Optimizing Proxy Usage for Large-Scale Scraping
How to cut costs of scraping with some simple rules
 • 
Pierluigi Vinciguerra
© 2025 Pierluigi
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great culture