Web Scraping in Python tutorials and resources

THE LAB #11: The Anti-Detect Anti-Bot matrix

This post is sponsored by Oxylabs, your premium proxy provider. Sponsorships help keep The Web Scraping Free and it’s a way to give back to the readers some value. In this case, for all The Web Scraping Club Readers, using the discount code WSC25 you can…

2 years ago

The Web Scraping Club

The most interesting GitHub Repositories about web scraping (2023)

This post is sponsored by Oxylabs, your premium proxy provider. Sponsorships help keep The Web Scraping Free and it’s a way to give back to the readers some value. In this case, for all The Web Scraping Club Readers, using the discount code WSC25 you can…

2 years ago · 2 likes · Pierluigi Vinciguerra

The Web Scraping Club

THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin

This article is sponsored by Serply, the solution to scrape search engine results easily. Web Scraping Club readers can save 25% on all SERP scraping plans by using the code TWSC25. Cloudflare anti-bot detection If you google “Cloudflare bypass”, you will find hundreds of articles and Github repositories explaining how to bypass Cloudflare (or sell a solution for doing it…

2 years ago · 2 likes · Pierluigi Vinciguerra

The Web Scraping Club

THE LAB #9: Scraping OpenSea NFT's data

The NFT hype cycle In the past week, a scandal that involves the famous influencer Logan Paul and his crypto project called “Criptozoo” exploded, thanks to the Cofeezilla investigations (you can see the full story here). Basically, it seems that this crypto game has never been delivered for multiple factors but people, trusting the public profile of Logan, put several million USD into it, hoping to have some return, a thing that never happened. It’s nothing new under the Crypto sun, Ponzi schemes promising impossible returns on investments are discovered every day, and surprisingly there’s always someone who got caught in the fishnet…

2 years ago · Pierluigi Vinciguerra

The Web Scraping Club

THE LAB #8: Using Bezier curves for human-like mouse movements

Here’s another post of “THE LAB”: in this series, we'll cover real-world use cases, with code and an explanation of the methodology used.The Web Scraping Club is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber…

2 years ago · 1 like · 1 comment · Pierluigi Vinciguerra

The Web Scraping Club

THE LAB #7: Scraping PerimeterX protected websites

Here’s another post of “THE LAB”: in this series, we'll cover real-world use cases, with code and an explanation of the methodology used.Thank you for reading The Web Scraping Club. This post is public so feel free to share it. Being a paying user gives…

2 years ago · 1 like · Pierluigi Vinciguerra

The Web Scraping Club

Selenium vs Playwright, a comparison

Hi, this is Pierluigi from The Web Scraping Club, a newsletter where you can find news, insights, and tutorials with real-world examples about web scraping. Being a paying user gives: Access to Paid Content, like the post series called “The LAB”, where we’ll go deep diving with code real-world cases …

2 years ago · Pierluigi Vinciguerra

The Web Scraping Club

THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting

Here’s another post of “THE LAB”: in this series, we'll cover real-world use cases, with code and an explanation of the methodology used.The Web Scraping Club is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber…

2 years ago · Pierluigi Vinciguerra

The Web Scraping Club

The Lab #5 - Scraping Airbnb.com using GraphQL

Here’s another post of “THE LAB”: in this series, we'll cover real-world use cases, with code and an explanation of the methodology used.The Web Scraping Club is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber…

3 years ago · Pierluigi Vinciguerra

The Web Scraping Club

The Kallax Index - Scraping Ikea websites

Hi, this is Pierluigi from The Web Scraping Club, a newsletter where you can find news, insights, and tutorials with real-world examples about web scraping. Being a paying user gives: Access to Paid Content, like the post series called “The LAB”, where we’ll go deep diving with code real-world cases …

3 years ago · 1 like · 1 comment · Pierluigi Vinciguerra

The Web Scraping Club

Create your first python scraper with Scrapy

Hi, this is Pierluigi from The Web Scraping Club, a newsletter where you can find news, insights, and tutorials with real-world examples about web scraping. Being a paying user gives: Access to Paid Content, like the post series called “The LAB”, where we’ll go deep diving with code real-world cases …

3 years ago · 1 like · Pierluigi Vinciguerra

The Web Scraping Club

THE LAB #3: Scraping Cloudflare protected websites

Here’s another post of “THE LAB”: in this series, we'll cover real-world use cases, with code and an explanation of the methodology used. In the future, this kind of content will be available only to paying subscribers. Being one of the first of the series, this one will be available for free until the 2nd of Oct 2022, then will be behind a paywall…

3 years ago · 4 likes · Pierluigi Vinciguerra

The Web Scraping Club

THE LAB #2: scraping data from a website with Datadome and xsrf tokens

Here’s another post of “THE LAB”: in this series, we'll cover real-world use cases, with code and an explanation of the methodology used. In the future, this kind of content will be available only to paying subscribers. Being one of the first of the series, this one will be available for free until 22th of Sept 2022, then will be behind a paywall…

3 years ago · 3 likes · Pierluigi Vinciguerra

The Web Scraping Club

THE LAB #1: Scraping data from an app

This is the first post of “THE LAB”: in this series, we'll cover real-world use cases, with code and an explanation of the methodology used. In the future, this kind of content will be available only to paying subscribers. Being the first of the series, this one will be available for free until Sunday 11th of Sept 2022, then will be behind a paywall…

3 years ago · 6 likes · 4 comments · Pierluigi Vinciguerra

The Web Scraping Club

Web Scraping in Python tutorials and resources

Tools, tutorials, and resources for web scraping in Python