The Web Scraping Club

Home
Consulting
The Lab
Special offers for our readers
Archive
Leaderboard
About

The Lab

THE LAB #61: Evaluating your proxy provider
Measuring programmatically the quality of the IPs offered by proxy providers
 • 
Pierluigi Vinciguerra
THE LAB #60: Writing scrapers with LLMs
Comparing LLama3.1, GPT4 and Mistral in creating scrapers
 • 
Pierluigi Vinciguerra
The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2
Rooting a virtual mobile Android Device to install Frida and discover API endpoints under the hood of apps
 • 
Pierluigi Vinciguerra
The Lab #58: Intercepting traffic from an App - part 1
Discover API endpoints called by an App to scrape its data
 • 
Pierluigi Vinciguerra
4
The Lab #57: Improving your Playwright scraper and avoid CDP detection
How to use the latest advancements to avoid CDP detection in your Playwright scrapers
 • 
Pierluigi Vinciguerra
3
The Lab #56: Bypassing PerimeterX 3
Testing the latest PerimeterX version by scraping Crunchbase public data
 • 
Pierluigi Vinciguerra
The Lab #55: Checking your browser fingerprint
Understanding your scrapers' browser fingerprint reliability with online tests.
 • 
Pierluigi Vinciguerra
3
The Lab #54: Scraping from Algolia APIs
Why internal APIs are always the best choice for scraping a website
 • 
Pierluigi Vinciguerra
2
The Lab #53: Bypassing AWS WAF
Scraping websites protected by AWS WAF using an hybrid approach
 • 
Pierluigi Vinciguerra
The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1
Are LLMs the Holy Graal for web scraping?
 • 
Pierluigi Vinciguerra
The Lab #51: APIs with Bearer Token
Scraping data from API endpoints requiring Bearer Token
 • 
Pierluigi Vinciguerra
Celebrating the 50th article of The Lab series
A brief review of the first 50 episodes of The Lab series
 • 
Pierluigi Vinciguerra
© 2024 Pierluigi
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great culture