Web Scraping 101 - A first tutorial

Pierluigi Vinciguerra
Feb 07, 2023
Share

In these pages, we’ll cover some basic aspects of web scraping, useful for someone who wants to taste the water before diving into more technical material.

  1. What does “web scraping” means?

    1. Is web scraping legal?

      1. Is it legal to scrape social networks like Linkedin or Facebook?

      2. Is it legal to scrape any public data I find?

    2. Web scraping typical use cases

  2. What do I need for web scraping?

    1. Python tools and framework for web scraping

      1. Scrapy

      2. Splash

      3. Selenium

      4. Playwright

      5. Undetected Chromedriver

    2. Proxies

      1. Different types of proxies

  3. Why do I get blocked while scraping?

    1. Most common bot detection techniques

      1. What is TLS fingerprinting?

      2. What is device fingerprinting?

      3. What is browser fingerprinting?

      4. What is Canvas fingerprinting?

      5. Why my IP is getting blocked?

    2. Anti-bot solutions

      1. Akamai

        1. What is Akamai anti-bot?

        2. How to bypass Akamai anti-bot?

      2. Cloudflare

        1. What is Cloudflare anti-bot?

        2. How to bypass Cloudflare anti-bot?

      3. Datadome

        1. What is Datadome anti-bot?

        2. How to bypass Datadome anti-bot?

      4. F5

        1. What is F5 anti-bot?

        2. How to bypass F5 anti-bot?

      5. Kasada

        1. What is Kadasa bot mitigation?

        2. How to bypass Kasada bot mitigation?

      6. PerimeterX

        1. What is PerimeterX anti-bot?

        2. How to bypass anti-bot?