Is web scraping becoming harder?

[Alex]

I'm also using headful playwright, along with uBlock which seems to block many tracking requests and is an indicator of a "normal" browser.

Here's how I start my chromium crawler and haven't had any issues until today:

```javascript

const uBlock = require("path").join(

__dirname,

"chromium-extensions/cjpalhdlnbpafiamejdnhcphjbkeiagm/1.41.8_4"

);

const browser = await chromium.launchPersistentContext('persistent-data-dir', {

headless: false,

devtools: false,

ignoreDefaultArgs: ["--enable-automation", "--hide-scrollbars"],

args: [

`--start-maximized`,

`--disable-extensions-except=${uBlock}`,

`--no-default-browser-check`,

],

userAgent:

"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36",

viewport: null,

});

context = browser;

page = await context.newPage({

viewport: null

});

await enableExtensions(page);

// Pass the Webdriver Test.

await page.addInitScript(() => {

const np = navigator.__proto__;

delete np.webdriver;

navigator.__proto__ = np;

});

```

[nick3499]

thanks for 'useful websites to test our settings'