5 Comments
May 21·edited May 21Liked by Pierluigi Vinciguerra

https://www.hermes.com/kr/ko/

https://www.hermes.com/us/en/faq/

Can you confirm if it also works on Hermes? Use playwright.

Expand full comment
author

Uncluckily Hermes is one of the hardest websites I've encountered and it has several layers of protection. Until some years ago this solution worked, but then also the API endpoint has been protected by other anti-bot solutions

Expand full comment

I'm studying web scraping and looking for a Python tool. I'm currently using Playwright. Have you had any success scraping Hermes recently? It's been really difficult for me.

Expand full comment
May 19Liked by Pierluigi Vinciguerra

Hi Pierluigi how are you hope youre fine

As i am a data engineer i am confronted to these bot managers i am specialised in retailers websites i was confronted months ago to akamai in the website of american retailer Kroger , the token expires after like 3 requests and the tokens created by headless gives partial results , and i found that this is intentional i made some tests to deobusfucate the challenge to understand it but no way it will take an eternity, please if you can take a look to this retailer website if you need any informations from my previous experience with feel free to contact me at nab3945@live.fr

Expand full comment
author

Thanks for sharing your experience, I’ll have a look! Thanks

Expand full comment