Great article, did you find a way to scrape companies though? I'm working on a similar project and the main thing I would want is search companies with their filters, I suppose we cant skip the auth wall for this? Linkedin does have explicit page for individual companies but I guess to add filters such as location, industry and keyword and search and get paginated results, I did find a way to create demo accounts and scrape it. But the account will get blocked in no time.
Just a heads-up: creating fake LinkedIn accounts for scraping purposes can lead to legal trouble, as I pointed out in the piece.
That said, from what I know, LinkedIn doesn’t really offer a public section where you can freely browse companies. Some alternative approaches you can explore are:
1. Automate company searches on Google, then scrape their public LinkedIn pages.
2. Use existing public LinkedIn datasets to find companies, and then scrape updated data from their individual public profiles.
Great article, did you find a way to scrape companies though? I'm working on a similar project and the main thing I would want is search companies with their filters, I suppose we cant skip the auth wall for this? Linkedin does have explicit page for individual companies but I guess to add filters such as location, industry and keyword and search and get paginated results, I did find a way to create demo accounts and scrape it. But the account will get blocked in no time.
Hey Navaneethan, I'm the author of the article!
Just a heads-up: creating fake LinkedIn accounts for scraping purposes can lead to legal trouble, as I pointed out in the piece.
That said, from what I know, LinkedIn doesn’t really offer a public section where you can freely browse companies. Some alternative approaches you can explore are:
1. Automate company searches on Google, then scrape their public LinkedIn pages.
2. Use existing public LinkedIn datasets to find companies, and then scrape updated data from their individual public profiles.
3. Browse curated lists like LinkedIn’s Top Companies by Country (https://www.linkedin.com/hubs/top-companies/)
Hope this helps!