A bot to scrape jobs data from linkedin and twitter. A scrapy crawler is used to fetch the links and data is extracted from the tags using BeautifulSoup.
This repository can be downloaded using
git clone https://github.com/gautham-apa/LinkedinJobScraper.git
The search keywords required can be added in the searchKeywords
list located in the /linkedinbot/linkedinbot/spiders/linkedinSpider.py
file.
To run the bot use the following command
scrapy crawl linkedinSpider
The scraped data would be transformed to a json and written at location /linkedinbot/linkedinbot/spiders/data
with current date and time.
This bot has been built for educational purposes only and not intended for commercial deployment. Please use it wisely at your own risk.