Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Periodically write data to file while scraping #11

Open
ChrisMuir opened this issue Mar 15, 2018 · 0 comments
Open

Periodically write data to file while scraping #11

ChrisMuir opened this issue Mar 15, 2018 · 0 comments

Comments

@ChrisMuir
Copy link
Owner

This was suggested in issue #10 by addieg. Currently, all of the scraped data is written to file as the last step in the process. It makes sense to write scraped data to file at some regular interval while the script is running....should probably dump to file after each individual search term.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ChrisMuir