Skip to content
#

scrapy

Here are 22 public repositories matching this topic...

项目整体分为scrapy-redis分布式爬虫爬取数据、基于ElasticSearch数据检索和前端界面展示三大模块。做此项目是为了熟悉scrapy-redis的基本流程,以及其背后的原理,同时熟悉ElasticSearch的使用。本项目可以作为一个基于ES存储的简单但是相对全面的全栈开发的Demo。项目中所采用的组件均在win10本地环境搭建(伪分布),旨在演示项目流程。你可以参考该项目,并将其扩展到多个主机上,实现分布式ES以及分布式Scrapy。

  • Updated Dec 8, 2022
  • CSS

Improve this page

Add a description, image, and links to the scrapy topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the scrapy topic, visit your repo's landing page and select "manage topics."

Learn more