- NewsBlur is a personal news reader bringing people together to talk about the world. A new sound of an old instrument.
- www.newsblur.com.
- Created by Samuel Clay.
- Twitter: @samuelclay and @newsblur.
- Shows the original site (you have to see it to believe it).
- Hides stories you don't want to read based on tags, keywords, authors, etc.
- Highlights stories you want to read, based on the same criteria.
- Python 3.7 : The language of choice.
- Django: Web framework written in Python, used to serve all pages.
- Celery & RabbitMQ: Asynchronous queueing server, used to fetch and parse RSS feeds.
- MongoDB, Pymongo, & Mongoengine: Non-relational database, used to store stories, read stories, feed/page fetch histories, and proxied sites.
- PostgreSQL: Relational database, used to store feeds, subscriptions, and user accounts.
- Redis: Programmer's database, used to assemble stories for the river, store story ids, manage feed fetching schedules, and the minuscule bit of caching that NewsBlur uses.
- Elasticsearch: Search database, use for searching stories. Optional.
- jQuery: Cross-browser compliant JavaScript code. IE works without effort.
- Underscore.js: Functional programming for JavaScript. Indispensable.
- Backbone.js: Framework for the web app. Also indispensable.
- Miscellaneous jQuery Plugins: Everything from resizable layouts, to progress bars, sortables, date handling, colors, corners, JSON, animations. See the complete list.
* Docker
* Docker-compose
-
Clone this repo
-
Run
make nb
to build all of the NewsBlur containers. This will set up all necessary databases, front-end django apps, celery tasks, node apps, flask database monitor and metrics, nginx, and a haproxy load balancer. -
Navigate to:
https://localhost
Note: You will be warned that you are using a self signed certificate. In order to get around this warning you must type "thisisunsafe" as per this blog post.
-
Run the custom domain script
bash ./utils/custom_domain.sh <domain name>
This script will do the following:
- Change
NEWSBLUR_URL
andSESSION_COOKIE_DOMAIN
innewsblur_web/docker_local_settings.py
- Change the domain in
config/fixtures/bootstrap.json
- Change
You can also change domains: bash ./utils/custom_domain.sh <old domain> <new domain>
-
If you're using a custom subdomain, you'll also want to add it to
ALLOWED_SUBDOMAINS
inapps/reader/views.py
-
A way to make sure you updated all the correct places:
- Go to the website address in your browser
- Open developer tools and look at the network tab
- Try to login
- Look again at the developer tools, there should be a POST call to /login
- Observe the Response headers for that call
- The value of the "set-cookie" header should contain a "Domain=" string
If the string after
Domain=
is not the domain you are using to access the website, then your configuration still needs your custom domain.You can also confirm that there is a domain name mismatch in the database by running
make shell
& typingSite.objects.all()[0]
to show the domain that NewsBlur is expecting.
To make docker-compose work with your database, upgrade your local database to the docker-compose version and then volumize the database data path by changing the ./docker/volumes/
part of the volume directive in the service to point to your local database's data directory.
To make docker-compose work with an older database version, change the image version for the database service in the docker-compose file.
-
Making Changes:
- To apply changes to the Python or JavaScript code, use the
make
command. - To apply changes to the docker-compose.yml file, use the
make rebuild
command. - To apply changes to the docker/haproxy/haproxy.conf file, node packages, or any new database migrations you will need to use the
make nb
command.
- To apply changes to the Python or JavaScript code, use the
-
Adding Python packages: Currently, the docker-compose.yml file uses the newsblur/newsblur_python3 image. It is built using the Dockerfile found in
docker/newsblur_base_image.Dockerfile
. Because of how the docker image is set up, you will need to create your own image and direct your docker-compose.yml file to use it. Please follow the following steps to do so.- Add your new site-packages to config/requirements.txt.
- Add the following lines of code to your docker-compose.yml file to replace anywhere where it says
image: newsblur/newsblur_python3
build: context: . dockerfile: docker/newsblur_base_image.Dockerfile
- Run the
make nb
command to rebuild your docker-compose containers
-
Debugging Python
- To debug your code, drop
import pdb; pdb.set_trace()
into the Python code where you would like to start debugging and runmake
and thenmake debug
.
- To debug your code, drop
-
Using Django shell within Docker
- Make sure your docker containers are up and run
make shell
to open the Django shell within the newsblur_web container.
- Make sure your docker containers are up and run
NewsBlur comes complete with a test suite that tests the functionality of the rss_feeds, reader, and feed importer. To run the test suite:
`make test`
Performance tests use the locust performance testing tool. To run performance tests via CLI, use
make perf-cli users=1 rate=1 host=https://localhost
. Feel free to change the users, rate, and host
variables in the command to meet you needs.
You can also run locust performance tests using a UI by running make perf-ui
and then navigating to
http://127.0.0.1:8089. This allows you to chart and export your performance data.
To run locust using docker, just run make perf-docker
and navigate to http://127.0.0.1:8089
- Created by Samuel Clay.
- Email address: [email protected]
- @samuelclay on Twitter.
NewsBlur is licensed under the MIT License. (See LICENSE)