a lightweight log store with time-based rotation
The main purpose of this project is to connect nginx and goaccess when both services are running as docker containers. It provides a nginx-compatible syslog endpoint to collect access log messages from nginx containers, handles log rotation and has some tweaks to work with goaccess.
A minimal configuration would look like this:
---
rotate_every_seconds: 120
flush_every_seconds: 2
storage:
path: ./test_storage
size: 5
endpoints:
- type: UdpEndpoint
port: 1024
The logrotor docker container expects a config file under /srv/config.yml
. To run with docker:
docker run \
--volume $(pwd)/config.yml:/srv/config.yml \
--publish 1024:1024/udp --name test_logrotor reifenbt/logrotor
Run without docker (Python 3.6 required):
pip install git https://github.com/tom-mi/logrotor
logrotor --config config.yml
Send a message:
echo -n "Hello World!" | nc -u -c localhost 1024
Read the logs (when running with docker):
docker exec test_logrotor cat ./test_storage/current
docker exec test_logrotor /bin/sh -c 'cat ./test_storage/data/*'
Read the logs (when running without docker):
cat ./test_storage/current
cat ./test_storage/data/*
A simple example is contained in the repository.
git clone https://github.com/tom-mi/logrotor
cd logrotor
cd examples/nginx_via_syslog
docker-compose up
The nginx webserver is reachable under http://localhost:8080/. The goaccess reports are served under http://localhost:8080/report/.
Currently, UDP and syslog (UDP, according to RFC 3164) endpoints are supported. See the following example for all currently available configuration options:
---
rotate_every_seconds: 30 # required
flush_every_seconds: 2 # required
storage:
path: ./test_storage # required
size: 5 # required
rotate_delay_seconds: 0 # delay between deleting and re-creating a file during rotation
# (used with goaccess)
endpoints:
- type: UdpEndpoint # required
bind: 0.0.0.0
port: 1024 # required
format: '{host} {message}' # available variables: host, port, message
splitlines: False # split multiline messages and apply format to each line
- type: SyslogUdpEndpoint # required
bind: 0.0.0.0
port: 514
format: '{timestamp:%b %e %H:%M:%S} {hostname} {message}'
# available variables: src_host, src_port, timestamp, level, facility,
# hostname, tag, pid, message