Procdog (as in "process watchdog"... get it?) is a simple command-line tool to start, stop, and check the health of processes. It works with any kind of process you can invoke from the command line (be it native, Java, Node, Python, or anything) on MacOS or Linux.
Why would you want another tool for this?
- For basic interactive situations, you can just run a process in a terminal
or use job management in your shell (
jobs
,kill
, etc.). - But you rapidly realize this won't work well once you have longer-lived processes
or are
ssh
ing to remote servers. Then you could usenohup
and manually checking withps
, or using screen. But these don't script easily. - You also might want to script starting and stopping.
With a little effort you might write a custom Bash script (writing a PID file,
using
pgrep
andpkill
, etc.), but it's a hassle and gets messy quickly. - Of course, you can just "do it right." Traditionally, in the Unix world, the way to control services are System V service scripts, start-stop-daemon, Upstart, or systemd. Using these is essential for production deployment, but they tend to be a bit arcane and highly OS-dependent, so aren't as convenient for casual use, and you can't easily develop and test on both MacOS and Linux (as many of us try to do).
Procdog is an alternative for developers that attempts to be easy to install,
simple and obvious to use, and cross-platform. Processes are independent of the
shell used to invoke them (i.e., detatched as with nohup
) and you can also check
status or kill them at any time (as with a Unix service).
Probably the most simlar tool to Procdog is Supervisor. Although it has similar goals, Supervisor is a more complex, production-oriented tool with a long-lived, centrailized XML-RPC server that monitors processes. Procdog is intended to be a single, simple, easy-to-use command that needn't be set up and managed itself or run as root. Each use of Procdog is independent and monitors just one process. It does not require a configuration file unless you want to save typing. You can even check the Procdog script into your own project so developers can immediately use it locally, since it has no dependencies (besides Python 2.7).
You'll find it useful if you have servers, databases, or other processes you want to manage on your personal machine when developing, in build systems and test harnesses, test deployments, etc. Currently, it doesn't have restart logic, log file rotation, or some other features you may want for a production environments; for this consider Supervisor, Upstart, systemd, & co. Procdog is also way less mature than these alternatives.
No dependencies except Python 2.7 . It's easiest to install with pip:
sudo pip install procdog
Or, since it's just one file, you can copy the single
procdog
script somewhere (perhaps within your own project) and make it executable.
Now you can start and monitor any process (here let's pick "sleep" -- not all that useful, but you already have it):
$ procdog start myprocess --command "sleep 100"
running, pid=14969
$ procdog status myprocess
running, pid=14969
$ procdog stop myprocess
stopped
$
Note you have to give your process an arbitrary name (myprocess
here), like a Unix
service name, so you can refer to it. Once the process is done, the monitor daemon
also exits.
Run procdog -h
for help on all options.
Now, with a real server we'd like to know if it's actually up and doing something,
like listening on a port. Procdog supports an arbitrary command to test health
(e.g. running curl
to see if it returns a result) and can wait until a server
is running and tell you.
$ procdog start backend --command="java -classpath my-backend.jar com.example.BackendServer server config.yml" \
--health-command="curl http://localhost:8080/ping" \
--health-count=10 --health-delay=2 \
--dir=$HOME/backend \
--stdout=backend.log --stderr=backend.log --append \
--ensure-healthy --strict
running, health=0, pid=15240
$ procdog status backend
running, health=0, pid=15240
$ procdog stop backend --strict
stopped
$ procdog stop backend --strict
procdog: error: process 'backend' is not running
$
Some notes on this:
- We can specify where to write stdout and stderr. They can be the same or different.
Existing log files are appended to if you use
--append
. - The health command simply calls a shell command to see if the server is healthy.
The return code of the health check command must be
0
for the server to be considered healthy. In this case, we're callincurl
on a known health-check URL, which will have return code 0 on an HTTP 200 - The
--ensure-healthy
option means the command will block until the process is healthy, or until the daemon gives up and kills the process (if necessary). In this example, it will try 10 times, sleeping 2 seconds each time, before giving up. - The
--dir
option means process will run from that directory. - We ask the client to be
--strict
, so that it returns non-zero status code when we try to start a process that's already running or or stop one that is already stopped.
It's possible to avoid typing by putting most options in a configuration file:
# Procdog config file. Each section is a process name.
[backend]
command=java -classpath my-backend.jar com.example.BackendServer server config.yml
health_command=curl http://localhost:8080/ping
health_count=10
health_delay=2.
dir=$HOME/backend
stdout=backend.log
stderr=backend.log
append=False
ensure_healthy=True
strict=True
You have any number of sections, one section per process. Procdog reads options
from ~/.procdog.cfg
or procdog.cfg
(in the same directory the procdog
script resides). Any options given on the command line override those in the
configuration file. Note that environment variables (like $HOME
) are
allowed and expanded. Once you have the above section in your config file,
you can run:
$ procdog start backend
running, health=0, pid=15396
$ procdog stop backend
stopped
$
Procdog starts a small daemon that popen()
s and monitors the process.
The daemon listens and accepts commands on a local
Unix domain socket,
making it possible to check the process is running or terminate it, and to do simple
health checks. You can see these sockets at /var/tmp/procdog.*.sock
(where the * is the
id of the process). For simplicity, there is a single Procdog daemon for each monitored
process, so each process is handled compeltely separately.
We use Unix domain sockets so that we don't have the headaches of pid files or choosing and binding to TCP ports. They're also available on most platforms.
Daemon logs are sent to /var/tmp/procdog.*.log
. Usually you'll want to redirect your
process stdout and stderr using the --stdout
and --stderr
options.
Procdog is quite new so probably not stable. Bug reports and contributions are welcome!
All basic features are covered with a simple Bash-based regression test.
To run it, clone this repo, invoke tests/run.sh
then follow the directions to diff
the output using git
.