This is a growing collection of threaded Python server bits, including custom HTTP server and WSGI classes, along with corresponding console entry points and some daemon scripts. The latest addition includes TFTP server support in both console and daemon formats.

These tools exist mainly to handle simple requests for local files in a small-ish engineering/development environment.

The original reason this version of the "project" exists was serving OTA firmware images to a small device over wifi, eg, an Android device or similar that requires an HTTP URL for firmware img/zip files. If that is what you need, then make sure the FW update files you want are in a directory in your virtual environment and run the serv command from that directory. The simple way to do that is:

• follow the steps below to create a virtual env (either venv or tox)
• connect your dev host to the same wifi network as the device
• copy your FW files into the source directory, then start the server

In another terminal, run your update command and provide a URL like this:

http://<dev_host_wifi_IP>:PORT/fw_update.img

where PORT is the port used below and fw_update.img is the name of your OTA update file.

Pyserv contains modules with some backported features and a fix for broken OTA clients. It provides multiple console commands for different protocols, and two daemon wrappers for http and tftp.

• console commands with simple arguments to run, well, from the console, or for running via Procfile with something like Honcho
• daemon scripts to run in the background for workflows that need a simple HTTP/WSGI server or TFTP server

• HTTP - the serv console command
• WSGI - the wsgi console command
• TFTP - the tftpd console command

The above standard Python console entry points all have these minimal/default "features" with no arguments:

• the document/server root is always the current directory
  • HTTP: default port is 8080 and the server listens on all interfaces
  • WSGI: default port is 8000, default app is builtin demo app, and the server listens on localhost
  • TFTP: default port is 9069 and the server listens on localhost
• the only allowed args are either port, or port and interface (or app_name and port for WSGI)

The httpdaemon and tftpdaemon commands are stand-alone Python daemon scripts with the same core server code, as well as a default user configuration adjustable via environment variables, and the following "extra" features:

• allowed command-line args are start | stop | restart | status
• default port is 8080 or 9069 and listen interface is 127.0.0.1
• default XDG user paths are set for pid and log files
• environment values are checked first; if not set, fallback to defaults
• logging using daemon package logger config

Sample environment display with tox overrides, ie, inside a Tox venv:

Python version: 3.11.4 (main, Jul  5 2023, 16:15:04) [GCC 12.3.1 20230526]
-------------------------------------------------------------------------------
pyserv 1.4.1.dev1

Pyserv default settings for server and daemon modes.

Default user vars:
  log_dir: /home/user/.local/state/pyserv/log
  pid_dir: /run/user/1001/pyserv
  work_dir: /home/user/src/pyserv

Current environment values:
  DEBUG: 1
  PORT: 8000
  IFACE: 127.0.0.1
  LPNAME: httpd
  LOG: /home/user/src/pyserv/.tox/dev/log/httpd.log
  PID: /home/user/src/pyserv/.tox/dev/tmp/httpd.pid
  DOCROOT: /home/user/src/pyserv
  SOCK_TIMEOUT: 5
-------------------------------------------------------------------------------

Use any of the variables under "Current environment values" to set your own custom environment.

Once installed in a virtual environment, check the help output:

$ httpdaemon -h
usage: httpdaemon [-h] [--version] {start,stop,restart,status}

Threaded HTTP server daemon

positional arguments:
  {start,stop,restart,status}

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit

One small wrinkle

• the daemon scripts are "traditional" forking daemons and thus will not work on Windows, however, the console command variants should Just Work (if not, please file an issue).

New

• experimental tftp server daemon based on tftpy
• even more experimental async tftp server daemon based on py3tftp
• run tox -e tftp to create a virtual env and view defaults
• run tox -e tftpd to create a virtual env with capabilities for low ports, eg, port 69
• ENV value SOCK_TIMEOUT is specific to tftp client/server connections
• script args and most ENV values are otherwise the same as httpdaemon

Run a simple test of the async daemon with tox:

$ LPNAME=atftpd tox -e tftpd
tftpd: install_deps> python -I -m pip install logging_tree 'pip>=23.1' 'setuptools_scm[toml]' .
tftpd: commands_pre[0]> bash -c 'dd if=/dev/zero of=$DOCROOT/$TST_FILE bs=1M count=40'
40 0 records in
40 0 records out
41943040 bytes (42 MB, 40 MiB) copied, 0.0127168 s, 3.3 GB/s
tftpd: commands_pre[1]> bash -c 'sudo setcap cap_net_bind_service ep /home/nerdboy/src/pyserv/.tox/tftpd/bin/python'
tftpd: commands_pre[2]> bash -c 'sudo setcap cap_net_bind_service ep /home/nerdboy/src/pyserv/.tox/tftpd/bin/python3'
tftpd: commands[0]> python -c 'from pyserv.settings import show_uservars; show_uservars()'
Python version: 3.12.7 (main, Oct 19 2024, 22:38:25) [GCC 14.2.1 20240921]
-------------------------------------------------------------------------------
pyserv 1.6.2.dev8 g684c689

Pyserv default settings for server and daemon modes.

Default user vars:
  log_dir: /home/nerdboy/.local/state/pyserv/log
  pid_dir: /run/user/1000/pyserv
  work_dir: /home/nerdboy/src/pyserv

Current environment values:
  DEBUG: 0
  PORT: 69
  IFACE: 0.0.0.0
  LPNAME: atftpd
  LOG: /home/nerdboy/src/pyserv/.tox/tftpd/log/atftpd.log
  PID: /home/nerdboy/src/pyserv/.tox/tftpd/tmp/atftpd.pid
  DOCROOT: tests/data
  SOCK_TIMEOUT: 5
-------------------------------------------------------------------------------
tftpd: commands[1]> atftpdaemon -h
usage: atftpdaemon [-h] [--version] [--host HOST] [-p PORT]
                   [--ack-timeout TIMEOUT] [--conn-timeout CONN_TIMEOUT] [-v]
                   [-q]
                   {start,stop,restart,status}

Async TFTP server daemon

positional arguments:
  {start,stop,restart,status}

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --host HOST           IP of the interface the server will listen on.
                        Default: 0.0.0.0 (default: )
  -p PORT, --port PORT  Port the server will listen on. Default: 9069. TFTP
                        standard-compliant port: 69 - requires additional
                        privileges. (default: 9069)
  --ack-timeout TIMEOUT
                        Timeout for each ACK of the lock-step. Default: 0.5.
                        (default: 0.5)
  --conn-timeout CONN_TIMEOUT
                        Timeout before the server gives up on a transfer and
                        closes the connection. Default: 3. (default: 5.0)
  -v, --verbose         Enable debug-level logging. (default: False)
  -q, --quiet           Inhibit extra console output. (default: False)
tftpd: commands[2]> atftpdaemon start
LOG: /home/nerdboy/src/pyserv/.tox/tftpd/log/atftpd.log
PID: /home/nerdboy/src/pyserv/.tox/tftpd/tmp/atftpd.pid
DOCROOT: tests/data
tftpd: commands[3]> bash -c 'sleep 2'
tftpd: commands[4]> curl --tftp-blksize 8192 --output tests/testbin.swu tftp://0.0.0.0:69/testbin.swu
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 40.0M  100 40.0M    0     0   275M      0 --:--:-- --:--:-- --:--:--  275M
100 40.0M  100 40.0M    0     0   275M      0 --:--:-- --:--:-- --:--:--  275M
tftpd: commands[5]> bash -c 'sleep 1'
tftpd: commands[6]> tail -n 5 /home/nerdboy/src/pyserv/.tox/tftpd/log/atftpd.log
2024-12-24 01:48:12 UTC INFO atftpd.daemonize(149) Started
2024-12-24 01:48:12 UTC INFO atftpd.connection_made(393) Listening...
2024-12-24 01:48:14 UTC INFO atftpd.__init__(273) Initiating RRQProtocol with ('127.0.0.1', 56554)
2024-12-24 01:48:14 UTC INFO atftpd.connection_lost(123) Connection to 127.0.0.1:56554 terminated
tftpd: commands[7]> cmp tests/data/testbin.swu tests/testbin.swu
tftpd: commands[8]> ls -l tests/data/testbin.swu tests/testbin.swu
-rw-r--r-- 1 nerdboy nerdboy 41943040 Dec 23 17:48 tests/data/testbin.swu
-rw-r--r-- 1 nerdboy nerdboy 41943040 Dec 23 17:48 tests/testbin.swu
tftpd: commands[9]> bash -c 'rm -f tests/data/testbin.swu tests/testbin.swu'
tftpd: commands_post[0]> atftpdaemon stop
LOG: /home/nerdboy/src/pyserv/.tox/tftpd/log/atftpd.log
PID: /home/nerdboy/src/pyserv/.tox/tftpd/tmp/atftpd.pid
DOCROOT: tests/data
  tftpd: OK (39.19=setup[35.53] cmd[0.02,0.01,0.01,0.07,0.09,0.10,2.00,0.15,1.00,0.01,0.02,0.00,0.01,0.18] seconds)
  congratulations :) (39.24 seconds)

This refactored fork of pyserv is not published on PyPI, thus use one of the following commands to install the latest pyserv in a Python virtual environment on any platform.

From source:

$ python3 -m venv env
$ source env/bin/activate
$ pip install git https://github.com/sarnold/pyserv.git
$ serv 8000      # optionally add interface, eg, 10.0.0.2

The output should be:

INFO:root:Starting HTTP SERVER at PORT :8000

The alternative to python venv is the Tox test driver. If you have it installed already, clone this repository and try the following commands from the pyserv source directory.

To install in dev mode:

$ tox -e dev

To run tests using default system Python:

$ tox -e py

To run pylint:

$ tox -e lint

To install the latest release, eg with your own tox.ini file in another project, use something like this:

$ pip install https://github.com/sarnold/pyserv/releases/download/1.2.4/pyserv-1.2.4-py3-none-any.whl

If you have a requirements.txt file, you can add something like this:

pyserv @ https://github.com/sarnold/pyserv/releases/download/1.2.4/pyserv-1.2.4.tar.gz

Note the newest pip versions may no longer work using -f with just the GH "releases" path to get the latest release from Github.

In the repo, use the tox env and start the server:

$ tox -e py
$ source .tox/py/bin/activate
(py) $ tftpd
INFO:tftpy.TftpServer:Server requested on ip 127.0.0.1, port 9069
INFO:tftpy.TftpServer:Starting receive loop...

Open a new terminal and try out downloading a file with curl using default options; note this will send the file directly to stdout:

$ curl tftp://127.0.0.1:9069/requirements.txt
# daemon requirements, useful for tox/pip
daemonizer @ git https://github.com/sarnold/[email protected]#5f6bc3c80a90344b2c8e4cc24ed0b8c098a7af50; platform_system!="Windows"
appdirs
tftpy

On the server side, ie, inside your virtual environment, you should see:

INFO:tftpy.TftpStates:Setting tidport to 51009
INFO:tftpy.TftpStates:Dropping unsupported option 'timeout'
INFO:tftpy.TftpStates:requested file is in the server root - good
INFO:tftpy.TftpStates:Opening file /home/user/src/pyserv/requirements.txt for reading
INFO:tftpy.TftpServer:Currently handling these sessions:
INFO:tftpy.TftpServer:    127.0.0.1:51009 <tftpy.TftpStates.TftpStateExpectACK object at 0xffff87d5d1d0>
INFO:tftpy.TftpStates:Reached EOF on file requirements.txt
INFO:tftpy.TftpStates:Received ACK to final DAT, we're done.
INFO:tftpy.TftpServer:Successful transfer.
INFO:tftpy.TftpServer:
INFO:tftpy.TftpServer:Session 127.0.0.1:51009 complete
INFO:tftpy.TftpServer:Transferred 257 bytes in 0.00 seconds
INFO:tftpy.TftpServer:Average rate: 1243.74 kbps
INFO:tftpy.TftpServer:0.00 bytes in resent data
INFO:tftpy.TftpServer:0 duplicate packets

If no port is provided the server attempts to run on port 9069.

If the given port (or the default port 9069) is already in use, you will need to pass a different port number, eg, 9169.

For larger/binary files, use -O to save the file in the current directory, and for better performance with large files, use curl's --tftp-blksize arg and set a larger size, eg, 8192.

In the repo, use the tox env and start the server:

$ tox -e py
$ source .tox/py/bin/activate
(py) $ serv
INFO:root:Starting HTTP SERVER at :8080

Open a new terminal and try out sending a GET request:

$ python
>>> import requests
>>> URL = 'http://0.0.0.0:8080'
>>> r = requests.get(URL)
>>> print(r.text)
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

On the server side, ie, inside your virtual environment, you should see:

INFO:root:Starting HTTP SERVER at :8080
INFO:root:Path in: /
INFO:root:Path out: /
INFO:root:Headers:
INFO:root:  Host: 0.0.0.0:8080
INFO:root:  User-Agent: python-requests/2.25.1
INFO:root:  Accept-Encoding: gzip, deflate
INFO:root:  Accept: */*
INFO:root:  Connection: keep-alive
INFO:root:127.0.0.1 - - [13/Jul/2022 20:52:22] "GET / HTTP/1.1" 200 -

If no port is provided the server attempts to run on port 8080.

If the given port (or the default port 8080) is already in use, you will need to pass a different port number, eg, 8088.

Motivation:

Small device firmware with non-compliant HTTP client implementations.

Original project from gist: https://pypi.org/project/pyserv/

Original gist: https://gist.github.com/mdonkers/63e115cc0c79b4f6b8b3a6b797e485c7

This repo is pre-commit enabled for python/rst source and file-type linting. The checks run automatically on commit and will fail the commit (if not clean) and perform simple file corrections. For example, if the mypy check fails on commit, you must first fix any fatal errors for the commit to succeed. That said, pre-commit does nothing if you don't install it first (both the program itself and the hooks in your local repository copy).

You will need to install pre-commit before contributing any changes; installing it using your system's package manager is recommended, otherwise install with pip into your usual virtual environment using something like:

$ sudo emerge pre-commit  --or--
$ pip install pre-commit

then install it into the repo you just cloned:

$ git clone https://github.com/sarnold/pyserv
$ cd pyserv/
$ pre-commit install

It's usually a good idea to update the hooks to the latest version:

$ pre-commit autoupdate

Most (but not all) of the pre-commit checks will make corrections for you, however, some will only report errors, so these you will need to correct manually.

Automatic-fix checks include ffffff, isort, autoflake, and miscellaneous file fixers. If any of these fail, you can review the changes with git diff and just add them to your commit and continue.

If any of the mypy, bandit, or rst source checks fail, you will get a report, and you must fix any errors before you can continue adding/committing.

To see a "replay" of any rst check errors, run:

$ pre-commit run rst-backticks -a
$ pre-commit run rst-directive-colons -a
$ pre-commit run rst-inline-touching-normal -a

To run all pre-commit checks manually, try:

$ pre-commit run -a