Package: filespooler (1.2.3-1 and others)

Sequential, distributed, POSIX-style job queue processing

Filespooler is a Unix-style tool that facilitates local or remote command execution, complete with stdin capture, with easy integration with various tools. Here's a brief Filespooler feature list:

 - It can easily use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB
  drives, CDs, etc. as transport.

  - Translation: you can use basically anything that is a filesystem as a
    transport

 - It can use arbitrary decoder command pipelines (eg, zcat, stdcat, gpg, age,
  etc) to pre-process stored packets.

 - It can send and receive packets by pipes.

 - Its storage format is simple on-disk files with locking.

 - It supports one-to-one and one-to-many configurations.

 - Locking is unnecessary when writing new jobs to the queue, and many arbitrary
  tools (eg, Syncthing, Dropbox, etc) can safely write directly to the queue
  without any assistance.

 - Queue processing is strictly ordered based on the order on the creation
  machine, even if job files are delivered out of order to the destination.

 - stdin can be piped into the job creation tool, and piped to a later executor
  at process time on a remote machine.

 - The file format is lightweight; less than 100 bytes overhead unless large
  extra parameters are given.

 - The queue format is lightweight; having 1000 different queues on a Raspberry
  Pi would be easy.

 - Processing is stream-based throughout; arbitrarily-large packets are fine and
  sizes in the TB range are no problem.

 - The Filespooler command, fspl, is extremely lightweight, consuming less than
  10MB of RAM on x86_64.

 - Filespooler has extensive documentation.

Filespooler consists of a command-line tool (fspl) for interacting with queues. It also consists of a Rust library that is used by fspl. main.rs for fspl is just a few lines long.

This page is also available in the following languages (How to set the default document language):