bitpocket is a small but smart script that does 2-way directory synchronization. It uses rsync to do efficient data transfer and tracks local file creation/removal to avoid known rsync problem when doing 2-way syncing with deletion.

bitpocket can use any server which you have ssh access to for its central storage. If you have gigabytes of free disk space on your hosting server you can finally make use of it.

Clone repository and symlink bitpocket bin to sth in your $PATH:

$ git clone git://github.com/ku1ik/bitpocket.git
$ ln -s `pwd`/bitpocket/bin/bitpocket ~/bin/bitpocket

Or download script and place it in a directory in your $PATH:

$ curl -sL https://raw.github.com/ku1ik/bitpocket/master/bin/bitpocket > ~/bin/bitpocket
$ chmod  x ~/bin/bitpocket

Create an empty directory on some host that will be the master copy of your files:

$ ssh [email protected]
$ mkdir ~/BitPocketMaster

On each machine you want to synchronize initialize an empty directory as your bitpocket:

$ mkdir ~/BitPocket
$ cd ~/BitPocket
$ bitpocket init [email protected] ~/BitPocketMaster

After installation, you use the bitpocket command for synchronization and other tasks. Running bitpocket help will display the following message.

usage:  bitpocket { init [<REMOTE_HOST>] <REMOTE_PATH>
                  | sync | help | pack | log | cron | list }

Available commands:
   sync    Run the sync process. If no command is specified, sync is run by
           default.
   init    Initialize a new bitpocket folder. Requires path and optional
           remote host params. Remote path must already exist.
   pack    Pack any existing (automatic) backups into a git repository.
   cron    Run sync optimized for cron, logging output to file instead of
           stdout.
   log     Display the log generated by the cron command
   list    List all files in the sync set (honoring include/exclude/filter
           config).
   help    Show this message.

Options:
   -f, --force      Clean up stale lock files automatically
   -p, --pretend    Don't really perform the sync or update the current
                    state. Instead, show what would be synchronized.

Note: All commands (apart from help), must be run in the root of a
      new or existing bitpocket directory structure.

To synchronize your local slave with master just run bitpocket sync inside your bitpocket directory:

$ cd ~/BitPocket
$ bitpocket sync

Ensure that you run bitpocket at least once immediately after creating a new slave and before adding new files to the slave directory. If there are files in the master they will be pulled into the slave. You may then move files into your slave directory and they will be detected as added.

bitpocket does not include a full-fledged versioning system at the moment, but it does automatically create a local backup of any files before overwriting or deleting with changes from the BitPocketMaster. These backups are placed into a timestamped directory (one directory per sync). The path to this directory is:

.bitpocket/backups/YYYY-MM-DD.hhmmss

As these files accumulate, you may want to remove them, but you can also run bitpocket pack to combine all the backup files into a git repository contained within the .bitpocket directory:

$ cd ~/BitPocket
$ bitpocket pack

This requires an installation of git, but allows all the space saving advantages of git when making repeated changes to the same files.

There is a discussion about potential directions for versioning direction here: github.com/ku1ik/bitpocket/issues/15

If bitpocket is run with the cron parameter ( bitpocket cron ), it will perform a sync, but instead of showing the progress on stdout, it will redirect all output to a log file:

$ cd ~/BitPocket
$ bitpocket cron

As the name of this parameter implies, this is mainly useful when running bitpocket through the cron command. (See "Automatic sync with cron" for more information about how to configure this).

When running bitpocket in cron with bitpocket cron it will append its output to .bitpocket/log file. You can review the tail end of an existing log file, or watch live log as it is generated, with following command:

$ cd ~/BitPocket
$ bitpocket log

You may want to know which files will be synchronized before actually performing the syncronization. You can verify which files are in the synchronization set by running bitpocket list:

$ cd ~/BitPocket
$ bitpocket list

Note that this does not list changed files, it only lists all the local files that bitpocket will look at in determining which files to sync. Also, note that if there are new files in the master that will be added on a sync, they will not be included here. This command is only intended to verify which files are in the synchronization set. (See "Configuring file exclusion and inclusion" for information about how to control which files are in the synchronization set).

Add following line to your crontab to synchronize every 5 minutes:

*/5 * * * * cd ~/BitPocket && nice ~/bin/bitpocket cron

Note that cron usually has very limited environment and your ssh keys with passphrases won't work in cron jobs as ssh-agents/keyrings don't work there. Thus it's preferable to generate passphrase-less ssh key for bitpocket authentication:

$ cd ~/BitPocket
$ ssh-keygen -t rsa -C bitpocket-`hostname` -N '' -f .bitpocket/id_rsa
$ ssh-copy-id -i .bitpocket/id_rsa [email protected]

and uncomment line with RSYNC_SSH in .bitpocket/config file.

If you want some files to be ignored by bitpocket you can create .bitpocket/exclude file and list the paths there:

*.avi
jola
/misio.txt

*.avi and jola will be matched anywhere in path, misio.txt will be matched at bitpocket root dir ( ~/BitPocket/misio.txt ).

This exclude file is passed to rsync as --exclude-from argument, check man rsync for INCLUDE/EXCLUDE PATTERN RULES.

You can set up even more advanced exclusion/inclusion rules. In all, there there are three files that you can create to change this configuration:

.bitpocket/exclude
.bitpocket/include
.bitpocket/filter

Be aware that all the quirks from rsync exclusion/inclusion rules carry over into bitpocket. If you decide that you need such advanced configuration, make sure that you understand those rules very well, and consider double checking them before syncing by running bitpocket list.

Both local and remote backups can be enabled or disabled in the configuration. By default, local backups are enabled and remote backups are not. Based on how intend to use bitpocket, you may wish to change this behavior. Add or change these lines in your .bitpocket/config file:

# BACKUPS=true
# REMOTE_BACKUPS=false

You can pass additional switches to rsync by setting RSYNC_OPTS in .bitpocket/config file. Generated config file includes (commented out) example setting for dereferencing symlinks:

# RSYNC_OPTS="-L"

Just uncomment it and change at will.

When syncing takes more than 10 seconds (SLOW_SYNC_TIME setting) bitpocket can fire off user provided command in background. This can be usefull to notify user about long sync happening, preventing him from turning off the machine during sync etc.

There are 3 settings that can be enabled in .bitpocket/config file:

# SLOW_SYNC_TIME=10
# SLOW_SYNC_START_CMD="notify-send 'BitPocket sync in progress...'"
# SLOW_SYNC_STOP_CMD="notify-send 'BitPocket sync finished'"

Just uncomment them and change at will.

You can show tray icon during long sync with traytor and following settings:

SLOW_SYNC_START_CMD='~/bin/traytor -t "BitPocket syncing..." -c "xdg-open ." .bitpocket/icons & echo $! >.bitpocket/traytor.pid'
SLOW_SYNC_STOP_CMD='kill `cat .bitpocket/traytor.pid`'

You can add a remote mount point check to ensure a remote path is available prior to the sync running. If the folder is not mounted on the remote server, then the sync will be aborted. The path must be an absolute path which is expected to be a mountpoint on the remote server. Add or change this lines in your .bitpocket/config file:

# REMOTE_MOUNTPOINT=/

• Marcin Kulik | @ku1ik | https://github.com/ku1ik | http://ku1ik.com/
• torfason | https://github.com/torfason