We had this feature a very long time ago, but it was quite broken - it expected every plugin to write it's backup contents to stdout and then tar'd it all up, which ended up causing broken backups in some cases.

This is actually a bit more complex than you'd expect.

• Datastore plugins don't tie a given service to an app (one datastore can be tied to multiple apps... should we back them up twice?
• If config is not specifically app-related, does it get backed up? If so, how?
• What if you want to backup to an external host? Or if you run out of disk space during the backup?
• If we implemented the interface, we'd need to implement it for the entire core (which isn't a single-day of work). And we'd need to ensure the process continues to work as new functionality is implemented.
• For things stored in the app folder (blech), in what order do they get restored?

I'd previously tried to raise money via Github Sponsors for backup/restore functionality across the project, but didn't get to my goal of $500. That said, if you'd like to work on this functionality, by all means go ahead. I'd be happy to provide feedback once you have a skeleton working to get it to a place that is usable by core, datastore, and community plugins.

Dokku is more or less at the $500 a month goal, so I'll start working on this in the next few releases. I think there is still some work to move a few files out of the git repositories (docker options, env vars, nginx configs) but those should be relatively straightforward to do.

Let me think about a good backup interface and then post my comments here. I'm guessing a few things will change about how datastores do backups, but I believe that will be a Good Thing™.

Okay I think what I'll do is something like this:

backup:export [--app $APP] [--service $SERVICE] [--backup-dir TMP]

This new command will create a tar.gz file in the /tmp directory with the desired data. The /tmp dir may be overriden via the --backup-dir flag, and the file will be output there if so. Dokku will attempt to write a temporary file to the --backup-dir, and if it fails, the backup will exit immediately.

The backup will be buffered to a directory in /tmp while being generated. Users are expected to have enough space in /tmp to hold backup data.

If no app/service is specified, the backup will contain all apps and all services.

The full path to the tarball will be written to stdout.

Log messages for backup:export should always appear on stderr.

• backup-global-export $PATH: New trigger for backing up global settings for a plugin. The plugin can create a directory in that path and export whatever it wants to that directory. I would suggest plugins create a data and a config dir, and place their files appropriately.
• backup-app-export $APP $PATH: New trigger that takes an app name and a path. The plugin can create a directory in that path and export whatever it wants to that directory. I would suggest plugins create a data and a config dir, and place their files appropriately.
• backup-service-export $SERVICE_TYPE $SERVICE_NAME $PATH: New trigger that takes a service type (eg. mysql), name and a path. The plugin can create a directory in that path and export whatever it wants to that directory. I would suggest plugins create a data and a config dir, and place their files appropriately.
• pre-backup-export $PATH $BACKUP_FILE: Will allow plugins to do things prior to the export (though likely unnecessary).
• pre-backup-app-export $APP $PATH: Will allow plugins to do things prior to the export (though likely unnecessary).
• post-backup-app-export $APP $PATH: Will allow plugins to do things after to the export (though likely unnecessary).
• pre-backup-service-export $APP $PATH: Will allow plugins to do things prior to the export (though likely unnecessary).
• post-backup-service-export $APP $PATH: Will allow plugins to do things after to the export (though likely unnecessary).
• post-backup-export $PATH $BACKUP_FILE: Will allow plugins to do things after to the export (though likely unnecessary).

backup:import <backup-file-path> [--app $APP] [--service $SERVICE]

This new command will take a backup file path and an optional app/service name. If no app/service is specified, it will attempt to import everything.

This command may be destructive, and users are expected to take caution when running it. Confirmation of restoring on top of existing data will occur.

• If an app already exists, dokku will ask for confirmation before deleting the old app and restoring the new backup.
• If an service already exists, dokku will ask for confirmation before deleting the old app and restoring the new backup.

When restoring, if an app or service is not specified, the entire backup is first extracted before being restored. Thus, users should have enough space for 2x the size of the extracted backup.

Restores aren't guaranteed to work in all cases - particularly if dependencies are missing or there are external issues like a docker bug. Users will be encouraged to regularly test their backups on new servers.

• backup-global-import $PATH: New trigger, but for imports. Plugins should not do anything here other than import files. No restarts or anything should be triggered as the import may not be complete.
• backup-app-import $APP $PATH: New trigger, but for imports. Plugins should not do anything here other than import files. No restarts or anything should be triggered as the import may not be complete.
• backup-service-import $SERVICE_TYPE $SERVICE_NAME $PATH: New trigger, but for imports. Plugins should not do anything here other than import files. No restarts or anything should be triggered as the import may not be complete.
• pre-backup-import $PATH $BACKUP_FILE: Will allow plugins to do things prior to the import
• pre-backup-app-import $APP $PATH: Will allow plugins to do things prior to the import
• post-backup-app-import $APP $PATH: Will allow plugins to do things after to the import
• pre-backup-service-import $APP $PATH: Will allow plugins to do things prior to the export, like ensure the service doesn't already exist
• post-backup-service-import $APP $PATH: Will allow plugins to do things after to the export, like start the service
• post-backup-import $PATH $BACKUP_FILE: Will allow plugins to do things after to the import, like rebuild all apps

That sounds great! If I was going to extend this functionality to e.g. store backups on S3, would I use the post-backup and pre-import hooks? How would that plugin prune old backups? (Totally fine if the last bit is just a custom command or something)

I think I would like to concentrate on this creating a tarball on disk. If someone wants to upload that somewhere, they can do so - similarly, if their backup process should prune old backups, they can handle that themselves.

To extend and do something custom with the file, you'd use a hook of some sort.

Absolutely -- not suggesting that s3 storage be implemented in Dokku directly. Just thinking through the ways that this could be extended by other plugins -- if the answer is that s3 storage stuff should be done without interacting with the backup plugin directly, that's totally workable.

I just want to avoid the issue we have with the datastore plugins where each one implements S3. Folks use things other than S3, so it makes us seem like users cant do something else.

I'd rather we document tools in the ecosystem to make those things possible, maybe even writing a tutorial for them.

Totally understood! Appreciate the time and thought :)