Snapshots, Retention for short lived replications #446
-
|
I'm interested in using litestream for syncing the state of a process which runs every half hour. Currently the process copies the sqlite db down from S3 at the start, writes data into it and then copies the database back up at the end. This is fine when the process takes a few seconds (as it normally does). It has the potential to take a lot longer though, if a large swathe of data needs reprocessing. In that case any failure could throw away hours of work. Enter litestream, and replicating the changes off to S3 every second (probably every minute in reality) via running the process with
|
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
No, it probably won't. It only checks periodically based on the
It would be a good idea to have it enforce on exit or even have a |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the reply @benbjohnson . In the end, I went with "background thread which periodically backs up the DB, gzips it and posts it back to S3". It's not ideal in that I'm copying more data than I need to, but it works which is the main thing. Having such an |
Beta Was this translation helpful? Give feedback.
No, it probably won't. It only checks periodically based on the
retention-check-intervalin the config.It would be a good idea to have it enforce on exit or even have a
litestream enforce-retentioncommand but that doesn't currently exist.