On Dec 24, 2021, at 6:09 PM, Ben Johnson ***@***.***> wrote: Hi @rwboyer <https://github.com/rwboyer>, good question. There's a section on the How it works page of the web site called Snapshots & Generations <https://litestream.io/how-it-works/#snapshots--generations> that gives some more detail but I'll try to be more clear here. Essentially, a "generation" contains at least one snapshot and then a contiguous series of WAL pages. Since Litestream can be stopped and restarted separate from your application, it could miss updates to the database while it is stopped. If Litestream detects that it can't resume from where it left off, it'll start a new generation. Here's a visual. Generation XXXXXXXX starts with a snapshot #1 <#1> and begins replicating its WAL. Eventually it may create another snapshot #2 <#2> (depending on your snapshot frequency settings). If Litestream gets shutdown or perhaps you restore to a new server after WAL #3 <#3> then Litestream detects that it can't resume a contiguous set of WAL pages, and it will start generation YYYYYYYY. Within a generation, you can restore from a snapshot and replay WAL files forward to get to a specific point-in-time. However, you can't use WAL files across generations as there was a break in the replication so Litestream wouldn't be able to guarantee that your restored database has all the correct pages. Generation XXXXXXXX ═══════════════════════════════════════════════════════════════════════════╗ ║ ║ ║ ║ ║ ┌────────────┐ ┌────────────┐ ║ ║ │ Snapshot │ │ Snapshot │ ║ ║ │ #0000000 │ │ #2 │ ║ ║ └────────────┘ └────────────┘ ║ ║ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ║ ║ │ WAL │ │ WAL │ │ WAL │ │ WAL │ ║ ║ │ #0000000 │ │ #1 │ │ #2 │ │ #3 │ ║ ║ └────────────┘ └────────────┘ └────────────┘ └────────────┘ ║ ║ ║ ║ ───────────────────────────────────────────────────────────────────────────────────────▶ ║ ║ ║ ╚═══════════════════════════════════════════════════════════════════════════════════════════════╝ Generation YYYYYYYY ═════════════════════════════════════════════╗ ║ ║ ║ ║ ║ ┌────────────┐ ║ ║ │ Snapshot │ ║ ║ │ #0000000 │ ║ ║ └────────────┘ ║ ║ ┌────────────┐ ┌────────────┐ ┌────────────┐ ║ ║ │ WAL │ │ WAL │ │ WAL │ ║ ║ │ #0000000 │ │ #1 │ │ #2 │ ║ ║ └────────────┘ └────────────┘ └────────────┘ ║ ║ ║ ║ ──────────────────────────────────────────────────────────▶ ║ ║ ║ ╚═════════════════════════════════════════════════════════════════╝ As for your other questions: Do [generations] get pruned on the target backup bucket? Yes. The retention enforcement removes old snapshots once the retention period elapses. If no more snapshots exist in a generation then the generation is removed as well. If no more snapshots exist in the current generation then a new snapshot will be created before removing the old one. WAL files are only retained from the index of the oldest snapshot. Do the snapshot and retention configuration items have any influence on what's being seen on the S3 bucket? Yes. The snapshot-interval configuration determines how often a new snapshot is created. If it isn't set then a new snapshot is created when an old snapshot is being removed because of retention. The retention configuration option specifies how old snapshots can be before they are removed. For example, you could set retention to 720h for 30 days of retention. You can also set snapshot-interval to 24h to create a new snapshot every day. The benefit to creating more frequent snapshots is that it will take less time to restore since it only needs to replay WAL files since the most recent snapshot. There's more details in the Replica settings <https://litestream.io/reference/config/#replica-settings> section on the web site. — Reply to this email directly, view it on GitHub <#258 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAU2GS5PBIIF5N7UBWTZTDUST4UHANCNFSM5KVTL5NQ>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign=notification-email&utm_medium=email&utm_source=github>. You are receiving this because you were mentioned.