[v5.x] feat(backup): standardize v4 database backups#9660
Draft
peaklabs-dev wants to merge 14 commits intonextfrom
Draft
[v5.x] feat(backup): standardize v4 database backups#9660peaklabs-dev wants to merge 14 commits intonextfrom
peaklabs-dev wants to merge 14 commits intonextfrom
Conversation
- One backup job can now contain multiple databases, not just one. Therefore, we will store the database names or "all" in this column instead of just one database per backup execution.
- when one database or multiple are listed for backup, back them up one by one and then bundle them into an archive. This eliminates multiple execution logs per backup (previously one for each database backed up) and consolidates them into a single execution regardless of how many separate databases are backed up, making backups and restores simpler and preparing for v5 - fixes mongo:4 single-DB backups dumping all databases instead of the selected one (missing --db flag) - fixes mariadb dump_all backup output not being compressed despite .gz extension (--compress does not compress output) - removes --compress from mysql dump_all backup (already gzipped) - remove unused code
- in previous versions, mongodb used empty databases_to_backup to dump all DBs. This is weird and it should better be like the other databases using the dump_all flag.
- add or improve backup settings helper text - make database to backup required if dumpAll is false
- add new restore commands to restore one or more databases from an archive - move single database restore command into a legacy option - improve ui with a select component instead of checkboxes - fixes postgres restore failing when the default database does not exist (use template1 which is guaranteed to exist) - fixes mysql/mariadb dump_all restore piping all databases into one target database instead of restoring each to its own
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why?
This PR standardizes the
v4.xdatabase backup format to prepare for thev5.xupgrade migration, turning what would be a high-impact breaking change into a low-impact one.In v5, backup and restore will be consolidated onto a single page.
Legacybackup formats, the import page andlegacyimport logic will be removed entirely.To facilitate this switch while maintaining the ability to restore some v4 backups in v5 the backup format needs to be standardized now in v4 so that backups created after this change are directly compatible with v5.
PR #8131 documents the problems with the current backup system in detail (3 different backup methods, confusing defaults, overcomplicated restores). That PR solved it by removing the per-database backup option entirely and always backing up all databases. While that simplified things, it introduced drawbacks that matter in practice:
pg_restore --tablecan restore a single table, for example.This PR takes a different approach that standardizes the format without removing flexibility:
.tar.gzarchive.legacy. But the backup files (.dmpand.sql) remain fully restorable in v4. They are marked aslegacynow as these files will not be migrated over to v5.Changes
databases_to_backup(empty ornull) todump_all = truemongo:4single-DB backups dumping all databases instead of the selected one (missing--dbflag)dump_allbackup output not being compressed despite.gzextension (--compressdoes not compress output)--compressfrom mysqldump_allbackup (already gzipped)template1which is guaranteed to exist)dump_allrestore piping all databases into one target database instead of restoring each to its owndump_allcheckbox to MongoDB (previously used emptydatabases_to_backupto mean "all")dump_allis disableddatabase_namecolumn todatabasesSupersedes