From: Stoiko Ivanov <s.ivanov@proxmox.com>
To: pmg-devel@pve.proxmox.com
Subject: [pmg-devel] [PATCH pve-common/api/gui] add initial PBS integration
Date: Wed, 28 Oct 2020 19:54:16 +0100 [thread overview]
Message-ID: <20201028185432.23067-1-s.ivanov@proxmox.com> (raw)
changes RFC->v1:
* moved the potentially reusable parts to pve-common (PBSTools.pm, and 2
functions in Systemd.pm)
* added GUI support (mostly adapted from the LDAPConfig) - huge thanks to
Dominik for his patience and help!
* added support for encrypted backups
* added support for pruning backups
* added a systemd-timer, which runs a daily prune based on the configured
settings
differences from the PBS storage-plugin config in PVE:
* the keep-options are kept as separate options instead of using a property
string for all (like they are in the datastore config in PBS)
cover-letter from the RFC:
This series is a minimal proof-of-concept for integrating PBS into PMG.
The code needs quite a bit of cleanup, and more testing, however I'll send it
as RFC, for an initial sanity-check - to see if I missed some needed
functionality.
It reuses quite a bit of code from pve-storage (PBSPlugin, and the systemd
unit manipulation from the PVE::API2::Disks::Directory) and I'd like to
refactor the common parts into pve-common.
The backup consists of the same files, that end up in a locally generated
PMG Backup (instead of creating a tar.gz we send the data to a PBS instance).
What works:
* creating and manipulating 'remotes' (PBS instances where we back up to)
* creating, restoring and forgetting backups to those remotes
* restoring a backup from another PMG installation from a configured remote
* creating schedules for backups (systemd-timer units) for periodic backups
via CLI (pmgbackup, pmgsh)
What's missing:
* quite a bit of refactoring (e.g. move the systemd-timer helpers and most of
PBSTools to pve-common (and reuse it in the PBS Storage plugin)
* encryption support
* pruneing support
* testing
* GUI
* Documentation
The first patch is a small cosmetic cleanup, and independent of the series.
A sample session (which I used for testing:
```
pmgbackup remote add pbs1 --datastore local --server 192.0.2.11 --user root@pam --password xxx --fingerprint ff:ff:...
pmgbackup remote list
pmgbackup pbsjob run pbs1
pmgbackup pbsjob restore pbs1 -althost pmg-live -config 1 -database 1 -statistic 1
pmgbackup pbsjob run pbs1
pmgbackup pbsjob list_backups pbs1
pmgbackup pbsjob create pbs1 --schedule 'minutely' --delay '0s'
systemctl list-timers --all
```
pve-common:
Stoiko Ivanov (2):
Systemd: add helpers for parsing unit files
add helper module for handling PBS Integration
src/Makefile | 1 +
src/PVE/PBSTools.pm | 309 ++++++++++++++++++++++++++++++++++++++++++++
src/PVE/Systemd.pm | 72 +++++++++++
3 files changed, 382 insertions(+)
create mode 100644 src/PVE/PBSTools.pm
pmg-api:
Stoiko Ivanov (11):
drop left-over commented out code
Backup: split backup creation and creating tar
Restore: optionally restore from directory
Backup: push restore options to PMG::Backup
debian: add dependency on proxmox-backup-client
add initial SectionConfig for pbs
Add API2 module for PBS configuration
Add API2 module for per-node backups to PBS
pbs-integration: add CLI calls to pmgbackup
add scheduled backup to PBS remotes
add daily timer for pruning configured remotes
debian/control | 1 +
debian/dirs | 1 +
debian/pmg-pbsbackup@.service | 6 +
debian/pmg-pbsprune.service | 6 +
debian/pmg-pbsprune.timer | 10 +
debian/rules | 4 +-
src/Makefile | 8 +-
src/PMG/API2/Backup.pm | 21 +-
src/PMG/API2/Config.pm | 7 +
src/PMG/API2/Nodes.pm | 7 +
src/PMG/API2/PBS/Job.pm | 544 ++++++++++++++++++++++++++++++++++
src/PMG/API2/PBS/Remote.pm | 248 ++++++++++++++++
src/PMG/Backup.pm | 86 +++++-
src/PMG/CLI/pmgbackup.pm | 102 +++++++
src/PMG/PBSConfig.pm | 209 +++++++++++++
src/PMG/PBSSchedule.pm | 104 +++++++
16 files changed, 1327 insertions(+), 37 deletions(-)
create mode 100644 debian/pmg-pbsbackup@.service
create mode 100644 debian/pmg-pbsprune.service
create mode 100644 debian/pmg-pbsprune.timer
create mode 100644 src/PMG/API2/PBS/Job.pm
create mode 100644 src/PMG/API2/PBS/Remote.pm
create mode 100644 src/PMG/PBSConfig.pm
create mode 100644 src/PMG/PBSSchedule.pm
pmg-gui:
Stoiko Ivanov (3):
Make Backup/Restore panel a menuentry
refactor RestoreWindow for PBS
add PBSConfig tab to Backup menu
js/BackupConfiguration.js | 23 ++
js/BackupRestore.js | 68 ++--
js/Makefile | 2 +
js/NavigationTree.js | 6 +
js/PBSConfig.js | 680 ++++++++++++++++++++++++++++++++++++++
js/SystemConfiguration.js | 4 -
6 files changed, 750 insertions(+), 33 deletions(-)
create mode 100644 js/BackupConfiguration.js
create mode 100644 js/PBSConfig.js
--
2.20.1
next reply other threads:[~2020-10-28 18:54 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-28 18:54 Stoiko Ivanov [this message]
2020-10-28 18:54 ` [pmg-devel] [PATCH pve-common 1/2] Systemd: add helpers for parsing unit files Stoiko Ivanov
2020-10-30 8:26 ` [pmg-devel] applied: " Dietmar Maurer
2020-11-10 8:34 ` [pmg-devel] " Thomas Lamprecht
2020-10-28 18:54 ` [pmg-devel] [PATCH pve-common 2/2] add helper module for handling PBS Integration Stoiko Ivanov
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 01/11] drop left-over commented out code Stoiko Ivanov
2020-10-30 5:58 ` [pmg-devel] applied: " Dietmar Maurer
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 02/11] Backup: split backup creation and creating tar Stoiko Ivanov
2020-10-30 6:20 ` [pmg-devel] applied: " Dietmar Maurer
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 03/11] Restore: optionally restore from directory Stoiko Ivanov
2020-10-30 6:26 ` [pmg-devel] applied: " Dietmar Maurer
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 04/11] Backup: push restore options to PMG::Backup Stoiko Ivanov
2020-10-30 6:32 ` [pmg-devel] applied: " Dietmar Maurer
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 05/11] debian: add dependency on proxmox-backup-client Stoiko Ivanov
2020-10-30 6:33 ` [pmg-devel] applied: " Dietmar Maurer
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 06/11] add initial SectionConfig for pbs Stoiko Ivanov
2020-10-30 6:38 ` Dietmar Maurer
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 07/11] Add API2 module for PBS configuration Stoiko Ivanov
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 08/11] Add API2 module for per-node backups to PBS Stoiko Ivanov
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 09/11] pbs-integration: add CLI calls to pmgbackup Stoiko Ivanov
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 10/11] add scheduled backup to PBS remotes Stoiko Ivanov
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-api 11/11] add daily timer for pruning configured remotes Stoiko Ivanov
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-gui 1/3] Make Backup/Restore panel a menuentry Stoiko Ivanov
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-gui 2/3] refactor RestoreWindow for PBS Stoiko Ivanov
2020-10-28 18:54 ` [pmg-devel] [PATCH pmg-gui 3/3] add PBSConfig tab to Backup menu Stoiko Ivanov
2020-10-30 5:47 ` [pmg-devel] [PATCH pve-common/api/gui] add initial PBS integration Dietmar Maurer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201028185432.23067-1-s.ivanov@proxmox.com \
--to=s.ivanov@proxmox.com \
--cc=pmg-devel@pve.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox