[pmg-devel] [PATCH pve-common/api/gui] add initial PBS integration

[Stoiko Ivanov <s.ivanov@proxmox.com>]

changes RFC->v1:
* moved the potentially reusable parts to pve-common (PBSTools.pm, and 2
  functions in Systemd.pm)
* added GUI support (mostly adapted from the LDAPConfig) - huge thanks to
  Dominik for his patience and help!
* added support for encrypted backups
* added support for pruning backups
* added a systemd-timer, which runs a daily prune based on the configured
  settings

differences from the PBS storage-plugin config in PVE:
* the keep-options are kept as separate options instead of using a property
  string for all (like they are in the datastore config in PBS)


cover-letter from the RFC:
This series is a minimal proof-of-concept for integrating PBS into PMG.

The code needs quite a bit of cleanup, and more testing, however I'll send it
as RFC, for an initial sanity-check - to see if I missed some needed
functionality.

It reuses quite a bit of code from pve-storage (PBSPlugin, and the systemd
unit manipulation from the PVE::API2::Disks::Directory) and I'd like to
refactor the common parts into pve-common.

The backup consists of the same files, that end up in a locally generated
PMG Backup (instead of creating a tar.gz we send the data to a PBS instance).

What works:
* creating and manipulating 'remotes' (PBS instances where we back up to)
* creating, restoring and forgetting backups to those remotes
* restoring a backup from another PMG installation from a configured remote
* creating schedules for backups (systemd-timer units) for periodic backups
via CLI (pmgbackup, pmgsh)

What's missing:
* quite a bit of refactoring (e.g. move the systemd-timer helpers and most of
  PBSTools to pve-common (and reuse it in the PBS Storage plugin)
  * encryption support
  * pruneing support
  * testing
  * GUI
  * Documentation

  The first patch is a small cosmetic cleanup, and independent of the series.

  A sample session (which I used for testing:
  ```
  pmgbackup remote add pbs1 --datastore local --server 192.0.2.11 --user root@pam --password xxx --fingerprint  ff:ff:...
  pmgbackup remote list
  pmgbackup pbsjob run pbs1
  pmgbackup pbsjob restore pbs1 -althost pmg-live -config 1 -database 1 -statistic 1
  pmgbackup pbsjob run pbs1
  pmgbackup pbsjob list_backups pbs1
  pmgbackup pbsjob create pbs1 --schedule 'minutely' --delay '0s'
  systemctl list-timers --all
  ```


pve-common:
Stoiko Ivanov (2):
  Systemd: add helpers for parsing unit files
  add helper module for handling PBS Integration

 src/Makefile        |   1 +
 src/PVE/PBSTools.pm | 309 ++++++++++++++++++++++++++++++++++++++++++++
 src/PVE/Systemd.pm  |  72 +++++++++++
 3 files changed, 382 insertions(+)
 create mode 100644 src/PVE/PBSTools.pm

pmg-api:
Stoiko Ivanov (11):
  drop left-over commented out code
  Backup: split backup creation and creating tar
  Restore: optionally restore from directory
  Backup: push restore options to PMG::Backup
  debian: add dependency on proxmox-backup-client
  add initial SectionConfig for pbs
  Add API2 module for PBS configuration
  Add API2 module for per-node backups to PBS
  pbs-integration: add CLI calls to pmgbackup
  add scheduled backup to PBS remotes
  add daily timer for pruning configured remotes

 debian/control                |   1 +
 debian/dirs                   |   1 +
 debian/pmg-pbsbackup@.service |   6 +
 debian/pmg-pbsprune.service   |   6 +
 debian/pmg-pbsprune.timer     |  10 +
 debian/rules                  |   4 +-
 src/Makefile                  |   8 +-
 src/PMG/API2/Backup.pm        |  21 +-
 src/PMG/API2/Config.pm        |   7 +
 src/PMG/API2/Nodes.pm         |   7 +
 src/PMG/API2/PBS/Job.pm       | 544 ++++++++++++++++++++++++++++++++++
 src/PMG/API2/PBS/Remote.pm    | 248 ++++++++++++++++
 src/PMG/Backup.pm             |  86 +++++-
 src/PMG/CLI/pmgbackup.pm      | 102 +++++++
 src/PMG/PBSConfig.pm          | 209 +++++++++++++
 src/PMG/PBSSchedule.pm        | 104 +++++++
 16 files changed, 1327 insertions(+), 37 deletions(-)
 create mode 100644 debian/pmg-pbsbackup@.service
 create mode 100644 debian/pmg-pbsprune.service
 create mode 100644 debian/pmg-pbsprune.timer
 create mode 100644 src/PMG/API2/PBS/Job.pm
 create mode 100644 src/PMG/API2/PBS/Remote.pm
 create mode 100644 src/PMG/PBSConfig.pm
 create mode 100644 src/PMG/PBSSchedule.pm

pmg-gui:
Stoiko Ivanov (3):
  Make Backup/Restore panel a menuentry
  refactor RestoreWindow for PBS
  add PBSConfig tab to Backup menu

 js/BackupConfiguration.js |  23 ++
 js/BackupRestore.js       |  68 ++--
 js/Makefile               |   2 +
 js/NavigationTree.js      |   6 +
 js/PBSConfig.js           | 680 ++++++++++++++++++++++++++++++++++++++
 js/SystemConfiguration.js |   4 -
 6 files changed, 750 insertions(+), 33 deletions(-)
 create mode 100644 js/BackupConfiguration.js
 create mode 100644 js/PBSConfig.js

-- 
2.20.1