[pmg-devel] [PATCH pve-common/pmg-api/pmg-gui v3] add initial PBS integration

[Stoiko Ivanov <s.ivanov@proxmox.com>]

changes v2->v3:
* rebased the gui-patches to latest master
* added param_mapping from pve-storage/pvesm (provide an interactive password
  entry)

cover-letter for v2:
changes v1->v2:
* renamed PBSTools to PBSClient, and made it a class (handling the config and
  backup-operations of one PBS instance)
* dropped encryption-key support from the GUI, the API calls and the
  SectionConfig - will be submitted again, when we have a clean solution in PVE
* dropped explicit prune support - each group now gets pruned directly after a
  backup
* one small fixup in debian/control (I mistakenly added two ',' in the
  proxmox-backup-client dependency)
* added /etc/pmg/pbs to the cluster-sync

cover-letter for v1:
changes RFC->v1:
* moved the potentially reusable parts to pve-common (PBSTools.pm, and 2
  functions in Systemd.pm)
* added GUI support (mostly adapted from the LDAPConfig) - huge thanks to
  Dominik for his patience and help!
* added support for encrypted backups
* added support for pruning backups
* added a systemd-timer, which runs a daily prune based on the configured
  settings

differences from the PBS storage-plugin config in PVE:
* the keep-options are kept as separate options instead of using a property
  string for all (like they are in the datastore config in PBS)


cover-letter from the RFC:
This series is a minimal proof-of-concept for integrating PBS into PMG.

The code needs quite a bit of cleanup, and more testing, however I'll send it
as RFC, for an initial sanity-check - to see if I missed some needed
functionality.

It reuses quite a bit of code from pve-storage (PBSPlugin, and the systemd
unit manipulation from the PVE::API2::Disks::Directory) and I'd like to
refactor the common parts into pve-common.

The backup consists of the same files, that end up in a locally generated
PMG Backup (instead of creating a tar.gz we send the data to a PBS instance).

What works:
* creating and manipulating 'remotes' (PBS instances where we back up to)
* creating, restoring and forgetting backups to those remotes
* restoring a backup from another PMG installation from a configured remote
* creating schedules for backups (systemd-timer units) for periodic backups
via CLI (pmgbackup, pmgsh)

What's missing:
* quite a bit of refactoring (e.g. move the systemd-timer helpers and most of
  PBSTools to pve-common (and reuse it in the PBS Storage plugin)
  * encryption support
  * pruneing support
  * testing
  * GUI
  * Documentation

  The first patch is a small cosmetic cleanup, and independent of the series.

  A sample session (which I used for testing:
  ```
  pmgbackup remote add pbs1 --datastore local --server 192.0.2.11 --user root@pam --password xxx --fingerprint  ff:ff:...
  pmgbackup remote list
  pmgbackup pbsjob run pbs1
  pmgbackup pbsjob restore pbs1 -althost pmg-live -config 1 -database 1 -statistic 1
  pmgbackup pbsjob run pbs1
  pmgbackup pbsjob list_backups pbs1
  pmgbackup pbsjob create pbs1 --schedule 'minutely' --delay '0s'
  systemctl list-timers --all
  ```

pve-common:
Stoiko Ivanov (1):
  add PBSClient module

 src/Makefile         |   1 +
 src/PVE/PBSClient.pm | 305 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 306 insertions(+)
 create mode 100644 src/PVE/PBSClient.pm

pmg-api:
Stoiko Ivanov (7):
  debian: drop duplicate ',' in dependencies
  add initial SectionConfig for PBS
  Add API2 module for PBS configuration
  Add API2 module for per-node backups to PBS
  pbs-integration: add CLI calls to pmgbackup
  add scheduled backup to PBS remotes
  add /etc/pmg/pbs to cluster-sync

 debian/control                |   2 +-
 debian/dirs                   |   1 +
 debian/pmg-pbsbackup@.service |   6 +
 debian/rules                  |   1 +
 src/Makefile                  |   6 +-
 src/PMG/API2/Config.pm        |   7 +
 src/PMG/API2/Nodes.pm         |   7 +
 src/PMG/API2/PBS/Job.pm       | 501 ++++++++++++++++++++++++++++++++++
 src/PMG/API2/PBS/Remote.pm    | 231 ++++++++++++++++
 src/PMG/CLI/pmgbackup.pm      |  60 ++++
 src/PMG/Cluster.pm            |   1 +
 src/PMG/PBSConfig.pm          | 195 +++++++++++++
 src/PMG/PBSSchedule.pm        | 104 +++++++
 13 files changed, 1120 insertions(+), 2 deletions(-)
 create mode 100644 debian/pmg-pbsbackup@.service
 create mode 100644 src/PMG/API2/PBS/Job.pm
 create mode 100644 src/PMG/API2/PBS/Remote.pm
 create mode 100644 src/PMG/PBSConfig.pm
 create mode 100644 src/PMG/PBSSchedule.pm

pmg-gui:
Stoiko Ivanov (3):
  Make Backup/Restore panel a menuentry
  refactor RestoreWindow for PBS
  add PBSConfig tab to Backup menu

 js/BackupConfiguration.js |  23 ++
 js/BackupRestore.js       |  67 ++--
 js/Makefile               |   2 +
 js/NavigationTree.js      |   6 +
 js/PBSConfig.js           | 678 ++++++++++++++++++++++++++++++++++++++
 js/SystemConfiguration.js |   4 -
 6 files changed, 748 insertions(+), 32 deletions(-)
 create mode 100644 js/BackupConfiguration.js
 create mode 100644 js/PBSConfig.js

-- 
2.20.1