public inbox for pmg-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Stoiko Ivanov <s.ivanov@proxmox.com>
To: pmg-devel@lists.proxmox.com
Subject: [pmg-devel] [RFC pmg-api 00/12] POC PBS integration
Date: Mon, 19 Oct 2020 21:01:57 +0200	[thread overview]
Message-ID: <20201019190209.11495-1-s.ivanov@proxmox.com> (raw)

This series is a minimal proof-of-concept for integrating PBS into PMG.

The code needs quite a bit of cleanup, and more testing, however I'll send it
as RFC, for an initial sanity-check - to see if I missed some needed
functionality.

It reuses quite a bit of code from pve-storage (PBSPlugin, and the systemd
unit manipulation from the PVE::API2::Disks::Directory) and I'd like to
refactor the common parts into pve-common.

The backup consists of the same files, that end up in a locally generated
PMG Backup (instead of creating a tar.gz we send the data to a PBS instance).

What works:
* creating and manipulating 'remotes' (PBS instances where we back up to)
* creating, restoring and forgetting backups to those remotes
* restoring a backup from another PMG installation from a configured remote
* creating schedules for backups (systemd-timer units) for periodic backups
via CLI (pmgbackup, pmgsh)

What's missing:
* quite a bit of refactoring (e.g. move the systemd-timer helpers and most of
  PBSTools to pve-common (and reuse it in the PBS Storage plugin)
* encryption support
* pruneing support
* testing
* GUI
* Documentation

The first patch is a small cosmetic cleanup, and independent of the series.

A sample session (which I used for testing:
```
pmgbackup remote add pbs1 --datastore local --server 192.0.2.11 --user root@pam --password xxx --fingerprint  ff:ff:...
pmgbackup remote list
pmgbackup pbsjob run pbs1
pmgbackup pbsjob restore pbs1 -althost pmg-live -config 1 -database 1 -statistic 1
pmgbackup pbsjob run pbs1
pmgbackup pbsjob list_backups pbs1
pmgbackup pbsjob create pbs1 --schedule 'minutely' --delay '0s'
systemctl list-timers --all
```

Stoiko Ivanov (12):
  drop left-over commented out code
  Backup: split backup creation and creating tar
  Restore: optionally restore from directory
  Backup: push restore options to PMG::Backup
  debian: add dependency on proxmox-backup-client
  add helper module for handling PBS Integration
  PBSTools: add methods for managing backups
  PBSTools: add systemd-timer helpers
  add initial SectionConfig for pbs
  Add API2 module for PBS configuration
  Add API2 module for per-node backups to PBS
  pbs-integration: add CLI calls to pmgbackup

 debian/control                |   1 +
 debian/dirs                   |   1 +
 debian/pmg-pbsbackup@.service |   6 +
 debian/rules                  |   1 +
 src/Makefile                  |   6 +-
 src/PMG/API2/Backup.pm        |  21 +-
 src/PMG/API2/Config.pm        |   7 +
 src/PMG/API2/Nodes.pm         |   7 +
 src/PMG/API2/PBS/Job.pm       | 467 ++++++++++++++++++++++++++++++++++
 src/PMG/API2/PBS/Remote.pm    | 218 ++++++++++++++++
 src/PMG/Backup.pm             |  86 +++++--
 src/PMG/CLI/pmgbackup.pm      |  27 ++
 src/PMG/PBSConfig.pm          | 168 ++++++++++++
 src/PMG/PBSTools.pm           | 466 +++++++++++++++++++++++++++++++++
 14 files changed, 1447 insertions(+), 35 deletions(-)
 create mode 100644 debian/pmg-pbsbackup@.service
 create mode 100644 src/PMG/API2/PBS/Job.pm
 create mode 100644 src/PMG/API2/PBS/Remote.pm
 create mode 100644 src/PMG/PBSConfig.pm
 create mode 100644 src/PMG/PBSTools.pm

-- 
2.20.1





             reply	other threads:[~2020-10-19 19:02 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-19 19:01 Stoiko Ivanov [this message]
2020-10-19 19:01 ` [pmg-devel] [RFC pmg-api 01/12] drop left-over commented out code Stoiko Ivanov
2020-10-19 19:01 ` [pmg-devel] [RFC pmg-api 02/12] Backup: split backup creation and creating tar Stoiko Ivanov
2020-10-20  5:43   ` Dietmar Maurer
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 03/12] Restore: optionally restore from directory Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 04/12] Backup: push restore options to PMG::Backup Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 05/12] debian: add dependency on proxmox-backup-client Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 06/12] add helper module for handling PBS Integration Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 07/12] PBSTools: add methods for managing backups Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 08/12] PBSTools: add systemd-timer helpers Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 09/12] add initial SectionConfig for pbs Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 10/12] Add API2 module for PBS configuration Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 11/12] Add API2 module for per-node backups to PBS Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 12/12] pbs-integration: add CLI calls to pmgbackup Stoiko Ivanov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201019190209.11495-1-s.ivanov@proxmox.com \
    --to=s.ivanov@proxmox.com \
    --cc=pmg-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal