all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Stoiko Ivanov <s.ivanov@proxmox.com>
To: pmg-devel@lists.proxmox.com
Subject: [pmg-devel] [RFC pmg-api 00/12] POC PBS integration
Date: Mon, 19 Oct 2020 21:01:57 +0200	[thread overview]
Message-ID: <20201019190209.11495-1-s.ivanov@proxmox.com> (raw)

This series is a minimal proof-of-concept for integrating PBS into PMG.

The code needs quite a bit of cleanup, and more testing, however I'll send it
as RFC, for an initial sanity-check - to see if I missed some needed
functionality.

It reuses quite a bit of code from pve-storage (PBSPlugin, and the systemd
unit manipulation from the PVE::API2::Disks::Directory) and I'd like to
refactor the common parts into pve-common.

The backup consists of the same files, that end up in a locally generated
PMG Backup (instead of creating a tar.gz we send the data to a PBS instance).

What works:
* creating and manipulating 'remotes' (PBS instances where we back up to)
* creating, restoring and forgetting backups to those remotes
* restoring a backup from another PMG installation from a configured remote
* creating schedules for backups (systemd-timer units) for periodic backups
via CLI (pmgbackup, pmgsh)

What's missing:
* quite a bit of refactoring (e.g. move the systemd-timer helpers and most of
  PBSTools to pve-common (and reuse it in the PBS Storage plugin)
* encryption support
* pruneing support
* testing
* GUI
* Documentation

The first patch is a small cosmetic cleanup, and independent of the series.

A sample session (which I used for testing:
```
pmgbackup remote add pbs1 --datastore local --server 192.0.2.11 --user root@pam --password xxx --fingerprint  ff:ff:...
pmgbackup remote list
pmgbackup pbsjob run pbs1
pmgbackup pbsjob restore pbs1 -althost pmg-live -config 1 -database 1 -statistic 1
pmgbackup pbsjob run pbs1
pmgbackup pbsjob list_backups pbs1
pmgbackup pbsjob create pbs1 --schedule 'minutely' --delay '0s'
systemctl list-timers --all
```

Stoiko Ivanov (12):
  drop left-over commented out code
  Backup: split backup creation and creating tar
  Restore: optionally restore from directory
  Backup: push restore options to PMG::Backup
  debian: add dependency on proxmox-backup-client
  add helper module for handling PBS Integration
  PBSTools: add methods for managing backups
  PBSTools: add systemd-timer helpers
  add initial SectionConfig for pbs
  Add API2 module for PBS configuration
  Add API2 module for per-node backups to PBS
  pbs-integration: add CLI calls to pmgbackup

 debian/control                |   1 +
 debian/dirs                   |   1 +
 debian/pmg-pbsbackup@.service |   6 +
 debian/rules                  |   1 +
 src/Makefile                  |   6 +-
 src/PMG/API2/Backup.pm        |  21 +-
 src/PMG/API2/Config.pm        |   7 +
 src/PMG/API2/Nodes.pm         |   7 +
 src/PMG/API2/PBS/Job.pm       | 467 ++++++++++++++++++++++++++++++++++
 src/PMG/API2/PBS/Remote.pm    | 218 ++++++++++++++++
 src/PMG/Backup.pm             |  86 +++++--
 src/PMG/CLI/pmgbackup.pm      |  27 ++
 src/PMG/PBSConfig.pm          | 168 ++++++++++++
 src/PMG/PBSTools.pm           | 466 +++++++++++++++++++++++++++++++++
 14 files changed, 1447 insertions(+), 35 deletions(-)
 create mode 100644 debian/pmg-pbsbackup@.service
 create mode 100644 src/PMG/API2/PBS/Job.pm
 create mode 100644 src/PMG/API2/PBS/Remote.pm
 create mode 100644 src/PMG/PBSConfig.pm
 create mode 100644 src/PMG/PBSTools.pm

-- 
2.20.1





             reply	other threads:[~2020-10-19 19:02 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-19 19:01 Stoiko Ivanov [this message]
2020-10-19 19:01 ` [pmg-devel] [RFC pmg-api 01/12] drop left-over commented out code Stoiko Ivanov
2020-10-19 19:01 ` [pmg-devel] [RFC pmg-api 02/12] Backup: split backup creation and creating tar Stoiko Ivanov
2020-10-20  5:43   ` Dietmar Maurer
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 03/12] Restore: optionally restore from directory Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 04/12] Backup: push restore options to PMG::Backup Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 05/12] debian: add dependency on proxmox-backup-client Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 06/12] add helper module for handling PBS Integration Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 07/12] PBSTools: add methods for managing backups Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 08/12] PBSTools: add systemd-timer helpers Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 09/12] add initial SectionConfig for pbs Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 10/12] Add API2 module for PBS configuration Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 11/12] Add API2 module for per-node backups to PBS Stoiko Ivanov
2020-10-19 19:02 ` [pmg-devel] [RFC pmg-api 12/12] pbs-integration: add CLI calls to pmgbackup Stoiko Ivanov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201019190209.11495-1-s.ivanov@proxmox.com \
    --to=s.ivanov@proxmox.com \
    --cc=pmg-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal