From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id DBA661FF162
for <inbox@lore.proxmox.com>; Mon, 7 Apr 2025 14:05:33 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id EFB103D53F;
Mon, 7 Apr 2025 14:05:31 +0200 (CEST)
From: Fiona Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Mon, 7 Apr 2025 14:04:39 +0200
Message-Id: <20250407120439.60725-1-f.ebner@proxmox.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.039 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
KAM_SHORT 0.001 Use of a URL Shortener for very short URL
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH pve-storage-examples] initial commit
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Includes two backup provider example plugins: one for a directory
storage and one for Borg.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
Makefile | 47 +
debian/changelog | 6 +
debian/control | 19 +
debian/copyright | 16 +
debian/docs | 1 +
debian/rules | 8 +
debian/triggers | 1 +
src/Makefile | 14 +
src/PVE/BackupProvider/Makefile | 3 +
src/PVE/BackupProvider/Plugin/Borg.pm | 466 ++++++++++
.../BackupProvider/Plugin/DirectoryExample.pm | 809 ++++++++++++++++++
src/PVE/BackupProvider/Plugin/Makefile | 5 +
src/PVE/Makefile | 6 +
.../Custom/BackupProviderDirExamplePlugin.pm | 308 +++++++
src/PVE/Storage/Custom/BorgBackupPlugin.pm | 689 +++++++++++++++
src/PVE/Storage/Custom/Makefile | 6 +
src/PVE/Storage/Makefile | 3 +
17 files changed, 2407 insertions(+)
create mode 100644 Makefile
create mode 100644 debian/changelog
create mode 100644 debian/control
create mode 100644 debian/copyright
create mode 100644 debian/docs
create mode 100755 debian/rules
create mode 100644 debian/triggers
create mode 100644 src/Makefile
create mode 100644 src/PVE/BackupProvider/Makefile
create mode 100644 src/PVE/BackupProvider/Plugin/Borg.pm
create mode 100644 src/PVE/BackupProvider/Plugin/DirectoryExample.pm
create mode 100644 src/PVE/BackupProvider/Plugin/Makefile
create mode 100644 src/PVE/Makefile
create mode 100644 src/PVE/Storage/Custom/BackupProviderDirExamplePlugin.pm
create mode 100644 src/PVE/Storage/Custom/BorgBackupPlugin.pm
create mode 100644 src/PVE/Storage/Custom/Makefile
create mode 100644 src/PVE/Storage/Makefile
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..1846daa
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,47 @@
+include /usr/share/dpkg/default.mk
+
+PACKAGE=libpve-storage-examples-perl
+
+GITVERSION:=$(shell git rev-parse HEAD)
+
+BUILDDIR ?= $(PACKAGE)-$(DEB_VERSION)
+DSC=$(PACKAGE)_$(DEB_VERSION).dsc
+
+DEB=$(PACKAGE)_$(DEB_VERSION_UPSTREAM_REVISION)_all.deb
+
+all: $(DEB)
+
+$(BUILDDIR): debian
+ rm -rf $@ $@.tmp
+ cp -a src $@.tmp
+ cp -a debian/ $@.tmp/
+ echo "git clone git://git.proxmox.com/git/pve-storage-examples.git\\ngit checkout $(GITVERSION)" > $@.tmp/debian/SOURCE
+ mv $@.tmp $@
+
+.PHONY: deb
+deb: $(DEB)
+$(DEB): $(BUILDDIR)
+ cd $(BUILDDIR); dpkg-buildpackage -b -us -uc
+ lintian $(DEB)
+
+.PHONY: dsc
+dsc: $(DSC)
+$(DSC): $(BUILDDIR)
+ cd $(BUILDDIR); dpkg-buildpackage -S -us -uc -d
+ lintian $(DSC)
+
+.PHONY: sbuild
+sbuild: $(DSC)
+ sbuild $(DSC)
+
+.PHONY: upload
+upload: UPLOAD_DIST ?= $(DEB_DISTRIBUTION)
+upload: $(DEB)
+ tar cf - $(DEB)|ssh repoman@repo.proxmox.com -- upload --product pve --dist $(UPLOAD_DIST)
+
+.PHONY: distclean
+distclean: clean
+
+.PHONY: clean
+clean:
+ rm -rf $(PACKAGE)-[0-9]*/ $(PACKAGE)*.tar.* *.deb *.dsc *.changes *.build *.buildinfo
diff --git a/debian/changelog b/debian/changelog
new file mode 100644
index 0000000..941e101
--- /dev/null
+++ b/debian/changelog
@@ -0,0 +1,6 @@
+libpve-storage-examples-perl (0.1.0) bookworm; urgency=medium
+
+ * Initial release. Includes two backup provider plugin examples: one for a
+ directory storage and one for Borg.
+
+ -- Proxmox Support Team <support@proxmox.com> Mon, 07 Apr 2025 13:49:40 +0200
diff --git a/debian/control b/debian/control
new file mode 100644
index 0000000..5f73b8c
--- /dev/null
+++ b/debian/control
@@ -0,0 +1,19 @@
+Source: libpve-storage-examples-perl
+Section: perl
+Priority: optional
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Build-Depends: debhelper-compat (= 13),
+ lintian,
+ perl,
+Standards-Version: 4.6.2
+Homepage: https://www.proxmox.com
+
+Package: libpve-storage-examples-perl
+Architecture: all
+Depends: libpve-storage-perl (>= 8.3.5),
+ ${misc:Depends},
+ ${perl:Depends},
+Recommends: libnbd-bin,
+Description: Example plugins for Proxmox VE storage management library
+ This package contains example plugins for the storage management library used
+ by Proxmox VE.
diff --git a/debian/copyright b/debian/copyright
new file mode 100644
index 0000000..1517c49
--- /dev/null
+++ b/debian/copyright
@@ -0,0 +1,16 @@
+Copyright (C) 2025 Proxmox Server Solutions GmbH
+
+This software is written by Proxmox Server Solutions GmbH <support@proxmox.com>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Affero General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
diff --git a/debian/docs b/debian/docs
new file mode 100644
index 0000000..8696672
--- /dev/null
+++ b/debian/docs
@@ -0,0 +1 @@
+debian/SOURCE
diff --git a/debian/rules b/debian/rules
new file mode 100755
index 0000000..218df65
--- /dev/null
+++ b/debian/rules
@@ -0,0 +1,8 @@
+#!/usr/bin/make -f
+# -*- makefile -*-
+
+# Uncomment this to turn on verbose mode.
+#export DH_VERBOSE=1
+
+%:
+ dh $@
diff --git a/debian/triggers b/debian/triggers
new file mode 100644
index 0000000..59dd688
--- /dev/null
+++ b/debian/triggers
@@ -0,0 +1 @@
+activate-noawait pve-api-updates
diff --git a/src/Makefile b/src/Makefile
new file mode 100644
index 0000000..2bd6667
--- /dev/null
+++ b/src/Makefile
@@ -0,0 +1,14 @@
+DESTDIR=
+PREFIX=/usr
+
+export PERLDIR=$(PREFIX)/share/perl5
+
+all:
+
+.PHONY: install
+install: PVE
+ $(MAKE) -C PVE install
+
+.PHONY: clean
+clean:
+ $(MAKE) -C PVE clean
diff --git a/src/PVE/BackupProvider/Makefile b/src/PVE/BackupProvider/Makefile
new file mode 100644
index 0000000..f018cef
--- /dev/null
+++ b/src/PVE/BackupProvider/Makefile
@@ -0,0 +1,3 @@
+.PHONY: install
+install:
+ make -C Plugin install
diff --git a/src/PVE/BackupProvider/Plugin/Borg.pm b/src/PVE/BackupProvider/Plugin/Borg.pm
new file mode 100644
index 0000000..decc78a
--- /dev/null
+++ b/src/PVE/BackupProvider/Plugin/Borg.pm
@@ -0,0 +1,466 @@
+package PVE::BackupProvider::Plugin::Borg;
+
+use strict;
+use warnings;
+
+use File::chdir;
+use File::Basename qw(basename);
+use File::Path qw(make_path remove_tree);
+use POSIX qw(strftime);
+
+use PVE::Tools;
+
+# ($vmtype, $vmid, $time_string)
+our $ARCHIVE_RE_3 = qr!^pve-(lxc|qemu)-([0-9]+)-([0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z)$!;
+
+sub archive_name {
+ my ($vmtype, $vmid, $backup_time) = @_;
+
+ return "pve-${vmtype}-${vmid}-" . strftime("%FT%TZ", gmtime($backup_time));
+}
+
+# remove_tree can be very verbose by default, do explicit error handling and limit to one message
+my sub _remove_tree {
+ my ($path) = @_;
+
+ remove_tree($path, { error => \my $err });
+ if ($err && @$err) { # empty array if no error
+ for my $diag (@$err) {
+ my ($file, $message) = %$diag;
+ die "cannot remove_tree '$path': $message\n" if $file eq '';
+ die "cannot remove_tree '$path': unlinking $file failed - $message\n";
+ }
+ }
+}
+
+my sub prepare_run_dir {
+ my ($storeid, $archive, $operation, $uid) = @_;
+
+ my $run_dir = "/run/pve-storage/borg-plugin/${storeid}.${archive}.${operation}.$$";
+ _remove_tree($run_dir);
+ make_path($run_dir) or die "unable to create directory $run_dir\n";
+ chmod(0700, $run_dir) or die "unable to chmod directory $run_dir - $!\n";
+ if ($uid) {
+ chown($uid, -1, $run_dir) or die "unable to change owner for $run_dir\n";
+ }
+
+ return $run_dir;
+}
+
+my sub log_info {
+ my ($self, $message) = @_;
+
+ $self->{'log-function'}->('info', $message);
+}
+
+my sub log_warning {
+ my ($self, $message) = @_;
+
+ $self->{'log-function'}->('warn', $message);
+}
+
+my sub log_error {
+ my ($self, $message) = @_;
+
+ $self->{'log-function'}->('err', $message);
+}
+
+my sub file_contents_from_archive {
+ my ($self, $archive, $file) = @_;
+
+ return $self->{'storage-plugin'}->borg_cmd_extract_contents(
+ $self->{scfg},
+ $self->{storeid},
+ $archive,
+ [$file],
+ );
+}
+
+# Plugin implementation
+
+sub new {
+ my ($class, $storage_plugin, $scfg, $storeid, $log_function) = @_;
+
+ my $self = bless {
+ scfg => $scfg,
+ storeid => $storeid,
+ 'storage-plugin' => $storage_plugin,
+ 'log-function' => $log_function,
+ }, $class;
+
+ return $self;
+}
+
+sub provider_name {
+ my ($self) = @_;
+
+ return "Borg";
+}
+
+sub job_init {
+ my ($self, $start_time) = @_;
+
+ $self->{'job-id'} = $start_time;
+ $self->{password} = $self->{'storage-plugin'}->borg_get_password(
+ $self->{scfg}, $self->{storeid});
+ $self->{'ssh-key-fh'} = $self->{'storage-plugin'}->borg_open_ssh_key(
+ $self->{scfg}, $self->{storeid});
+}
+
+sub job_cleanup {
+ my ($self) = @_;
+
+ delete($self->{password});
+ close($self->{'ssh-key-fh'});
+
+ return;
+}
+
+sub backup_init {
+ my ($self, $vmid, $vmtype, $start_time) = @_;
+
+ $self->{$vmid}->{archive} = archive_name($vmtype, $vmid, $start_time);
+
+ return { 'archive-name' => $self->{$vmid}->{archive} };
+}
+
+sub backup_cleanup {
+ my ($self, $vmid, $vmtype, $success, $info) = @_;
+
+ if (defined($vmtype) && $vmtype eq 'lxc') {
+ if (my $run_dir = $self->{$vmid}->{'run-dir'}) {
+ eval {
+ # tmpfs for temporary SSH files gets mounted there in backup_container()
+ eval { PVE::Tools::run_command(['umount', "${run_dir}/ssh"]); };
+ eval { PVE::Tools::run_command(['umount', '-R', "${run_dir}/backup/filesystem"]); };
+ _remove_tree($run_dir);
+ };
+ die "unable to clean up $run_dir - $@" if $@;
+ }
+ }
+ return { stats => { 'archive-size' => 0 } }; # TODO get size
+}
+
+sub backup_container_prepare {
+ my ($self, $vmid, $info) = @_;
+
+ my $archive = $self->{$vmid}->{archive};
+ my $run_dir = prepare_run_dir(
+ $self->{storeid}, $archive, "backup-container", $info->{'backup-user-id'});
+ $self->{$vmid}->{'run-dir'} = $run_dir;
+
+ my $create_dir = sub {
+ my $dir = shift;
+ make_path($dir) or die "unable to create directory $dir\n";
+ chmod(0700, $dir) or die "unable to chmod directory $dir\n";
+ chown($info->{'backup-user-id'}, -1, $dir)
+ or die "unable to change owner for $dir\n";
+ };
+
+ $create_dir->("${run_dir}/backup/");
+ $create_dir->("${run_dir}/backup/filesystem");
+ $create_dir->("${run_dir}/ssh");
+ $create_dir->("${run_dir}/.config");
+ $create_dir->("${run_dir}/.cache");
+
+ for my $subdir ($info->{sources}->@*) {
+ PVE::Tools::run_command([
+ 'mount',
+ '-o', 'bind,ro',
+ "$info->{directory}/${subdir}",
+ "${run_dir}/backup/filesystem/${subdir}",
+ ]);
+ }
+}
+
+sub backup_get_mechanism {
+ my ($self, $vmid, $vmtype) = @_;
+
+ return 'file-handle' if $vmtype eq 'qemu';
+ return 'directory' if $vmtype eq 'lxc';
+
+ die "unsupported VM type '$vmtype'\n";
+}
+
+sub backup_handle_log_file {
+ my ($self, $vmid, $filename) = @_;
+
+ return; # don't upload, Proxmox VE keeps the task log too
+}
+
+sub backup_vm_query_incremental {
+ my ($self, $vmid, $volumes) = @_;
+
+ return; # no support currently
+}
+
+my sub backup_vm_setup_loopdev {
+ my ($file) = @_;
+
+ my $device;
+ my $parser = sub {
+ my $line = shift;
+ if ($line =~ m@^(/dev/loop\d+)$@) {
+ $device = $1;
+ }
+ };
+ my $losetup_cmd = [
+ 'losetup',
+ '--show',
+ '-r',
+ '-f',
+ $file,
+ ];
+ PVE::Tools::run_command($losetup_cmd, outfunc => $parser);
+ return $device;
+}
+
+sub backup_vm {
+ my ($self, $vmid, $guest_config, $volumes, $info) = @_;
+
+ # TODO honor bandwith limit
+ # TODO discard?
+
+ my $archive = $self->{$vmid}->{archive};
+
+ my $run_dir = prepare_run_dir($self->{storeid}, $archive, "backup-vm");
+ my $volume_dir = "${run_dir}/volumes";
+ make_path($volume_dir) or die "unable to create directory $volume_dir\n";
+
+ PVE::Tools::file_set_contents("${run_dir}/guest.config", $guest_config);
+ my $paths = ['./guest.config'];
+
+ if (my $firewall_config = $info->{'firewall-config'}) {
+ PVE::Tools::file_set_contents("${run_dir}/firewall.config", $firewall_config);
+ push $paths->@*, './firewall.config';
+ }
+
+ my @blockdevs = ();
+
+ # TODO --stats for size?
+
+ eval {
+ for my $device_name (sort keys $volumes->%*) {
+ # FIXME there is no option to follow symlinks except in combination with special files,
+ # so loop devices are set up here for this purpose. Newer versions of Borg (since 1.4)
+ # could use the slashdot hack instead:
+ # https://github.com/borgbackup/borg/commit/e7bd18d7f38ddf9e58a4587ae4a2ad8a24d67374
+ my $path = "/proc/$$/fd/" . fileno($volumes->{$device_name}->{'file-handle'});
+ my $blockdev = backup_vm_setup_loopdev($path);
+ push @blockdevs, $blockdev;
+
+ my $link_name = "${volume_dir}/${device_name}.raw";
+ symlink($blockdev, $link_name) or die "could not create symlink $link_name -> $blockdev\n";
+ push $paths->@*, "./volumes/" . basename($link_name, ());
+ }
+
+ local $CWD = $run_dir;
+
+ $self->{'storage-plugin'}->borg_cmd_create(
+ $self->{scfg},
+ $self->{storeid},
+ $self->{$vmid}->{archive},
+ $paths,
+ ['--read-special', '--progress'],
+ );
+ };
+ my $err = $@;
+ for my $blockdev (@blockdevs) {
+ eval { PVE::Tools::run_command(['losetup', '-d', $blockdev]); };
+ log_warning($self, "cannot cleanup loop device - $@") if $@;
+ }
+ eval { _remove_tree($run_dir) };
+ log_warning($self, $@) if $@;
+ die $err if $err;
+}
+
+sub backup_container {
+ my ($self, $vmid, $guest_config, $exclude_patterns, $info) = @_;
+
+ # TODO honor bandwith limit
+
+ my $run_dir = $self->{$vmid}->{'run-dir'};
+ my $backup_dir = "${run_dir}/backup";
+
+ my $archive = $self->{$vmid}->{archive};
+
+ my $ssh_key;
+ if ($self->{'ssh-key-fh'}) {
+ $ssh_key =
+ PVE::Tools::safe_read_from($self->{'ssh-key-fh'}, 1024 * 1024, 0, "SSH key file");
+ }
+
+ my (undef, $ssh_options) =
+ $self->{'storage-plugin'}->borg_setup_ssh_dir($self->{scfg}, "${run_dir}/ssh", $ssh_key);
+
+ PVE::Tools::file_set_contents("${backup_dir}/guest.config", $guest_config);
+ my $paths = ['./guest.config'];
+
+ if (my $firewall_config = $info->{'firewall-config'}) {
+ PVE::Tools::file_set_contents("${backup_dir}/firewall.config", $firewall_config);
+ push $paths->@*, './firewall.config';
+ }
+
+ push $paths->@*, "./filesystem";
+
+ my $opts = ['--numeric-ids', '--sparse', '--progress'];
+
+ for my $pattern ($exclude_patterns->@*) {
+ if ($pattern =~ m|^/|) {
+ push $opts->@*, '-e', "filesystem${pattern}";
+ } else {
+ push $opts->@*, '-e', "filesystem/**${pattern}";
+ }
+ }
+
+ push $opts->@*, '-e', "filesystem/**lost+found" if $info->{'backup-user-id'} != 0;
+
+ # TODO --stats for size?
+
+ # Don't make it local to avoid permission denied error when changing back, because the method is
+ # executed in a user namespace.
+ $CWD = $backup_dir if $info->{'backup-user-id'} != 0;
+ {
+ local $CWD = $backup_dir;
+ local $ENV{BORG_BASE_DIR} = ${run_dir};
+ local $ENV{BORG_PASSPHRASE} = $self->{password};
+
+ local $ENV{BORG_RSH} = "ssh " . join(" ", $ssh_options->@*);
+
+ my $uri = $self->{'storage-plugin'}->borg_repository_uri($self->{scfg}, $self->{storeid});
+ my $archive = $self->{$vmid}->{archive};
+
+ my $cmd = ['borg', 'create', $opts->@*, "${uri}::${archive}", $paths->@*];
+
+ PVE::Tools::run_command($cmd, errmsg => "command @$cmd failed");
+ }
+}
+
+sub restore_get_mechanism {
+ my ($self, $volname) = @_;
+
+ my (undef, $archive) = $self->{'storage-plugin'}->parse_volname($volname);
+ my ($vmtype) = $archive =~ m!^pve-([^\s-]+)!
+ or die "cannot parse guest type from archive name '$archive'\n";
+
+ return ('qemu-img', $vmtype) if $vmtype eq 'qemu';
+ return ('directory', $vmtype) if $vmtype eq 'lxc';
+
+ die "unexpected guest type '$vmtype'\n";
+}
+
+sub archive_get_guest_config {
+ my ($self, $volname) = @_;
+
+ my (undef, $archive) = $self->{'storage-plugin'}->parse_volname($volname);
+ return file_contents_from_archive($self, $archive, 'guest.config');
+}
+
+sub archive_get_firewall_config {
+ my ($self, $volname) = @_;
+
+ my (undef, $archive) = $self->{'storage-plugin'}->parse_volname($volname);
+ my $config = eval {
+ file_contents_from_archive($self, $archive, 'firewall.config');
+ };
+ if (my $err = $@) {
+ return if $err =~ m!Include pattern 'firewall\.config' never matched\.!;
+ die $err;
+ }
+ return $config;
+}
+
+sub restore_vm_init {
+ my ($self, $volname) = @_;
+
+ my $res = {};
+
+ my (undef, $archive, $vmid) = $self->{'storage-plugin'}->parse_volname($volname);
+
+ my $run_dir = prepare_run_dir($self->{storeid}, $archive, "restore-vm");
+ $self->{$volname}->{'run-dir'} = $run_dir;
+
+ my $mount_point = "${run_dir}/mount";
+ make_path($mount_point) or die "unable to create directory $mount_point\n";
+ $self->{$volname}->{'mount-point'} = $mount_point;
+
+ $self->{'storage-plugin'}->borg_cmd_mount(
+ $self->{scfg},
+ $self->{storeid},
+ $archive,
+ $mount_point,
+ );
+
+ my @backup_files = glob("$mount_point/volumes/*");
+ for my $backup_file (@backup_files) {
+ next if $backup_file !~ m!^(.*/(.*)\.raw)$!; # untaint
+ ($backup_file, my $device_name) = ($1, $2);
+ # TODO avoid dependency on base plugin?
+ $res->{$device_name}->{size} =
+ PVE::Storage::Plugin::file_size_info($backup_file, undef, 'raw');
+ }
+
+ return $res;
+}
+
+sub restore_vm_cleanup {
+ my ($self, $volname) = @_;
+
+ my $run_dir = $self->{$volname}->{'run-dir'} or return;
+ my $mount_point = $self->{$volname}->{'mount-point'};
+
+ eval { PVE::Tools::run_command(['umount', $mount_point]) };
+ eval { _remove_tree($run_dir); };
+ die "unable to clean up $run_dir - $@" if $@;
+
+ return;
+}
+
+sub restore_vm_volume_init {
+ my ($self, $volname, $device_name, $info) = @_;
+
+ my $mount_point = $self->{$volname}->{'mount-point'}
+ or die "expected mount point for archive not present\n";
+
+ return { 'qemu-img-path' => "${mount_point}/volumes/${device_name}.raw" };
+}
+
+sub restore_vm_volume_cleanup {
+ my ($self, $volname, $device_name, $info) = @_;
+
+ return;
+}
+
+sub restore_container_init {
+ my ($self, $volname, $info) = @_;
+
+ my (undef, $archive, $vmid) = $self->{'storage-plugin'}->parse_volname($volname);
+ my $run_dir = prepare_run_dir($self->{storeid}, $archive, "restore-container");
+ $self->{$volname}->{'run-dir'} = $run_dir;
+
+ my $mount_point = "${run_dir}/mount";
+ make_path($mount_point) or die "unable to create directory $mount_point\n";
+ $self->{$volname}->{'mount-point'} = $mount_point;
+
+ $self->{'storage-plugin'}->borg_cmd_mount(
+ $self->{scfg},
+ $self->{storeid},
+ $archive,
+ $mount_point,
+ );
+
+ return { 'archive-directory' => "${mount_point}/filesystem" };
+}
+
+sub restore_container_cleanup {
+ my ($self, $volname, $info) = @_;
+
+ my $run_dir = $self->{$volname}->{'run-dir'} or return;
+ my $mount_point = $self->{$volname}->{'mount-point'};
+
+ eval { PVE::Tools::run_command(['umount', $mount_point]) };
+ eval { _remove_tree($run_dir); };
+ die "unable to clean up $run_dir - $@" if $@;
+}
+
+1;
diff --git a/src/PVE/BackupProvider/Plugin/DirectoryExample.pm b/src/PVE/BackupProvider/Plugin/DirectoryExample.pm
new file mode 100644
index 0000000..f375f4f
--- /dev/null
+++ b/src/PVE/BackupProvider/Plugin/DirectoryExample.pm
@@ -0,0 +1,809 @@
+package PVE::BackupProvider::Plugin::DirectoryExample;
+
+use strict;
+use warnings;
+
+use Fcntl qw(SEEK_SET);
+use File::Path qw(make_path remove_tree);
+use IO::File;
+use IPC::Open3;
+use JSON qw(from_json to_json);
+
+use PVE::Storage::Common;
+use PVE::Storage::Plugin;
+use PVE::Tools qw(file_get_contents file_read_firstline file_set_contents run_command);
+
+use base qw(PVE::BackupProvider::Plugin::Base);
+
+# Private helpers
+
+my sub log_info {
+ my ($self, $message) = @_;
+
+ $self->{'log-function'}->('info', $message);
+}
+
+my sub log_warning {
+ my ($self, $message) = @_;
+
+ $self->{'log-function'}->('warn', $message);
+}
+
+my sub log_error {
+ my ($self, $message) = @_;
+
+ $self->{'log-function'}->('err', $message);
+}
+
+# NOTE: This is just for proof-of-concept testing! A backup provider plugin should either use the
+# 'nbd' backup mechansim and use the NBD protocol or use the 'file-handle' mechanism. There should
+# be no need to use /dev/nbdX nodes for proper plugins.
+my sub bind_next_free_dev_nbd_node {
+ my ($options) = @_;
+
+ # /dev/nbdX devices are reserved in a file. Those reservations expires after $expiretime.
+ # This avoids race conditions between allocation and use.
+
+ die "file '/sys/module/nbd' does not exist - 'nbd' kernel module not loaded?"
+ if !-e "/sys/module/nbd";
+
+ my $line = PVE::Tools::file_read_firstline("/sys/module/nbd/parameters/nbds_max")
+ or die "could not read 'nbds_max' parameter file for 'nbd' kernel module\n";
+ my ($nbds_max) = ($line =~ m/(\d+)/)
+ or die "could not determine 'nbds_max' parameter for 'nbd' kernel module\n";
+
+ my $filename = "/run/qemu-server/reserved-dev-nbd-nodes";
+
+ my $code = sub {
+ my $expiretime = 60;
+ my $ctime = time();
+
+ my $used = {};
+ my $latest = [0, 0];
+
+ if (my $fh = IO::File->new ($filename, "r")) {
+ while (my $line = <$fh>) {
+ if ($line =~ m/^(\d+)\s(\d+)$/) {
+ my ($n, $timestamp) = ($1, $2);
+
+ $latest = [$n, $timestamp] if $latest->[1] <= $timestamp;
+
+ if (($timestamp + $expiretime) > $ctime) {
+ $used->{$n} = $timestamp; # not expired
+ }
+ }
+ }
+ }
+
+ my $new_n;
+ for (my $count = 0; $count < $nbds_max; $count++) {
+ my $n = ($latest->[0] + $count) % $nbds_max;
+ my $block_device = "/dev/nbd${n}";
+ next if $used->{$n}; # reserved
+ next if !-e $block_device;
+
+ my $st = File::stat::stat("/run/lock/qemu-nbd-nbd${n}");
+ next if defined($st) && S_ISSOCK($st->mode) && $st->uid == 0; # in use
+
+ # Used to avoid looping if there are other issues then the NBD node being in use
+ my $socket_error = 0;
+ eval {
+ my $errfunc = sub {
+ my ($line) = @_;
+ $socket_error = 1 if $line =~ m/^qemu-nbd: Failed to set NBD socket$/;
+ log_warn($line);
+ };
+ run_command(["qemu-nbd", "-c", $block_device, $options->@*], errfunc => $errfunc);
+ };
+ if (my $err = $@) {
+ die $err if !$socket_error;
+ log_warn("unable to bind $block_device - trying next one");
+ next;
+ }
+ $used->{$n} = $ctime;
+ $new_n = $n;
+ last;
+ }
+
+ my $data = "";
+ $data .= "$_ $used->{$_}\n" for keys $used->%*;
+
+ PVE::Tools::file_set_contents($filename, $data);
+
+ return defined($new_n) ? "/dev/nbd${new_n}" : undef;
+ };
+
+ my $block_device =
+ PVE::Tools::lock_file('/run/lock/qemu-server/reserved-dev-nbd-nodes.lock', 10, $code);
+ die $@ if $@;
+
+ die "unable to find free /dev/nbdX block device node\n" if !$block_device;
+
+ return $block_device;
+}
+
+# Backup Provider API
+
+sub new {
+ my ($class, $storage_plugin, $scfg, $storeid, $log_function) = @_;
+
+ my $self = bless {
+ scfg => $scfg,
+ storeid => $storeid,
+ 'storage-plugin' => $storage_plugin,
+ 'log-function' => $log_function,
+ }, $class;
+
+ return $self;
+}
+
+sub provider_name {
+ my ($self) = @_;
+
+ return 'dir provider example';
+}
+
+# Hooks
+
+sub job_init {
+ my ($self, $start_time) = @_;
+
+ log_info($self, "job init called");
+
+ if (!-e '/sys/module/nbd/parameters') {
+ die "required 'nbd' kernel module not loaded - use 'modprobe nbd nbds_max=128' to load it"
+ ." manually\n";
+ }
+
+ log_info($self, "backup provider initialized successfully for new job $start_time");
+
+ return;
+}
+
+sub job_cleanup {
+ my ($self) = @_;
+
+ log_info($self, "job cleanup called");
+
+ return;
+}
+
+sub backup_init {
+ my ($self, $vmid, $vmtype, $backup_time) = @_;
+
+ my $archive_subdir = "${vmtype}-${backup_time}";
+ my $archive = "${vmid}/${archive_subdir}";
+
+ log_info($self, "backup start hook called");
+
+ my $backup_dir = $self->{scfg}->{path} . "/" . $archive;
+
+ make_path($backup_dir);
+ die "unable to create directory $backup_dir\n" if !-d $backup_dir;
+
+ $self->{$vmid}->{'backup-time'} = $backup_time;
+ $self->{$vmid}->{'backup-dir'} = $backup_dir;
+
+ $self->{$vmid}->{'archive-subdir'} = $archive_subdir;
+ $self->{$vmid}->{archive} = $archive;
+ return { 'archive-name' => $archive };
+}
+
+my sub get_previous_info_tainted {
+ my ($self, $vmid) = @_;
+
+ my $previous_info_file = "$self->{scfg}->{path}/$vmid/previous-info";
+
+ return eval { from_json(file_get_contents($previous_info_file)) } // {};
+}
+
+my sub update_previous_info {
+ my ($self, $vmid) = @_;
+
+ my $previous_info_file = "$self->{scfg}->{path}/$vmid/previous-info";
+
+ if (defined(my $info = $self->{$vmid}->{previous})) {
+ file_set_contents($previous_info_file, to_json($info));
+ } else {
+ unlink($previous_info_file);
+ }
+}
+
+
+sub backup_cleanup {
+ my ($self, $vmid, $vmtype, $success, $info) = @_;
+
+ if ($success) {
+ log_info($self, "backup cleanup called - success");
+ eval {
+ update_previous_info($self, $vmid, $self->{$vmid}->{previous});
+ };
+ if (my $err = $@) {
+ log_error($self, "failed to update previous-info file: $err");
+ }
+ my $size = 0;
+ my $backup_dir = $self->{$vmid}->{'backup-dir'};
+ my @backup_files = glob("$backup_dir/*");
+ $size += -s $_ for @backup_files;
+ my $stats = { 'archive-size' => $size };
+ return { 'stats' => $stats };
+ } else {
+ log_info($self, "backup cleanup called - failure");
+
+ $self->{$vmid}->{failed} = 1;
+
+ if (my $dir = $self->{$vmid}->{'backup-dir'}) {
+ eval { remove_tree($dir) };
+ log_warning($self, "unable to clean up $dir - $@") if $@;
+ }
+
+ # Restore old previous-info so next attempt can re-use bitmap again
+ if (my $info = $self->{$vmid}->{'old-previous-info'}) {
+ my $previous_info_dir = "$self->{scfg}->{path}/$vmid/";
+ my $previous_info_file = "$previous_info_dir/previous-info";
+ file_set_contents($previous_info_file, $info);
+ }
+ }
+}
+
+sub backup_container_prepare {
+ my ($self, $vmid, $info) = @_;
+
+ my $dir = $self->{$vmid}->{'backup-dir'};
+ chown($info->{'backup-user-id'}, -1, $dir) or die "unable to change owner for $dir\n";
+
+ return;
+}
+
+sub backup_vm_query_incremental {
+ my ($self, $vmid, $volumes) = @_;
+
+ # Try to use the last backup's disks for incremental backup if the storage
+ # is configured for incremental VM backup. Need to start fresh if there is
+ # no previous backup or the associated backup doesn't exist.
+
+ return if $self->{'storage-plugin'}->get_vm_backup_mode($self->{scfg}) ne 'incremental';
+
+ my $vmtype = 'qemu';
+
+ my $out = {};
+
+ my $info = get_previous_info_tainted($self, $vmid);
+ for my $device_name (keys $volumes->%*) {
+ $out->{$device_name} = 'new';
+
+ my $prev_file = $info->{$device_name};
+ next if !defined $prev_file;
+ # it's type-time/disk.qcow2
+ next if $prev_file !~ m!^([^/]+/[^/]+\.qcow2)$!;
+ $prev_file = $1; # untaint
+
+ my $full_path = "$self->{scfg}->{path}/$vmid/$prev_file";
+
+ if (-e $full_path) {
+ $self->{$vmid}->{previous}->{$device_name} = $prev_file;
+ $out->{$device_name} = 'use';
+ }
+ }
+
+ return $out;
+}
+
+sub backup_get_mechanism {
+ my ($self, $vmid, $vmtype) = @_;
+
+ return 'directory' if $vmtype eq 'lxc';
+ return $self->{'storage-plugin'}->get_vm_backup_mechanism($self->{scfg}) if $vmtype eq 'qemu';
+
+ die "unsupported guest type '$vmtype'\n";
+}
+
+sub backup_handle_log_file {
+ my ($self, $vmid, $filename) = @_;
+
+ my $log_dir = $self->{$vmid}->{'backup-dir'};
+ if ($self->{$vmid}->{failed}) {
+ $log_dir .= ".failed";
+ }
+ make_path($log_dir);
+ die "unable to create directory $log_dir\n" if !-d $log_dir;
+
+ my $data = file_get_contents($filename);
+ my $target = "${log_dir}/backup.log";
+ file_set_contents($target, $data);
+}
+
+my sub backup_file {
+ my ($self, $vmid, $device_name, $size, $in_fh, $bitmap_mode, $next_dirty_region, $bandwidth_limit) = @_;
+
+ # TODO honor bandwidth_limit
+
+ my $target = "$self->{$vmid}->{'backup-dir'}/${device_name}.qcow2";
+
+ my $create_cmd = ["qemu-img", "create", "-f", "qcow2", $target, $size];
+ if (my $previous_file = $self->{$vmid}->{previous}->{$device_name}) {
+ my $target_base = "../$previous_file";
+ push $create_cmd->@*, "-b", $target_base, "-F", "qcow2";
+ }
+ run_command($create_cmd);
+
+ my $nbd_node;
+ eval {
+ # allows to easily write to qcow2 target
+ $nbd_node = bind_next_free_dev_nbd_node([$target, '--format=qcow2']);
+ # FIXME use nbdfuse like in qemu-server rather than qemu-nbd. Seems like there is a race and
+ # sysseek() can fail with "Invalid argument" if done too early...
+ sleep 1;
+
+ my $block_size = 4 * 1024 * 1024; # 4 MiB
+
+ my $out_fh = IO::File->new($nbd_node, "r+")
+ or die "unable to open NBD backup target - $!\n";
+
+ my $buffer = '';
+ my $skip_discard;
+
+ while (scalar((my $region_offset, my $region_length) = $next_dirty_region->())) {
+ sysseek($in_fh, $region_offset, SEEK_SET)
+ // die "unable to seek '$region_offset' in NBD backup source - $!\n";
+ sysseek($out_fh, $region_offset, SEEK_SET)
+ // die "unable to seek '$region_offset' in NBD backup target - $!\n";
+
+ my $local_offset = 0; # within the region
+ while ($local_offset < $region_length) {
+ my $remaining = $region_length - $local_offset;
+ my $request_size = $remaining < $block_size ? $remaining : $block_size;
+ my $offset = $region_offset + $local_offset;
+
+ my $read = sysread($in_fh, $buffer, $request_size);
+ die "failed to read from backup source - $!\n" if !defined($read);
+ die "premature EOF while reading backup source\n" if $read == 0;
+
+ my $written = 0;
+ while ($written < $read) {
+ my $res = syswrite($out_fh, $buffer, $request_size - $written, $written);
+ die "failed to write to backup target - $!\n" if !defined($res);
+ die "unable to progress writing to backup target\n" if $res == 0;
+ $written += $res;
+ }
+
+ if (!$skip_discard) {
+ eval { PVE::Storage::Common::deallocate($in_fh, $offset, $request_size); };
+ if (my $err = $@) {
+ # Just assume that if one request didn't work, others won't either.
+ log_warning(
+ $self, "discard source failed (skipping further discards) - $err");
+ $skip_discard = 1;
+ }
+ }
+
+ $local_offset += $request_size;
+ }
+ }
+ $out_fh->sync();
+ };
+ my $err = $@;
+
+ $self->{$vmid}->{previous}->{$device_name} = "$self->{$vmid}->{'archive-subdir'}/${device_name}.qcow2";
+
+ eval { run_command(['qemu-nbd', '-d', $nbd_node ]); };
+ log_warning($self, "unable to disconnect NBD backup target - $@") if $@;
+
+ die $err if $err;
+}
+
+my sub backup_nbd {
+ my ($self, $vmid, $device_name, $size, $nbd_path, $bitmap_mode, $bitmap_name, $bandwidth_limit) = @_;
+
+ # TODO honor bandwidth_limit
+
+ die "need 'nbdinfo' binary from package libnbd-bin\n" if !-e "/usr/bin/nbdinfo";
+
+ my $nbd_info_uri = "nbd+unix:///${device_name}?socket=${nbd_path}";
+ my $qemu_nbd_uri = "nbd:unix:${nbd_path}:exportname=${device_name}";
+
+ my $cpid;
+ my $error_fh;
+ my $next_dirty_region;
+
+ # If there is no dirty bitmap, it can be treated as if there's a full dirty one. The output of
+ # nbdinfo is a list of tuples with offset, length, type, description. The first bit of 'type' is
+ # set when the bitmap is dirty, see QEMU's docs/interop/nbd.txt
+ my $dirty_bitmap = [];
+ if ($bitmap_mode ne 'none') {
+ my $input = IO::File->new();
+ my $info = IO::File->new();
+ $error_fh = IO::File->new();
+ my $nbdinfo_cmd = ["nbdinfo", $nbd_info_uri, "--map=qemu:dirty-bitmap:${bitmap_name}"];
+ $cpid = open3($input, $info, $error_fh, $nbdinfo_cmd->@*)
+ or die "failed to spawn nbdinfo child - $!\n";
+
+ $next_dirty_region = sub {
+ my ($offset, $length, $type);
+ do {
+ my $line = <$info>;
+ return if !$line;
+ die "unexpected output from nbdinfo - $line\n"
+ if $line !~ m/^\s*(\d+)\s*(\d+)\s*(\d+)/; # also untaints
+ ($offset, $length, $type) = ($1, $2, $3);
+ } while (($type & 0x1) == 0); # not dirty
+ return ($offset, $length);
+ };
+ } else {
+ my $done = 0;
+ $next_dirty_region = sub {
+ return if $done;
+ $done = 1;
+ return (0, $size);
+ };
+ }
+
+ my $nbd_node;
+ eval {
+ $nbd_node = bind_next_free_dev_nbd_node([$qemu_nbd_uri, "--format=raw", "--discard=on"]);
+
+ my $in_fh = IO::File->new($nbd_node, 'r+')
+ or die "unable to open NBD backup source '$nbd_node' - $!\n";
+
+ backup_file(
+ $self,
+ $vmid,
+ $device_name,
+ $size,
+ $in_fh,
+ $bitmap_mode,
+ $next_dirty_region,
+ $bandwidth_limit,
+ );
+ };
+ my $err = $@;
+
+ eval { run_command(["qemu-nbd", "-d", $nbd_node ]); };
+ log_warning($self, "unable to disconnect NBD backup source - $@") if $@;
+
+ if ($cpid) {
+ my $waited;
+ my $wait_limit = 5;
+ for ($waited = 0; $waited < $wait_limit && waitpid($cpid, POSIX::WNOHANG) == 0; $waited++) {
+ kill 15, $cpid if $waited == 0;
+ sleep 1;
+ }
+ if ($waited == $wait_limit) {
+ kill 9, $cpid;
+ sleep 1;
+ log_warning($self, "unable to collect nbdinfo child process")
+ if waitpid($cpid, POSIX::WNOHANG) == 0;
+ }
+ }
+
+ die $err if $err;
+}
+
+my sub backup_vm_volume {
+ my ($self, $vmid, $device_name, $info, $bandwidth_limit) = @_;
+
+ my $backup_mechanism = $self->{'storage-plugin'}->get_vm_backup_mechanism($self->{scfg});
+
+ if ($backup_mechanism eq 'nbd') {
+ backup_nbd(
+ $self,
+ $vmid,
+ $device_name,
+ $info->{size},
+ $info->{'nbd-path'},
+ $info->{'bitmap-mode'},
+ $info->{'bitmap-name'},
+ $bandwidth_limit,
+ );
+ } elsif ($backup_mechanism eq 'file-handle') {
+ backup_file(
+ $self,
+ $vmid,
+ $device_name,
+ $info->{size},
+ $info->{'file-handle'},
+ $info->{'bitmap-mode'},
+ $info->{'next-dirty-region'},
+ $bandwidth_limit,
+ );
+ } else {
+ die "internal error - unknown VM backup mechansim '$backup_mechanism'\n";
+ }
+}
+
+sub backup_vm {
+ my ($self, $vmid, $guest_config, $volumes, $info) = @_;
+
+ my $target = "$self->{$vmid}->{'backup-dir'}/guest.conf";
+ file_set_contents($target, $guest_config);
+
+ if (my $firewall_config = $info->{'firewall-config'}) {
+ $target = "$self->{$vmid}->{'backup-dir'}/firewall.conf";
+ file_set_contents($target, $firewall_config);
+ }
+
+ for my $device_name (sort keys $volumes->%*) {
+ backup_vm_volume(
+ $self, $vmid, $device_name, $volumes->{$device_name}, $info->{'bandwidth-limit'});
+ }
+}
+
+my sub backup_directory_tar {
+ my ($self, $vmid, $directory, $exclude_patterns, $sources, $bandwidth_limit) = @_;
+
+ # essentially copied from PVE/VZDump/LXC.pm' archive()
+
+ # copied from PVE::Storage::Plugin::COMMON_TAR_FLAGS
+ my @tar_flags = qw(
+ --one-file-system
+ -p --sparse --numeric-owner --acls
+ --xattrs --xattrs-include=user.* --xattrs-include=security.capability
+ --warning=no-file-ignored --warning=no-xattr-write
+ );
+
+ my $tar = ['tar', 'cpf', '-', '--totals', @tar_flags];
+
+ push @$tar, "--directory=$directory";
+
+ my @exclude_no_anchored = ();
+ my @exclude_anchored = ();
+ for my $pattern ($exclude_patterns->@*) {
+ if ($pattern !~ m|^/|) {
+ push @exclude_no_anchored, $pattern;
+ } else {
+ push @exclude_anchored, $pattern;
+ }
+ }
+
+ push @$tar, '--no-anchored';
+ push @$tar, '--exclude=lost+found';
+ push @$tar, map { "--exclude=$_" } @exclude_no_anchored;
+
+ push @$tar, '--anchored';
+ push @$tar, map { "--exclude=.$_" } @exclude_anchored;
+
+ push @$tar, $sources->@*;
+
+ my $cmd = [ $tar ];
+
+ push @$cmd, [ 'cstream', '-t', $bandwidth_limit * 1024 ] if $bandwidth_limit;
+
+ my $target = "$self->{$vmid}->{'backup-dir'}/archive.tar";
+ push @{$cmd->[-1]}, \(">" . PVE::Tools::shellquote($target));
+
+ my $logfunc = sub {
+ my $line = shift;
+ log_info($self, "tar: $line");
+ };
+
+ PVE::Tools::run_command($cmd, logfunc => $logfunc);
+
+ return;
+};
+
+# NOTE This only serves as an example to illustrate the 'directory' restore mechanism. It is not
+# fleshed out properly, e.g. I didn't check if exclusion is compatible with
+# proxmox-backup-client/rsync or xattrs/ACL/etc. work as expected!
+my sub backup_directory_squashfs {
+ my ($self, $vmid, $directory, $exclude_patterns, $bandwidth_limit) = @_;
+
+ my $target = "$self->{$vmid}->{'backup-dir'}/archive.sqfs";
+
+ my $mksquashfs = ['mksquashfs', $directory, $target, '-quiet', '-no-progress'];
+
+ push $mksquashfs->@*, '-wildcards';
+
+ for my $pattern ($exclude_patterns->@*) {
+ if ($pattern !~ m|^/|) { # non-anchored
+ push $mksquashfs->@*, '-e', "... $pattern";
+ } else { # anchored
+ push $mksquashfs->@*, '-e', substr($pattern, 1); # need to strip leading slash
+ }
+ }
+
+ my $cmd = [ $mksquashfs ];
+
+ push @$cmd, [ 'cstream', '-t', $bandwidth_limit * 1024 ] if $bandwidth_limit;
+
+ my $logfunc = sub {
+ my $line = shift;
+ log_info($self, "mksquashfs: $line");
+ };
+
+ PVE::Tools::run_command($cmd, logfunc => $logfunc);
+
+ return;
+};
+
+sub backup_container {
+ my ($self, $vmid, $guest_config, $exclude_patterns, $info) = @_;
+
+ my $target = "$self->{$vmid}->{'backup-dir'}/guest.conf";
+ file_set_contents($target, $guest_config);
+
+ if (my $firewall_config = $info->{'firewall-config'}) {
+ $target = "$self->{$vmid}->{'backup-dir'}/firewall.conf";
+ file_set_contents($target, $firewall_config);
+ }
+
+ my $backup_mode = $self->{'storage-plugin'}->get_lxc_backup_mode($self->{scfg});
+ if ($backup_mode eq 'tar') {
+ backup_directory_tar(
+ $self,
+ $vmid,
+ $info->{directory},
+ $exclude_patterns,
+ $info->{sources},
+ $info->{'bandwidth-limit'},
+ );
+ } elsif ($backup_mode eq 'squashfs') {
+ backup_directory_squashfs(
+ $self,
+ $vmid,
+ $info->{directory},
+ $exclude_patterns,
+ $info->{'bandwidth-limit'},
+ );
+ } else {
+ die "got unexpected backup mode '$backup_mode' from storage plugin\n";
+ }
+}
+
+# Restore API
+
+sub restore_get_mechanism {
+ my ($self, $volname) = @_;
+
+ my (undef, $relative_backup_dir) = $self->{'storage-plugin'}->parse_volname($volname);
+ my ($vmtype) = $relative_backup_dir =~ m!^\d+/([a-z]+)-!;
+
+ return ('qemu-img', $vmtype) if $vmtype eq 'qemu';
+
+ if ($vmtype eq 'lxc') {
+ my (undef, $relative_backup_dir) = $self->{'storage-plugin'}->parse_volname($volname);
+
+ if (-e "$self->{scfg}->{path}/${relative_backup_dir}/archive.tar") {
+ $self->{'restore-mechanisms'}->{$volname} = 'tar';
+ return ('tar', $vmtype);
+ }
+
+ if (-e "$self->{scfg}->{path}/${relative_backup_dir}/archive.sqfs") {
+ $self->{'restore-mechanisms'}->{$volname} = 'directory';
+ return ('directory', $vmtype)
+ }
+
+ die "unable to find archive '$volname'\n";
+ }
+
+ die "cannot restore unexpected guest type '$vmtype'\n";
+}
+
+sub archive_get_guest_config {
+ my ($self, $volname) = @_;
+
+ my (undef, $relative_backup_dir) = $self->{'storage-plugin'}->parse_volname($volname);
+ my $filename = "$self->{scfg}->{path}/${relative_backup_dir}/guest.conf";
+
+ return file_get_contents($filename);
+}
+
+sub archive_get_firewall_config {
+ my ($self, $volname) = @_;
+
+ my (undef, $relative_backup_dir) = $self->{'storage-plugin'}->parse_volname($volname);
+ my $filename = "$self->{scfg}->{path}/${relative_backup_dir}/firewall.conf";
+
+ return if !-e $filename;
+
+ return file_get_contents($filename);
+}
+
+sub restore_vm_init {
+ my ($self, $volname) = @_;
+
+ my $res = {};
+
+ my (undef, $relative_backup_dir) = $self->{'storage-plugin'}->parse_volname($volname);
+ my $backup_dir = "$self->{scfg}->{path}/${relative_backup_dir}";
+
+ my @backup_files = glob("$backup_dir/*");
+ for my $backup_file (@backup_files) {
+ next if $backup_file !~ m!^(.*/(.*)\.qcow2)$!;
+ $backup_file = $1; # untaint
+ $res->{$2}->{size} = PVE::Storage::Plugin::file_size_info($backup_file, undef, 'qcow2');
+ }
+
+ return $res;
+}
+
+sub restore_vm_cleanup {
+ my ($self, $volname) = @_;
+
+ return; # nothing to do
+}
+
+sub restore_vm_volume_init {
+ my ($self, $volname, $device_name, $info) = @_;
+
+ my (undef, $relative_backup_dir) = $self->{'storage-plugin'}->parse_volname($volname);
+ my $image = "$self->{scfg}->{path}/${relative_backup_dir}/${device_name}.qcow2";
+ # NOTE Backing files are not allowed by Proxmox VE when restoring. The reason is that an
+ # untrusted qcow2 image can specify an arbitrary backing file and thus leak data from the host.
+ # For the sake of the directory example plugin, an NBD export is created, but this side-steps
+ # the check and would allow the attack again. An actual implementation should check that the
+ # backing file (or rather, the whole backing chain) is safe first!
+ my $nbd_node = bind_next_free_dev_nbd_node([$image]);
+ $self->{"${volname}/${device_name}"}->{'nbd-node'} = $nbd_node;
+ return {
+ 'qemu-img-path' => $nbd_node,
+ };
+}
+
+sub restore_vm_volume_cleanup {
+ my ($self, $volname, $device_name, $info) = @_;
+
+ if (my $nbd_node = delete($self->{"${volname}/${device_name}"}->{'nbd-node'})) {
+ PVE::Tools::run_command(['qemu-nbd', '-d', $nbd_node]);
+ }
+
+ return;
+}
+
+my sub restore_tar_init {
+ my ($self, $volname) = @_;
+
+ my (undef, $relative_backup_dir) = $self->{'storage-plugin'}->parse_volname($volname);
+ return { 'tar-path' => "$self->{scfg}->{path}/${relative_backup_dir}/archive.tar" };
+}
+
+my sub restore_directory_init {
+ my ($self, $volname) = @_;
+
+ my (undef, $relative_backup_dir, $vmid) = $self->{'storage-plugin'}->parse_volname($volname);
+ my $archive = "$self->{scfg}->{path}/${relative_backup_dir}/archive.sqfs";
+
+ my $mount_point = "/run/backup-provider-example/${vmid}.mount";
+ make_path($mount_point);
+ die "unable to create directory $mount_point\n" if !-d $mount_point;
+
+ run_command(['mount', '-o', 'ro', $archive, $mount_point]);
+
+ return { 'archive-directory' => $mount_point };
+}
+
+my sub restore_directory_cleanup {
+ my ($self, $volname) = @_;
+
+ my (undef, undef, $vmid) = $self->{'storage-plugin'}->parse_volname($volname);
+ my $mount_point = "/run/backup-provider-example/${vmid}.mount";
+
+ run_command(['umount', $mount_point]);
+
+ return;
+}
+
+sub restore_container_init {
+ my ($self, $volname, $info) = @_;
+
+ if ($self->{'restore-mechanisms'}->{$volname} eq 'tar') {
+ return restore_tar_init($self, $volname);
+ } elsif ($self->{'restore-mechanisms'}->{$volname} eq 'directory') {
+ return restore_directory_init($self, $volname);
+ } else {
+ die "no restore mechanism set for '$volname'\n";
+ }
+}
+
+sub restore_container_cleanup {
+ my ($self, $volname, $info) = @_;
+
+ if ($self->{'restore-mechanisms'}->{$volname} eq 'tar') {
+ return; # nothing to do
+ } elsif ($self->{'restore-mechanisms'}->{$volname} eq 'directory') {
+ return restore_directory_cleanup($self, $volname);
+ } else {
+ die "no restore mechanism set for '$volname'\n";
+ }
+}
+
+1;
diff --git a/src/PVE/BackupProvider/Plugin/Makefile b/src/PVE/BackupProvider/Plugin/Makefile
new file mode 100644
index 0000000..936885e
--- /dev/null
+++ b/src/PVE/BackupProvider/Plugin/Makefile
@@ -0,0 +1,5 @@
+SOURCES = Borg.pm DirectoryExample.pm
+
+.PHONY: install
+install:
+ for i in ${SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/BackupProvider/Plugin/$$i; done
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
new file mode 100644
index 0000000..4ea12dd
--- /dev/null
+++ b/src/PVE/Makefile
@@ -0,0 +1,6 @@
+.PHONY: install
+install:
+ make -C Storage install
+ make -C BackupProvider install
+
+clean:
diff --git a/src/PVE/Storage/Custom/BackupProviderDirExamplePlugin.pm b/src/PVE/Storage/Custom/BackupProviderDirExamplePlugin.pm
new file mode 100644
index 0000000..a57e126
--- /dev/null
+++ b/src/PVE/Storage/Custom/BackupProviderDirExamplePlugin.pm
@@ -0,0 +1,308 @@
+package PVE::Storage::Custom::BackupProviderDirExamplePlugin;
+
+use strict;
+use warnings;
+
+use File::Basename qw(basename);
+
+use PVE::BackupProvider::Plugin::DirectoryExample;
+use PVE::Tools;
+
+use base qw(PVE::Storage::Plugin);
+
+# Helpers
+
+sub get_vm_backup_mechanism {
+ my ($class, $scfg) = @_;
+
+ return $scfg->{'vm-backup-mechanism'} // properties()->{'vm-backup-mechanism'}->{'default'};
+}
+
+sub get_vm_backup_mode {
+ my ($class, $scfg) = @_;
+
+ return $scfg->{'vm-backup-mode'} // properties()->{'vm-backup-mode'}->{'default'};
+}
+
+sub get_lxc_backup_mode {
+ my ($class, $scfg) = @_;
+
+ return $scfg->{'lxc-backup-mode'} // properties()->{'lxc-backup-mode'}->{'default'};
+}
+
+# Configuration
+
+sub api {
+ return 11;
+}
+
+sub type {
+ return 'backup-provider-dir-example';
+}
+
+sub plugindata {
+ return {
+ content => [ { backup => 1, none => 1 }, { backup => 1 } ],
+ features => { 'backup-provider' => 1 },
+ 'sensitive-properties' => {},
+ };
+}
+
+sub properties {
+ return {
+ 'lxc-backup-mode' => {
+ description => "How to create LXC backups. tar - create a tar archive."
+ ." squashfs - create a squashfs image. Requires squashfs-tools to be installed.",
+ type => 'string',
+ enum => [qw(tar squashfs)],
+ default => 'tar',
+ },
+ 'vm-backup-mechanism' => {
+ description => "Which mechanism to use for creating VM backups. nbd - access data via "
+ ." NBD export. file-handle - access data via file handle.",
+ type => 'string',
+ enum => [qw(nbd file-handle)],
+ default => 'file-handle',
+ },
+ 'vm-backup-mode' => {
+ description => "How to create VM backups. full - always create full backups."
+ ." incremental - create incremental backups when possible, fallback to full when"
+ ." necessary, e.g. VM disk's bitmap is invalid.",
+ type => 'string',
+ enum => [qw(full incremental)],
+ default => 'full',
+ },
+ };
+}
+
+sub options {
+ return {
+ path => { fixed => 1 },
+ 'lxc-backup-mode' => { optional => 1 },
+ 'vm-backup-mechanism' => { optional => 1 },
+ 'vm-backup-mode' => { optional => 1 },
+ disable => { optional => 1 },
+ nodes => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
+ 'max-protected-backups' => { optional => 1 },
+ };
+}
+
+# Storage implementation
+
+# NOTE a proper backup storage should implement this
+sub prune_backups {
+ my ($class, $scfg, $storeid, $keep, $vmid, $type, $dryrun, $logfunc) = @_;
+
+ die "not implemented";
+}
+
+sub parse_volname {
+ my ($class, $volname) = @_;
+
+ if ($volname =~ m!^backup/((\d+)/[a-z]+-\d+)$!) {
+ my ($filename, $vmid) = ($1, $2);
+ return ('backup', $filename, $vmid);
+ }
+
+ die "unable to parse volume name '$volname'\n";
+}
+
+sub path {
+ my ($class, $scfg, $volname, $storeid, $snapname) = @_;
+
+ die "volume snapshot is not possible on backup-provider-dir-example volume" if $snapname;
+
+ my ($type, $filename, $vmid) = $class->parse_volname($volname);
+
+ return ("$scfg->{path}/${filename}", $vmid, $type);
+}
+
+sub create_base {
+ my ($class, $storeid, $scfg, $volname) = @_;
+
+ die "cannot create base image in backup-provider-dir-example storage\n";
+}
+
+sub clone_image {
+ my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
+
+ die "can't clone images in backup-provider-dir-example storage\n";
+}
+
+sub alloc_image {
+ my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
+
+ die "can't allocate space in backup-provider-dir-example storage\n";
+}
+
+# NOTE a proper backup storage should implement this
+sub free_image {
+ my ($class, $storeid, $scfg, $volname, $isBase) = @_;
+
+ # if it's a backing file, it would need to be merged into the upper image first.
+
+ die "not implemented";
+}
+
+sub list_images {
+ my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
+
+ my $res = [];
+
+ return $res;
+}
+
+sub list_volumes {
+ my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
+
+ my $path = $scfg->{path};
+
+ my $res = [];
+ for my $type ($content_types->@*) {
+ next if $type ne 'backup';
+
+ my @guest_dirs = glob("$path/*");
+ for my $guest_dir (@guest_dirs) {
+ next if !-d $guest_dir || $guest_dir !~ m!/(\d+)$!;
+
+ my $backup_vmid = basename($guest_dir);
+
+ next if defined($vmid) && $backup_vmid != $vmid;
+
+ my @backup_dirs = glob("$guest_dir/*");
+ for my $backup_dir (@backup_dirs) {
+ next if !-d $backup_dir || $backup_dir !~ m!/(lxc|qemu)-(\d+)$!;
+ my ($subtype, $backup_id) = ($1, $2);
+
+ my $size = 0;
+ my @backup_files = glob("$backup_dir/*");
+ $size += -s $_ for @backup_files;
+
+ push $res->@*, {
+ volid => "$storeid:backup/${backup_vmid}/${subtype}-${backup_id}",
+ vmid => $backup_vmid,
+ format => "directory",
+ ctime => $backup_id,
+ size => $size,
+ subtype => $subtype,
+ content => $type,
+ # TODO parent for incremental
+ };
+ }
+ }
+ }
+
+ return $res;
+}
+
+sub activate_storage {
+ my ($class, $storeid, $scfg, $cache) = @_;
+
+ my $path = $scfg->{path};
+
+ my $timeout = 2;
+ if (!PVE::Tools::run_fork_with_timeout($timeout, sub {-d $path})) {
+ die "unable to activate storage '$storeid' - directory '$path' does not exist or is"
+ ." unreachable\n";
+ }
+
+ return 1;
+}
+
+sub deactivate_storage {
+ my ($class, $storeid, $scfg, $cache) = @_;
+
+ return 1;
+}
+
+sub activate_volume {
+ my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+
+ die "volume snapshot is not possible on backup-provider-dir-example volume" if $snapname;
+
+ return 1;
+}
+
+sub deactivate_volume {
+ my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+
+ die "volume snapshot is not possible on backup-provider-dir-example volume" if $snapname;
+
+ return 1;
+}
+
+sub get_volume_attribute {
+ my ($class, $scfg, $storeid, $volname, $attribute) = @_;
+
+ return;
+}
+
+# NOTE a proper backup storage should implement this to support backup notes and
+# setting protected status.
+sub update_volume_attribute {
+ my ($class, $scfg, $storeid, $volname, $attribute, $value) = @_;
+
+ die "attribute '$attribute' is not supported on backup-provider-dir-example volume";
+}
+
+sub volume_size_info {
+ my ($class, $scfg, $storeid, $volname, $timeout) = @_;
+
+ my (undef, $relative_backup_dir) = $class->parse_volname($volname);
+ my ($ctime) = $relative_backup_dir =~ m/-(\d+)$/;
+ my $backup_dir = "$scfg->{path}/${relative_backup_dir}";
+
+ my $size = 0;
+ my @backup_files = glob("$backup_dir/*");
+ for my $backup_file (@backup_files) {
+ if ($backup_file =~ m!\.qcow2$!) {
+ $size += PVE::Storage::Plugin::file_size_info($backup_file, undef, 'qcow2');
+ } else {
+ $size += -s $backup_file;
+ }
+ }
+
+ my $parent; # TODO for incremental
+
+ return wantarray ? ($size, 'directory', $size, $parent, $ctime) : $size;
+}
+
+sub volume_resize {
+ my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+
+ die "volume resize is not possible on backup-provider-dir-example volume";
+}
+
+sub volume_snapshot {
+ my ($class, $scfg, $storeid, $volname, $snap) = @_;
+
+ die "volume snapshot is not possible on backup-provider-dir-example volume";
+}
+
+sub volume_snapshot_rollback {
+ my ($class, $scfg, $storeid, $volname, $snap) = @_;
+
+ die "volume snapshot rollback is not possible on backup-provider-dir-example volume";
+}
+
+sub volume_snapshot_delete {
+ my ($class, $scfg, $storeid, $volname, $snap) = @_;
+
+ die "volume snapshot delete is not possible on backup-provider-dir-example volume";
+}
+
+sub volume_has_feature {
+ my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running) = @_;
+
+ return 0;
+}
+
+sub new_backup_provider {
+ my ($class, $scfg, $storeid, $bandwidth_limit, $log_function) = @_;
+
+ return PVE::BackupProvider::Plugin::DirectoryExample->new(
+ $class, $scfg, $storeid, $bandwidth_limit, $log_function);
+}
+
+1;
diff --git a/src/PVE/Storage/Custom/BorgBackupPlugin.pm b/src/PVE/Storage/Custom/BorgBackupPlugin.pm
new file mode 100644
index 0000000..84b12b9
--- /dev/null
+++ b/src/PVE/Storage/Custom/BorgBackupPlugin.pm
@@ -0,0 +1,689 @@
+package PVE::Storage::Custom::BorgBackupPlugin;
+
+use strict;
+use warnings;
+
+use Fcntl qw(F_GETFD F_SETFD FD_CLOEXEC);
+use File::Path qw(make_path remove_tree);
+use JSON qw(from_json);
+use MIME::Base64 qw(decode_base64 encode_base64);
+use Net::IP;
+use POSIX;
+
+use PVE::BackupProvider::Plugin::Borg;
+use PVE::Tools;
+
+use base qw(PVE::Storage::Plugin);
+
+sub api {
+ return 11;
+}
+
+sub check_config {
+ my ($class, $sectionId, $config, $create, $skipSchemaCheck) = @_;
+
+ if (my $ssh_public_keys = $config->{'ssh-server-public-keys'}) {
+ if ($ssh_public_keys !~ m/^[A-Za-z0-9+\/]+={0,2}$/) {
+ $config->{'ssh-server-public-keys'} = encode_base64($ssh_public_keys, '');
+ }
+ }
+
+ return $class->SUPER::check_config($sectionId, $config, $create, $skipSchemaCheck);
+}
+
+sub borg_repository_uri {
+ my ($class, $scfg, $storeid) = @_;
+
+ my $uri = '';
+ my $server = $scfg->{server} or die "no server configured for $storeid\n";
+ my $username = $scfg->{username} or die "no username configured for $storeid\n";
+ my $prefix = "ssh://$username@";
+ $server = "[$server]" if Net::IP::ip_is_ipv6($server);
+ if (my $port = $scfg->{port}) {
+ $uri = "${prefix}${server}:${port}";
+ } else {
+ $uri = "${prefix}${server}";
+ }
+ $uri .= $scfg->{'repository-path'};
+
+ return $uri;
+}
+
+my sub borg_password_file_name {
+ my ($scfg, $storeid) = @_;
+
+ return "/etc/pve/priv/storage/${storeid}.pw";
+}
+
+my sub borg_set_password {
+ my ($scfg, $storeid, $password) = @_;
+
+ my $pwfile = borg_password_file_name($scfg, $storeid);
+ mkdir "/etc/pve/priv/storage";
+
+ PVE::Tools::file_set_contents($pwfile, "$password\n");
+}
+
+my sub borg_delete_password {
+ my ($scfg, $storeid) = @_;
+
+ my $pwfile = borg_password_file_name($scfg, $storeid);
+
+ unlink $pwfile;
+}
+
+sub borg_get_password {
+ my ($class, $scfg, $storeid) = @_;
+
+ my $pwfile = borg_password_file_name($scfg, $storeid);
+
+ return PVE::Tools::file_read_firstline($pwfile);
+}
+
+sub borg_setup_ssh_dir {
+ my ($class, $scfg, $ssh_dir, $ssh_key) = @_;
+
+ my $dir_created;
+ my $ssh_opts = [];
+
+ my $ensure_dir_created = sub {
+ return if $dir_created;
+ # for container backup, it needs to be created while privileged and already exists
+ if (!-d $ssh_dir) {
+ make_path($ssh_dir) or die "unable to create directory $ssh_dir\n";
+ chmod(0700, $ssh_dir) or die "unable to chmod directory $ssh_dir\n";
+ }
+ PVE::Tools::run_command(
+ ['mount', '-t', 'tmpfs', '-o', 'size=1M,mode=0700', 'tmpfs', $ssh_dir]);
+ $dir_created = 1;
+ };
+
+ if ($ssh_key) {
+ $ensure_dir_created->();
+ PVE::Tools::file_set_contents("${ssh_dir}/ssh.key", $ssh_key, 0600);
+ push $ssh_opts->@*, '-i', "${ssh_dir}/ssh.key";
+ }
+
+ if ($scfg->{'ssh-server-public-keys'}) {
+ $ensure_dir_created->();
+ my $raw = decode_base64($scfg->{'ssh-server-public-keys'});
+ PVE::Tools::file_set_contents("${ssh_dir}/known_hosts", $raw, 0600);
+ push $ssh_opts->@*, '-o', "UserKnownHostsFile=${ssh_dir}/known_hosts";
+ push $ssh_opts->@*, '-o', "GlobalKnownHostsFile=none";
+ }
+
+ return ($dir_created, $ssh_opts);
+}
+
+sub borg_cmd_env {
+ my ($class, $scfg, $storeid, $sub) = @_;
+
+ my $ssh_dir = "/run/pve-storage/borg-plugin/${storeid}.ssh.$$";
+ my $ssh_key = borg_get_ssh_key($scfg, $storeid);
+ my ($uses_ssh_dir, $ssh_options) = $class->borg_setup_ssh_dir($scfg, $ssh_dir, $ssh_key);
+ local $ENV{BORG_RSH} = "ssh " . join(" ", $ssh_options->@*);
+
+ local $ENV{BORG_PASSPHRASE} = $class->borg_get_password($scfg, $storeid);
+
+ my $res = eval {
+ my $uri = $class->borg_repository_uri($scfg, $storeid);
+ return $sub->($uri);
+ };
+ my $err = $@;
+
+ if ($uses_ssh_dir) {
+ eval { PVE::Tools::run_command(['umount', "$ssh_dir"]); };
+ warn "unable to unmount directory $ssh_dir - $@" if $@;
+ eval { remove_tree($ssh_dir); };
+ warn "unable to cleanup directory $ssh_dir - $@" if $@;
+ }
+
+ die $err if $err;
+
+ return $res;
+}
+
+sub borg_cmd_list {
+ my ($class, $scfg, $storeid) = @_;
+
+ return $class->borg_cmd_env($scfg, $storeid, sub {
+ my ($uri) = @_;
+
+ my $json = '';
+ my $cmd = ['borg', 'list', '--json', $uri];
+
+ my $errfunc = sub { warn $_[0]; };
+ my $outfunc = sub { $json .= $_[0]; };
+
+ PVE::Tools::run_command(
+ $cmd, errmsg => "command @$cmd failed", outfunc => $outfunc, errfunc => $errfunc);
+
+ my $res = eval { from_json($json) };
+ die "unable to parse 'borg list' output - $@\n" if $@;
+ return $res;
+ });
+}
+
+sub borg_cmd_create {
+ my ($class, $scfg, $storeid, $archive, $paths, $opts) = @_;
+
+ return $class->borg_cmd_env($scfg, $storeid, sub {
+ my ($uri) = @_;
+
+ my $cmd = ['borg', 'create', $opts->@*, "${uri}::${archive}", $paths->@*];
+
+ PVE::Tools::run_command($cmd, errmsg => "command @$cmd failed");
+
+ return;
+ });
+}
+
+sub borg_cmd_extract_contents {
+ my ($class, $scfg, $storeid, $archive, $paths) = @_;
+
+ return $class->borg_cmd_env($scfg, $storeid, sub {
+ my ($uri) = @_;
+
+ my $output = '';
+ my $outfunc = sub {
+ $output .= "$_[0]\n";
+ };
+
+ my $cmd = ['borg', 'extract', '--stdout', "${uri}::${archive}", $paths->@*];
+
+ PVE::Tools::run_command($cmd, errmsg => "command @$cmd failed", outfunc => $outfunc);
+
+ return $output;
+ });
+}
+
+sub borg_cmd_delete {
+ my ($class, $scfg, $storeid, $archive) = @_;
+
+ return $class->borg_cmd_env($scfg, $storeid, sub {
+ my ($uri) = @_;
+
+ my $cmd = ['borg', 'delete', "${uri}::${archive}"];
+
+ PVE::Tools::run_command($cmd, errmsg => "command @$cmd failed");
+
+ return;
+ });
+}
+
+sub borg_cmd_info {
+ my ($class, $scfg, $storeid, $archive, $timeout) = @_;
+
+ return $class->borg_cmd_env($scfg, $storeid, sub {
+ my ($uri) = @_;
+
+ my $json = '';
+ my $cmd = ['borg', 'info', '--json', "${uri}::${archive}"];
+
+ my $errfunc = sub { warn $_[0]; };
+ my $outfunc = sub { $json .= $_[0]; };
+
+ PVE::Tools::run_command(
+ $cmd,
+ errmsg => "command @$cmd failed",
+ timeout => $timeout,
+ outfunc => $outfunc,
+ errfunc => $errfunc,
+ );
+
+ my $res = eval { from_json($json) };
+ die "unable to parse 'borg info' output for archive '$archive' - $@\n" if $@;
+ return $res;
+ });
+}
+
+sub borg_cmd_mount {
+ my ($class, $scfg, $storeid, $archive, $mount_point) = @_;
+
+ return $class->borg_cmd_env($scfg, $storeid, sub {
+ my ($uri) = @_;
+
+ my $cmd = ['borg', 'mount', "${uri}::${archive}", $mount_point];
+
+ PVE::Tools::run_command($cmd, errmsg => "command @$cmd failed");
+
+ return;
+ });
+}
+
+my sub parse_backup_time {
+ my ($time_string) = @_;
+
+ my @tm = (POSIX::strptime($time_string, "%FT%TZ"));
+ # expect sec, min, hour, mday, mon, year
+ if (grep { !defined($_) } @tm[0..5]) {
+ warn "error parsing time from string '$time_string'\n";
+ return 0;
+ } else {
+ local $ENV{TZ} = 'UTC'; # time string is UTC
+
+ # Fill in isdst to avoid undef warning. No daylight saving time for UTC.
+ $tm[8] //= 0;
+
+ if (my $since_epoch = mktime(@tm)) {
+ return int($since_epoch);
+ } else {
+ warn "error parsing time from string '$time_string'\n";
+ return 0;
+ }
+ }
+}
+
+# Helpers
+
+sub type {
+ return 'borg';
+}
+
+sub plugindata {
+ return {
+ content => [ { backup => 1, none => 1 }, { backup => 1 } ],
+ features => { 'backup-provider' => 1 },
+ 'sensitive-properties' => {
+ password => 1,
+ 'ssh-private-key' => 1,
+ },
+ };
+}
+
+sub properties {
+ return {
+ 'repository-path' => {
+ description => "Path to the backup repository",
+ type => 'string',
+ },
+ 'ssh-private-key' => {
+ # Since 1 is written to the config when the key is present, the format is not checked.
+ description => "SSH identity/private key for the client-side in PEM format.",
+ type => 'string',
+ },
+ 'ssh-server-public-keys' => {
+ description => "SSH public key(s) for the server-side, (one key per line, OpenSSH"
+ ." format).",
+ type => 'string',
+ },
+ };
+}
+
+sub options {
+ return {
+ 'repository-path' => { fixed => 1 },
+ server => { fixed => 1 },
+ port => { optional => 1 },
+ username => { fixed => 1 },
+ 'ssh-private-key' => { optional => 1 },
+ 'ssh-server-public-keys' => { optional => 1 },
+ password => { optional => 1 },
+ disable => { optional => 1 },
+ nodes => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
+ 'max-protected-backups' => { optional => 1 },
+ };
+}
+
+sub borg_ssh_key_file_name {
+ my ($scfg, $storeid) = @_;
+
+ return "/etc/pve/priv/storage/${storeid}.ssh.key";
+}
+
+sub borg_set_ssh_key {
+ my ($scfg, $storeid, $key) = @_;
+
+ my $keyfile = borg_ssh_key_file_name($scfg, $storeid);
+ mkdir "/etc/pve/priv/storage";
+
+ PVE::Tools::file_set_contents($keyfile, "$key\n");
+}
+
+sub borg_delete_ssh_key {
+ my ($scfg, $storeid) = @_;
+
+ my $keyfile = borg_ssh_key_file_name($scfg, $storeid);
+
+ if (!unlink $keyfile) {
+ return if $! == ENOENT;
+ die "failed to delete SSH key! $!\n";
+ }
+ delete $scfg->{'ssh-private-key'};
+}
+
+sub borg_get_ssh_key {
+ my ($scfg, $storeid) = @_;
+
+ my $keyfile = borg_ssh_key_file_name($scfg, $storeid);
+
+ return if !-f $keyfile;
+
+ return PVE::Tools::file_get_contents($keyfile);
+}
+
+# Returns a file handle with FD_CLOEXEC disabled if there is an SSH key , or `undef` if there is
+# not. Dies on error.
+sub borg_open_ssh_key {
+ my ($self, $scfg, $storeid) = @_;
+
+ my $ssh_key_file = borg_ssh_key_file_name($scfg, $storeid);
+
+ my $keyfd;
+ if (!open($keyfd, '<', $ssh_key_file)) {
+ if ($! == ENOENT) {
+ die "SSH key configured but no key file found!\n" if $scfg->{'ssh-private-key'};
+ return undef;
+ }
+ die "failed to open SSH key: $ssh_key_file: $!\n";
+ }
+ my $flags = fcntl($keyfd, F_GETFD, 0)
+ // die "failed to get file descriptor flags for SSH key FD: $!\n";
+ fcntl($keyfd, F_SETFD, $flags & ~FD_CLOEXEC)
+ or die "failed to remove FD_CLOEXEC from SSH key file descriptor\n";
+
+ return $keyfd;
+}
+
+# Storage implementation
+
+sub on_add_hook {
+ my ($class, $storeid, $scfg, %param) = @_;
+
+ if (defined(my $password = $param{password})) {
+ borg_set_password($scfg, $storeid, $password);
+ } else {
+ borg_delete_password($scfg, $storeid);
+ }
+
+ if (defined(my $ssh_key = delete $param{'ssh-private-key'})) {
+ borg_set_ssh_key($scfg, $storeid, $ssh_key);
+ $scfg->{'ssh-private-key'} = 1;
+ } else {
+ borg_delete_ssh_key($scfg, $storeid);
+ }
+
+ if ($scfg->{'ssh-server-public-keys'}) {
+ my $ssh_public_keys = decode_base64($scfg->{'ssh-server-public-keys'});
+ PVE::Tools::validate_ssh_public_keys($ssh_public_keys);
+ }
+
+ return;
+}
+
+sub on_update_hook {
+ my ($class, $storeid, $scfg, %param) = @_;
+
+ if (exists($param{password})) {
+ if (defined($param{password})) {
+ borg_set_password($scfg, $storeid, $param{password});
+ } else {
+ borg_delete_password($scfg, $storeid);
+ }
+ }
+
+ if (exists($param{'ssh-private-key'})) {
+ if (defined(my $ssh_key = delete($param{'ssh-private-key'}))) {
+ borg_set_ssh_key($scfg, $storeid, $ssh_key);
+ $scfg->{'ssh-private-key'} = 1;
+ } else {
+ borg_delete_ssh_key($scfg, $storeid);
+ }
+ }
+
+ if ($scfg->{'ssh-server-public-keys'}) {
+ my $ssh_public_keys = decode_base64($scfg->{'ssh-server-public-keys'});
+ PVE::Tools::validate_ssh_public_keys($ssh_public_keys);
+ }
+
+ return;
+}
+
+sub on_delete_hook {
+ my ($class, $storeid, $scfg) = @_;
+
+ borg_delete_password($scfg, $storeid);
+ borg_delete_ssh_key($scfg, $storeid);
+
+ return;
+}
+
+sub prune_backups {
+ my ($class, $scfg, $storeid, $keep, $vmid, $type, $dryrun, $logfunc) = @_;
+
+ # FIXME - is 'borg prune' compatible with ours?
+ die "not implemented";
+}
+
+sub parse_volname {
+ my ($class, $volname) = @_;
+
+ if ($volname =~ m!^backup/(.*)$!) {
+ my $archive = $1;
+ if ($archive =~ $PVE::BackupProvider::Plugin::Borg::ARCHIVE_RE_3) {
+ return ('backup', $archive, $2);
+ }
+ }
+
+ die "unable to parse Borg volume name '$volname'\n";
+}
+
+sub path {
+ my ($class, $scfg, $volname, $storeid, $snapname) = @_;
+
+ die "volume snapshot is not possible on Borg volume" if $snapname;
+
+ my $uri = $class->borg_repository_uri($scfg, $storeid);
+ my (undef, $archive) = $class->parse_volname($volname);
+
+ return "${uri}::${archive}";
+}
+
+sub create_base {
+ my ($class, $storeid, $scfg, $volname) = @_;
+
+ die "cannot create base image in Borg storage\n";
+}
+
+sub clone_image {
+ my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_;
+
+ die "can't clone images in Borg storage\n";
+}
+
+sub alloc_image {
+ my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
+
+ die "can't allocate space in Borg storage\n";
+}
+
+sub free_image {
+ my ($class, $storeid, $scfg, $volname, $isBase) = @_;
+
+ my (undef, $archive) = $class->parse_volname($volname);
+
+ borg_cmd_delete($class, $scfg, $storeid, $archive);
+
+ return;
+}
+
+sub list_images {
+ my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
+
+ return []; # guest images are not supported, only backups
+}
+
+sub list_volumes {
+ my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
+
+ my $res = [];
+
+ return $res if !grep { $_ eq 'backup' } $content_types->@*;
+
+ my $archives = $class->borg_cmd_list($scfg, $storeid)->{archives}
+ or die "expected 'archives' key in 'borg list' JSON output missing\n";
+
+ for my $info ($archives->@*) {
+ my $archive = $info->{archive};
+ my ($vmtype, $backup_vmid, $time_string) =
+ $archive =~ $PVE::BackupProvider::Plugin::Borg::ARCHIVE_RE_3 or next;
+
+ next if defined($vmid) && $vmid != $backup_vmid;
+
+ push $res->@*, {
+ volid => "${storeid}:backup/${archive}",
+ size => 0, # FIXME how to cheaply get?
+ content => 'backup',
+ ctime => parse_backup_time($time_string),
+ vmid => $backup_vmid,
+ format => "borg-archive",
+ subtype => $vmtype,
+ }
+ }
+
+ return $res;
+}
+
+sub status {
+ my ($class, $storeid, $scfg, $cache) = @_;
+
+ my $uri = $class->borg_repository_uri($scfg, $storeid);
+
+ my $res;
+
+ if ($uri =~ m!^ssh://!) {
+ #FIXME ssh and df on target?
+ return;
+ } else { # $uri is a local path
+ my $timeout = 2;
+ $res = PVE::Tools::df($uri, $timeout);
+
+ return if !$res || !$res->{total};
+ }
+
+
+ return ($res->{total}, $res->{avail}, $res->{used}, 1);
+}
+
+sub activate_storage {
+ my ($class, $storeid, $scfg, $cache) = @_;
+
+ # TODO how to cheaply check? split ssh and non-ssh?
+
+ return 1;
+}
+
+sub deactivate_storage {
+ my ($class, $storeid, $scfg, $cache) = @_;
+
+ return 1;
+}
+
+sub activate_volume {
+ my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+
+ die "volume snapshot is not possible on Borg volume" if $snapname;
+
+ return 1;
+}
+
+sub deactivate_volume {
+ my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+
+ die "volume snapshot is not possible on Borg volume" if $snapname;
+
+ return 1;
+}
+
+sub get_volume_attribute {
+ my ($class, $scfg, $storeid, $volname, $attribute) = @_;
+
+ return;
+}
+
+sub update_volume_attribute {
+ my ($class, $scfg, $storeid, $volname, $attribute, $value) = @_;
+
+ # FIXME notes or protected possible?
+
+ die "attribute '$attribute' is not supported on Borg volume";
+}
+
+sub volume_size_info {
+ my ($class, $scfg, $storeid, $volname, $timeout) = @_;
+
+ my (undef, $archive) = $class->parse_volname($volname);
+ my (undef, undef, $time_string) =
+ $archive =~ $PVE::BackupProvider::Plugin::Borg::ARCHIVE_RE_3;
+
+ my $backup_time = 0;
+ if ($time_string) {
+ $backup_time = parse_backup_time($time_string)
+ } else {
+ warn "could not parse time from archive name '$archive'\n";
+ }
+
+ my $archives = borg_cmd_info($class, $scfg, $storeid, $archive, $timeout)->{archives}
+ or die "expected 'archives' key in 'borg info' JSON output missing\n";
+
+ my $stats = eval { $archives->[0]->{stats} }
+ or die "expected entry in 'borg info' JSON output missing\n";
+ my ($size, $used) = $stats->@{qw(original_size deduplicated_size)};
+
+ ($size) = ($size =~ /^(\d+)$/); # untaint
+ die "size '$size' not an integer\n" if !defined($size);
+ # coerce back from string
+ $size = int($size);
+ ($used) = ($used =~ /^(\d+)$/); # untaint
+ die "used '$used' not an integer\n" if !defined($used);
+ # coerce back from string
+ $used = int($used);
+
+ return wantarray ? ($size, 'borg-archive', $used, undef, $backup_time) : $size;
+}
+
+sub volume_resize {
+ my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+
+ die "volume resize is not possible on Borg volume";
+}
+
+sub volume_snapshot {
+ my ($class, $scfg, $storeid, $volname, $snap) = @_;
+
+ die "volume snapshot is not possible on Borg volume";
+}
+
+sub volume_snapshot_rollback {
+ my ($class, $scfg, $storeid, $volname, $snap) = @_;
+
+ die "volume snapshot rollback is not possible on Borg volume";
+}
+
+sub volume_snapshot_delete {
+ my ($class, $scfg, $storeid, $volname, $snap) = @_;
+
+ die "volume snapshot delete is not possible on Borg volume";
+}
+
+sub volume_has_feature {
+ my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running) = @_;
+
+ return 0;
+}
+
+sub rename_volume {
+ my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
+
+ die "volume rename is not implemented in Borg storage plugin\n";
+}
+
+sub new_backup_provider {
+ my ($class, $scfg, $storeid, $bandwidth_limit, $log_function) = @_;
+
+ return PVE::BackupProvider::Plugin::Borg->new(
+ $class, $scfg, $storeid, $bandwidth_limit, $log_function);
+}
+
+1;
diff --git a/src/PVE/Storage/Custom/Makefile b/src/PVE/Storage/Custom/Makefile
new file mode 100644
index 0000000..886442d
--- /dev/null
+++ b/src/PVE/Storage/Custom/Makefile
@@ -0,0 +1,6 @@
+SOURCES = BackupProviderDirExamplePlugin.pm \
+ BorgBackupPlugin.pm
+
+.PHONY: install
+install:
+ for i in ${SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/Storage/Custom/$$i; done
diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile
new file mode 100644
index 0000000..530ce54
--- /dev/null
+++ b/src/PVE/Storage/Makefile
@@ -0,0 +1,3 @@
+.PHONY: install
+install:
+ make -C Custom install
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel