From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id E2A871FF16E
for <inbox@lore.proxmox.com>; Mon, 31 Mar 2025 15:29:29 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 65FB95F5C;
Mon, 31 Mar 2025 15:29:17 +0200 (CEST)
From: Fiona Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Mon, 31 Mar 2025 15:20:14 +0200
Message-Id: <20250331132020.105324-32-f.ebner@proxmox.com>
X-Mailer: git-send-email 2.39.5
In-Reply-To: <20250331132020.105324-1-f.ebner@proxmox.com>
References: <20250331132020.105324-1-f.ebner@proxmox.com>
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.040 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH container v6 31/37] backup: implement restore
for external providers
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
First, the provider is asked about what restore mechanism to use.
Currently, 'directory' and 'tar' are possible. The 'directory'
mechanism is for restoring from a directory containing the container's
full filesystem structure, which is restored by piping from a
privileged tar cf - to tar xf - in the associated user namespace. The
'tar' mechanism is for restoring a (potentially compressed) tar file
containing the container's full filesystem structure.
The new functions are copied and adapted from the existing ones for
PBS or tar and it might be worth to factor out the common parts.
Restore of containers as privileged are prohibited, because the
archives from an external provider are considered less trusted than
from Proxmox VE storages. If ever allowing that in the future, at
least it would be worth extracting the tar archive in a restricted
context (e.g. user namespace with ID mapped mount or seccomp).
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
Changes in v6:
* Adapt to renamed archive_get_*_config() methods.
src/PVE/LXC/Create.pm | 149 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 149 insertions(+)
diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm
index 8c8cb9a..c3c7640 100644
--- a/src/PVE/LXC/Create.pm
+++ b/src/PVE/LXC/Create.pm
@@ -7,6 +7,7 @@ use File::Path;
use Fcntl;
use PVE::RPCEnvironment;
+use PVE::RESTEnvironment qw(log_warn);
use PVE::Storage::PBSPlugin;
use PVE::Storage::Plugin;
use PVE::Storage;
@@ -26,6 +27,24 @@ sub restore_archive {
if ($scfg->{type} eq 'pbs') {
return restore_proxmox_backup_archive($storage_cfg, $archive, $rootdir, $conf, $no_unpack_error, $bwlimit);
}
+ if (PVE::Storage::storage_has_feature($storage_cfg, $storeid, 'backup-provider')) {
+ my $log_function = sub {
+ my ($log_level, $message) = @_;
+ my $prefix = $log_level eq 'err' ? 'ERROR' : uc($log_level);
+ print "$prefix: $message\n";
+ };
+ my $backup_provider =
+ PVE::Storage::new_backup_provider($storage_cfg, $storeid, $log_function);
+ return restore_external_archive(
+ $backup_provider,
+ $storeid,
+ $volname,
+ $rootdir,
+ $conf,
+ $no_unpack_error,
+ $bwlimit,
+ );
+ }
}
$archive = PVE::Storage::abs_filesystem_path($storage_cfg, $archive) if $archive ne '-';
@@ -127,6 +146,62 @@ sub restore_tar_archive {
die $err if $err && !$no_unpack_error;
}
+sub restore_external_archive {
+ my ($backup_provider, $storeid, $volname, $rootdir, $conf, $no_unpack_error, $bwlimit) = @_;
+
+ die "refusing to restore privileged container backup from external source\n"
+ if !$conf->{unprivileged};
+
+ my ($mechanism, $vmtype) = $backup_provider->restore_get_mechanism($volname, $storeid);
+ die "cannot restore non-LXC guest of type '$vmtype'\n" if $vmtype ne 'lxc';
+
+ my $info = $backup_provider->restore_container_init($volname, $storeid, {});
+ eval {
+ if ($mechanism eq 'tar') {
+ my $tar_path = $info->{'tar-path'}
+ or die "did not get path to tar file from backup provider\n";
+ die "not a regular file '$tar_path'" if !-f $tar_path;
+ restore_tar_archive($tar_path, $rootdir, $conf, $no_unpack_error, $bwlimit);
+ } elsif ($mechanism eq 'directory') {
+ my $directory = $info->{'archive-directory'}
+ or die "did not get path to archive directory from backup provider\n";
+ die "not a directory '$directory'" if !-d $directory;
+
+ my $create_cmd = [
+ 'tar',
+ 'cpf',
+ '-',
+ @PVE::Storage::Plugin::COMMON_TAR_FLAGS,
+ "--directory=$directory",
+ '.',
+ ];
+
+ my $extract_cmd = restore_tar_archive_command($conf, undef, $rootdir, $bwlimit);
+
+ my $cmd;
+ # if there is a bandwidth limit, the command is already a nested array reference
+ if (ref($extract_cmd) eq 'ARRAY' && ref($extract_cmd->[0]) eq 'ARRAY') {
+ $cmd = [$create_cmd, $extract_cmd->@*];
+ } else {
+ $cmd = [$create_cmd, $extract_cmd];
+ }
+
+ eval { PVE::Tools::run_command($cmd); };
+ die $@ if $@ && !$no_unpack_error;
+ } else {
+ die "mechanism '$mechanism' requested by backup provider is not supported for LXCs\n";
+ }
+ };
+ my $err = $@;
+ eval { $backup_provider->restore_container_cleanup($volname, $storeid, {}); };
+ if (my $cleanup_err = $@) {
+ die $cleanup_err if !$err;
+ warn $cleanup_err;
+ }
+ die $err if $err;
+
+}
+
sub recover_config {
my ($storage_cfg, $volid, $vmid) = @_;
@@ -135,6 +210,8 @@ sub recover_config {
my $scfg = PVE::Storage::storage_check_enabled($storage_cfg, $storeid);
if ($scfg->{type} eq 'pbs') {
return recover_config_from_proxmox_backup($storage_cfg, $volid, $vmid);
+ } elsif (PVE::Storage::storage_has_feature($storage_cfg, $storeid, 'backup-provider')) {
+ return recover_config_from_external_backup($storage_cfg, $volid, $vmid);
}
}
@@ -209,6 +286,26 @@ sub recover_config_from_tar {
return wantarray ? ($conf, $mp_param) : $conf;
}
+sub recover_config_from_external_backup {
+ my ($storage_cfg, $volid, $vmid) = @_;
+
+ $vmid //= 0;
+
+ my $raw = PVE::Storage::extract_vzdump_config($storage_cfg, $volid);
+
+ my $conf = PVE::LXC::Config::parse_pct_config("/lxc/${vmid}.conf" , $raw);
+
+ delete $conf->{snapshots};
+
+ my $mp_param = {};
+ PVE::LXC::Config->foreach_volume($conf, sub {
+ my ($ms, $mountpoint) = @_;
+ $mp_param->{$ms} = $conf->{$ms};
+ });
+
+ return wantarray ? ($conf, $mp_param) : $conf;
+}
+
sub restore_configuration {
my ($vmid, $storage_cfg, $archive, $rootdir, $conf, $restricted, $unique, $skip_fw) = @_;
@@ -218,6 +315,26 @@ sub restore_configuration {
if ($scfg->{type} eq 'pbs') {
return restore_configuration_from_proxmox_backup($vmid, $storage_cfg, $archive, $rootdir, $conf, $restricted, $unique, $skip_fw);
}
+ if (PVE::Storage::storage_has_feature($storage_cfg, $storeid, 'backup-provider')) {
+ my $log_function = sub {
+ my ($log_level, $message) = @_;
+ my $prefix = $log_level eq 'err' ? 'ERROR' : uc($log_level);
+ print "$prefix: $message\n";
+ };
+ my $backup_provider =
+ PVE::Storage::new_backup_provider($storage_cfg, $storeid, $log_function);
+ return restore_configuration_from_external_backup(
+ $backup_provider,
+ $vmid,
+ $storage_cfg,
+ $archive,
+ $rootdir,
+ $conf,
+ $restricted,
+ $unique,
+ $skip_fw,
+ );
+ }
}
restore_configuration_from_etc_vzdump($vmid, $rootdir, $conf, $restricted, $unique, $skip_fw);
}
@@ -258,6 +375,38 @@ sub restore_configuration_from_proxmox_backup {
}
}
+sub restore_configuration_from_external_backup {
+ my ($backup_provider, $vmid, $storage_cfg, $archive, $rootdir, $conf, $restricted, $unique, $skip_fw) = @_;
+
+ my ($storeid, $volname) = PVE::Storage::parse_volume_id($archive);
+ my $scfg = PVE::Storage::storage_config($storage_cfg, $storeid);
+
+ my ($vtype, $name, undef, undef, undef, undef, $format) =
+ PVE::Storage::parse_volname($storage_cfg, $archive);
+
+ my $oldconf = recover_config_from_external_backup($storage_cfg, $archive, $vmid);
+
+ sanitize_and_merge_config($conf, $oldconf, $restricted, $unique);
+
+ my $firewall_config =
+ $backup_provider->archive_get_firewall_config($volname, $storeid);
+
+ if ($firewall_config) {
+ my $pve_firewall_dir = '/etc/pve/firewall';
+ my $pct_fwcfg_target = "${pve_firewall_dir}/${vmid}.fw";
+ if ($skip_fw) {
+ warn "ignoring firewall config from backup archive, lacking API permission to modify firewall.\n";
+ warn "old firewall configuration in '$pct_fwcfg_target' left in place!\n"
+ if -e $pct_fwcfg_target;
+ } else {
+ mkdir $pve_firewall_dir; # make sure the directory exists
+ PVE::Tools::file_set_contents($pct_fwcfg_target, $firewall_config);
+ }
+ }
+
+ return;
+}
+
sub sanitize_and_merge_config {
my ($conf, $oldconf, $restricted, $unique) = @_;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel