From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 0FC641FF163 for ; Thu, 12 Sep 2024 14:44:05 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D901430DB2; Thu, 12 Sep 2024 14:44:05 +0200 (CEST) Date: Thu, 12 Sep 2024 14:43:59 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox VE development discussion References: <20240813132829.117460-1-f.ebner@proxmox.com> <20240813132829.117460-24-f.ebner@proxmox.com> In-Reply-To: <20240813132829.117460-24-f.ebner@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.16.0 (https://github.com/astroidmail/astroid) Message-Id: <1726144190.zjc530zmr3.astroid@yuna.none> X-SPAM-LEVEL: Spam detection results: 0 AWL 0.049 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [RFC container v2 23/25] backup: implement restore for external providers X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" On August 13, 2024 3:28 pm, Fiona Ebner wrote: > First, the provider is asked about what restore mechanism to use. > Currently, 'directory' and 'tar' are possible, for restoring either > from a directory containing the full filesystem structure (for which > rsync is used) or a potentially compressed tar file containing the > same. > > The new functions are copied and adapted from the existing ones for > PBS or tar and it might be worth to factor out the common parts. > > Signed-off-by: Fiona Ebner > --- > > Changes in v2: > * Adapt to API changes. > > src/PVE/LXC/Create.pm | 141 ++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 141 insertions(+) > > diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm > index 117103c..9d1c337 100644 > --- a/src/PVE/LXC/Create.pm > +++ b/src/PVE/LXC/Create.pm > @@ -25,6 +25,24 @@ sub restore_archive { > if ($scfg->{type} eq 'pbs') { > return restore_proxmox_backup_archive($storage_cfg, $archive, $rootdir, $conf, $no_unpack_error, $bwlimit); > } > + my $log_function = sub { > + my ($log_level, $message) = @_; > + my $prefix = $log_level eq 'err' ? 'ERROR' : uc($log_level); > + print "$prefix: $message\n"; > + }; > + my $backup_provider = > + PVE::Storage::new_backup_provider($storage_cfg, $storeid, $log_function); > + if ($backup_provider) { > + return restore_external_archive( > + $backup_provider, > + $storeid, > + $volname, > + $rootdir, > + $conf, > + $no_unpack_error, > + $bwlimit, > + ); > + } > } > > $archive = PVE::Storage::abs_filesystem_path($storage_cfg, $archive) if $archive ne '-'; > @@ -118,6 +136,55 @@ sub restore_tar_archive { > die $err if $err && !$no_unpack_error; > } > > +sub restore_external_archive { > + my ($backup_provider, $storeid, $volname, $rootdir, $conf, $no_unpack_error, $bwlimit) = @_; > + > + my ($mechanism, $vmtype) = $backup_provider->restore_get_mechanism($volname, $storeid); > + die "cannot restore non-LXC guest of type '$vmtype'\n" if $vmtype ne 'lxc'; > + > + my $info = $backup_provider->restore_container_init($volname, $storeid, {}); > + eval { > + if ($mechanism eq 'tar') { > + my $tar_path = $info->{'tar-path'} > + or die "did not get path to tar file from backup provider\n"; > + die "not a regular file '$tar_path'" if !-f $tar_path; > + restore_tar_archive($tar_path, $rootdir, $conf, $no_unpack_error, $bwlimit); shouldn't this be `lxc-userns-exec`-ed? > + } elsif ($mechanism eq 'directory') { > + my $directory = $info->{'archive-directory'} > + or die "did not get path to archive directory from backup provider\n"; > + die "not a directory '$directory'" if !-d $directory; > + > + my $rsync = ['rsync', '--stats', '-h', '-X', '-A', '--numeric-ids', '-aH', '--delete', > + '--no-whole-file', '--sparse', '--one-file-system', '--relative']; > + push $rsync->@*, '--bwlimit', $bwlimit if $bwlimit; > + push $rsync->@*, "${directory}/./", $rootdir; and this as well? also, for both tar and rsync we probably need to think about how to prevent bogus input here (which might be user-creatable if they have write access to the backup storage) from violating our assumptions.. > + > + my $transferred = ''; > + my $outfunc = sub { > + return if $_[0] !~ /^Total transferred file size: (.+)$/; > + $transferred = $1; > + }; > + my $errfunc = sub { log_warn($_[0]); }; > + > + my $starttime = time(); > + PVE::Tools::run_command($rsync, outfunc => $outfunc, errfunc => $errfunc); > + my $delay = time () - $starttime; > + > + print "sync finished - transferred ${transferred} in ${delay}s\n"; _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel