From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Subject: Re: [pve-devel] [RFC container v2 23/25] backup: implement restore for external providers
Date: Thu, 12 Sep 2024 14:43:59 +0200 [thread overview]
Message-ID: <1726144190.zjc530zmr3.astroid@yuna.none> (raw)
In-Reply-To: <20240813132829.117460-24-f.ebner@proxmox.com>
On August 13, 2024 3:28 pm, Fiona Ebner wrote:
> First, the provider is asked about what restore mechanism to use.
> Currently, 'directory' and 'tar' are possible, for restoring either
> from a directory containing the full filesystem structure (for which
> rsync is used) or a potentially compressed tar file containing the
> same.
>
> The new functions are copied and adapted from the existing ones for
> PBS or tar and it might be worth to factor out the common parts.
>
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
>
> Changes in v2:
> * Adapt to API changes.
>
> src/PVE/LXC/Create.pm | 141 ++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 141 insertions(+)
>
> diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm
> index 117103c..9d1c337 100644
> --- a/src/PVE/LXC/Create.pm
> +++ b/src/PVE/LXC/Create.pm
> @@ -25,6 +25,24 @@ sub restore_archive {
> if ($scfg->{type} eq 'pbs') {
> return restore_proxmox_backup_archive($storage_cfg, $archive, $rootdir, $conf, $no_unpack_error, $bwlimit);
> }
> + my $log_function = sub {
> + my ($log_level, $message) = @_;
> + my $prefix = $log_level eq 'err' ? 'ERROR' : uc($log_level);
> + print "$prefix: $message\n";
> + };
> + my $backup_provider =
> + PVE::Storage::new_backup_provider($storage_cfg, $storeid, $log_function);
> + if ($backup_provider) {
> + return restore_external_archive(
> + $backup_provider,
> + $storeid,
> + $volname,
> + $rootdir,
> + $conf,
> + $no_unpack_error,
> + $bwlimit,
> + );
> + }
> }
>
> $archive = PVE::Storage::abs_filesystem_path($storage_cfg, $archive) if $archive ne '-';
> @@ -118,6 +136,55 @@ sub restore_tar_archive {
> die $err if $err && !$no_unpack_error;
> }
>
> +sub restore_external_archive {
> + my ($backup_provider, $storeid, $volname, $rootdir, $conf, $no_unpack_error, $bwlimit) = @_;
> +
> + my ($mechanism, $vmtype) = $backup_provider->restore_get_mechanism($volname, $storeid);
> + die "cannot restore non-LXC guest of type '$vmtype'\n" if $vmtype ne 'lxc';
> +
> + my $info = $backup_provider->restore_container_init($volname, $storeid, {});
> + eval {
> + if ($mechanism eq 'tar') {
> + my $tar_path = $info->{'tar-path'}
> + or die "did not get path to tar file from backup provider\n";
> + die "not a regular file '$tar_path'" if !-f $tar_path;
> + restore_tar_archive($tar_path, $rootdir, $conf, $no_unpack_error, $bwlimit);
shouldn't this be `lxc-userns-exec`-ed?
> + } elsif ($mechanism eq 'directory') {
> + my $directory = $info->{'archive-directory'}
> + or die "did not get path to archive directory from backup provider\n";
> + die "not a directory '$directory'" if !-d $directory;
> +
> + my $rsync = ['rsync', '--stats', '-h', '-X', '-A', '--numeric-ids', '-aH', '--delete',
> + '--no-whole-file', '--sparse', '--one-file-system', '--relative'];
> + push $rsync->@*, '--bwlimit', $bwlimit if $bwlimit;
> + push $rsync->@*, "${directory}/./", $rootdir;
and this as well?
also, for both tar and rsync we probably need to think about how to
prevent bogus input here (which might be user-creatable if they have
write access to the backup storage) from violating our assumptions..
> +
> + my $transferred = '';
> + my $outfunc = sub {
> + return if $_[0] !~ /^Total transferred file size: (.+)$/;
> + $transferred = $1;
> + };
> + my $errfunc = sub { log_warn($_[0]); };
> +
> + my $starttime = time();
> + PVE::Tools::run_command($rsync, outfunc => $outfunc, errfunc => $errfunc);
> + my $delay = time () - $starttime;
> +
> + print "sync finished - transferred ${transferred} in ${delay}s\n";
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next prev parent reply other threads:[~2024-09-12 12:44 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-13 13:28 [pve-devel] [RFC qemu/storage/qemu-server/container/manager v2 00/25] backup provider API Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu v2 01/25] block/reqlist: allow adding overlapping requests Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu v2 02/25] PVE backup: fixup error handling for fleecing Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu v2 03/25] PVE backup: factor out setting up snapshot access " Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu v2 04/25] PVE backup: save device name in device info structure Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu v2 05/25] PVE backup: include device name in error when setting up snapshot access fails Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC qemu v2 06/25] PVE backup: add target ID in backup state Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC qemu v2 07/25] PVE backup: get device info: allow caller to specify filter for which devices use fleecing Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC qemu v2 08/25] PVE backup: implement backup access setup and teardown API for external providers Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC qemu v2 09/25] PVE backup: implement bitmap support for external backup access Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC storage v2 10/25] plugin: introduce new_backup_provider() method Fiona Ebner
2024-09-12 12:43 ` Fabian Grünbichler
2024-09-12 13:21 ` Fiona Ebner
2024-09-13 6:13 ` Fabian Grünbichler
2024-08-13 13:28 ` [pve-devel] [RFC storage v2 11/25] extract backup config: delegate to backup provider if there is one Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [POC storage v2 12/25] add backup provider example Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [POC storage v2 13/25] Borg plugin Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu-server v2 14/25] move nbd_stop helper to QMPHelpers module Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu-server v2 15/25] backup: move cleanup of fleecing images to cleanup method Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu-server v2 16/25] backup: cleanup: check if VM is running before issuing QMP commands Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu-server v2 17/25] backup: keep track of block-node size instead of volume size Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC qemu-server v2 18/25] backup: allow adding fleecing images also for EFI and TPM Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC qemu-server v2 19/25] backup: implement backup for external providers Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [PATCH qemu-server v2 20/25] restore: die early when there is no size for a device Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC qemu-server v2 21/25] backup: implement restore for external providers Fiona Ebner
2024-09-12 12:44 ` Fabian Grünbichler
2024-09-12 13:32 ` Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC container v2 22/25] backup: implement backup " Fiona Ebner
2024-09-12 12:43 ` Fabian Grünbichler
2024-09-12 13:38 ` Fiona Ebner
2024-09-13 6:19 ` Fabian Grünbichler
2024-09-16 11:40 ` Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC container v2 23/25] backup: implement restore " Fiona Ebner
2024-09-12 12:43 ` Fabian Grünbichler [this message]
2024-09-12 13:56 ` Fiona Ebner
2024-09-12 14:08 ` Fiona Ebner
2024-09-13 6:35 ` Fabian Grünbichler
2024-09-13 13:05 ` Fiona Ebner
2024-09-19 9:44 ` Fabian Grünbichler
2024-09-13 6:34 ` Fabian Grünbichler
2024-08-13 13:28 ` [pve-devel] [PATCH manager v2 24/25] ui: backup: also check for backup subtype to classify archive Fiona Ebner
2024-08-13 13:28 ` [pve-devel] [RFC manager v2 25/25] backup: implement backup for external providers Fiona Ebner
2024-09-12 12:43 ` [pve-devel] [RFC qemu/storage/qemu-server/container/manager v2 00/25] backup provider API Fabian Grünbichler
2024-09-12 15:31 ` Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1726144190.zjc530zmr3.astroid@yuna.none \
--to=f.gruenbichler@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox