From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 068076A315 for ; Wed, 16 Mar 2022 10:29:47 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id EEB1925789 for ; Wed, 16 Mar 2022 10:29:16 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 095F725780 for ; Wed, 16 Mar 2022 10:29:16 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id CC9634641A for ; Wed, 16 Mar 2022 10:29:15 +0100 (CET) Message-ID: <834cda55-7463-bb21-eb32-da3d2bc514cd@proxmox.com> Date: Wed, 16 Mar 2022 10:29:08 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.6.2 From: Fabian Ebner To: pve-devel@lists.proxmox.com, =?UTF-8?Q?Fabian_Gr=c3=bcnbichler?= References: <20220309100919.31512-1-f.ebner@proxmox.com> <20220309100919.31512-16-f.ebner@proxmox.com> <1647267065.5tbtb00mlw.astroid@nora.none> Content-Language: en-US In-Reply-To: <1647267065.5tbtb00mlw.astroid@nora.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.122 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: Re: [pve-devel] [PATCH v12 qemu-server 15/16] api: support VM disk import X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Mar 2022 09:29:47 -0000 Am 14.03.22 um 16:54 schrieb Fabian Grünbichler: > On March 9, 2022 11:09 am, Fabian Ebner wrote: ---snip--- >> my $check_drive_param = sub { >> my ($param, $storecfg, $extra_checks) = @_; >> >> for my $opt (sort keys $param->%*) { >> next if !PVE::QemuServer::is_valid_drivename($opt); >> >> - my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); >> + my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}, 1); > > technically belongs into the previous patch, our non-alloc schema is > just tolerant enough because it doesn't look at the volids too closely > and accepts the NEW_DISK_RE syntax as potential existing volid.. > Makes sense. I guess I wanted to keep the other patch minimal, but there's no good reason for that. >> raise_param_exc({ $opt => "unable to parse drive options" }) if !$drive; >> >> + if ($drive->{'import-from'}) { >> + die "'import-from' requires special syntax - use :0,import-from=\n" >> + if $drive->{file} !~ $NEW_DISK_RE || $3 != 0; > > should probably be a param_exc > >> + >> + if ($opt eq 'efidisk0') { >> + for my $required (qw(efitype pre-enrolled-keys)) { >> + die "$opt - need to specify '$required' when using 'import-from'\n" >> + if !defined($drive->{$required}); > > same here > >> + } >> + } elsif ($opt eq 'tpmstate0') { >> + die "$opt - need to specify 'version' when using 'import-from'\n" >> + if !defined($drive->{version}); > > and here > Will change it. >> + } >> + } >> + >> PVE::QemuServer::cleanup_drive_path($opt, $storecfg, $drive); >> >> $extra_checks->($drive) if $extra_checks; >> >> - $param->{$opt} = PVE::QemuServer::print_drive($drive); >> + $param->{$opt} = PVE::QemuServer::print_drive($drive, 1); >> } >> }; >> >> -my $NEW_DISK_RE = qr!^(([^/:\s]+):)?(\d+(\.\d+)?)$!; >> my $check_storage_access = sub { >> my ($rpcenv, $authuser, $storecfg, $vmid, $settings, $default_storage) = @_; >> >> - PVE::QemuConfig->foreach_volume($settings, sub { >> + $foreach_volume_with_alloc->($settings, sub { >> my ($ds, $drive) = @_; >> >> my $isCDROM = PVE::QemuServer::drive_is_cdrom($drive); >> @@ -106,6 +137,20 @@ my $check_storage_access = sub { >> } else { >> PVE::Storage::check_volume_access($rpcenv, $authuser, $storecfg, $vmid, $volid); >> } >> + >> + if (my $src_image = $drive->{'import-from'}) { >> + my $src_vmid; >> + my ($src_storeid) = PVE::Storage::parse_volume_id($src_image, 1); >> + if ($src_storeid) { # PVE-managed volume > > nit, could be > > if (PVE::Storage::parse_volume_id($src_image, 1)) { # PVE-managed > > since we don't actually care about the sid here, and parse_volume_id > will return undef when $noerr is set. > >> + $src_vmid = (PVE::Storage::parse_volname($storecfg, $src_image))[2] > > is there some case where we expect parse_volume_id to work, but the > volume to not have an associated guest? because perl doesn't mind us > accessing the resulting array at arbitrary indices, so this doesn't fail > if $src_vmid is undef.. > Yes, when importing from an iscsi storage (not sure if there's other cases). The check below and $import_from_volid both handle the case where $src_vmid is undef. > these should probably also check some more stuff (at least the volume > type?) - else we get strange errors when attempting to import > non-image-volumes (some of which even have owners, for example backup > archives..), and what exactly gets caught where is basically up to the > storage plugin via parse_volname and volume_has_feature.. Will add a check for vtype. > >> + } >> + >> + if ($src_vmid) { # might be actively used by VM and will be copied via clone_disk() >> + $rpcenv->check($authuser, "/vms/${src_vmid}", ['VM.Clone']); >> + } else { >> + PVE::Storage::check_volume_access($rpcenv, $authuser, $storecfg, $vmid, $src_image); >> + } >> + } >> }); >> >> $rpcenv->check($authuser, "/storage/$settings->{vmstatestorage}", ['Datastore.AllocateSpace']) >> @@ -164,6 +209,87 @@ my $check_storage_access_migrate = sub { >> if !$scfg->{content}->{images}; >> }; >> >> +my $import_from_volid = sub { >> + my ($storecfg, $src_volid, $dest_info, $vollist) = @_; >> + >> + die "cannot import from cloudinit disk\n" >> + if PVE::QemuServer::Drive::drive_is_cloudinit({ file => $src_volid }); >> + >> + my ($src_storeid, $src_volname) = PVE::Storage::parse_volume_id($src_volid); > > technically this is already implied by the sub's name, we checked it > already outside, but we need the store id for the bwlimit below.. > Yes, it's not intended to be a check, although if it does fail something went terribly wrong and it's good that we abort ;) I'll move it closer to where it's actually used and I'll drop the unused $src_volname. ---snip--- >> + if (my $source = delete $disk->{'import-from'}) { >> + my $dst_volid; >> + my ($src_storeid) = PVE::Storage::parse_volume_id($source, 1); >> + >> + if ($src_storeid) { # PVE-managed volume > > same as above applies here as well, $src_storeid is not used here, so > can be shortened. > >> + die "could not get size of $source\n" >> + if !PVE::Storage::volume_size_info($storecfg, $source, 10); > > this could move into $import_from_volid? Will do. ---snip--- >> @@ -242,7 +415,7 @@ my $create_disks = sub { >> } >> }; >> >> - eval { PVE::QemuConfig->foreach_volume($settings, $code); }; >> + eval { $foreach_volume_with_alloc->($settings, $code); }; >> >> # free allocated images on error >> if (my $err = $@) { >> @@ -1285,7 +1458,7 @@ my $update_vm_api = sub { >> >> my $check_drive_perms = sub { >> my ($opt, $val) = @_; >> - my $drive = PVE::QemuServer::parse_drive($opt, $val); >> + my $drive = PVE::QemuServer::parse_drive($opt, $val, 1); > > same applies here (move to previous patch?) > >> # FIXME: cloudinit: CDROM or Disk? >> if (PVE::QemuServer::drive_is_cdrom($drive)) { # CDROM >> $rpcenv->check_vm_perm($authuser, $vmid, undef, ['VM.Config.CDROM']); >> @@ -1391,7 +1564,7 @@ my $update_vm_api = sub { >> # default legacy boot order implies all cdroms anyway >> if (@bootorder) { >> # append new CD drives to bootorder to mark them bootable >> - my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); >> + my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}, 1); > > same > ---snip--- >> @@ -547,7 +566,7 @@ sub drive_is_read_only { >> # [,iothread=on][,serial=serial][,model=model] >> >> sub parse_drive { >> - my ($key, $data) = @_; >> + my ($key, $data, $with_alloc) = @_; > > technically previous patch, same as all the other changes in this file > below this change > Ack.