From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 810FD63B94 for ; Thu, 27 Jan 2022 09:21:26 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 761191F322 for ; Thu, 27 Jan 2022 09:21:26 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 24AC11F316 for ; Thu, 27 Jan 2022 09:21:25 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id EEAFA41808 for ; Thu, 27 Jan 2022 09:21:24 +0100 (CET) Message-ID: <3dc0288b-9fad-669a-9076-3e011da6e575@proxmox.com> Date: Thu, 27 Jan 2022 09:21:19 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.4.1 Content-Language: en-US To: =?UTF-8?Q?Fabian_Gr=c3=bcnbichler?= , pve-devel@lists.proxmox.com References: <20220113100831.34113-1-f.ebner@proxmox.com> <20220113100831.34113-8-f.ebner@proxmox.com> <1643200113.0ad0cpn1af.astroid@nora.none> From: Fabian Ebner In-Reply-To: <1643200113.0ad0cpn1af.astroid@nora.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.016 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [RFC v10 qemu-server 6/7] api: support VM disk import X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2022 08:21:26 -0000 Am 26.01.22 um 13:42 schrieb Fabian Grünbichler: > On January 26, 2022 12:40 pm, Fabian Ebner wrote: >> Am 13.01.22 um 11:08 schrieb Fabian Ebner: >>> + >>> + if (my $source = delete $disk->{'import-from'}) { >> >> I'm adding a comment here in v11, because otherwise it's not clear where >> volume activation happens: >> + # abs_filesystem_path also calls activate_volume when >> $source is a volid >> >> I'm also adding "The source should not be actively used by another >> process!" to the description of the import-from parameter in v11. > > sounds good > >> >>> + $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); >> >> But there are a couple of issues here: >> >> 1. There's no protection against using a source volume that's actively >> used by a guest/other operation. While it's not possible to detect in >> general, I wonder if we should behave more like a full clone and lock >> the owning VM? >> >> 1a. we could check if the volume is referenced in the config/snapshots, >> but migration picks up everything, so it might be preferable not to. >> >> 1b. the volume might be configured in a VM that doesn't own it... >> >> 2. Related: avoiding concurrent activation of volumes on a shared LVM. >> >> 3. Related: cannot deactivate any volumes as the might be used by >> something else. >> >> 4. abs_filesystem_path does not work for RBD when krbd=0, because the >> plugin produces an "rbd:XYZ" path and the -f || -b check doesn't like >> that. But full clone does work, passing the volid to qemu_img_convert >> and that's likely what we should do here as well, when we are dealing >> with an existing volid rather than an absolute path. >> >> 5. Add your own ;) >> >> TL;DR: I'd like to behave much more like full clone, when we are dealing >> with a volid rather than an absolute path. > > yeah. it sounds to me like we could do most of that properly by just > (iff import source is a volume) check whether the owning VM resides on > the current node, and lock if so (and fail if not?). not sure whether > we'd want to require it to be stopped as well if the volume is > referenced in the current config? For full clone, we use qemu_drive_mirror if the VM is running (and not cloning from a snapshot). I'll think about re-using clone_disk(), then having a running VM shouldn't be a problem either. > for full clones we pass the 'running' > state to the storage layer's volume_has_feature - but that seems to not > use the information in any way? Yes, seems like it was a intended for future use, but was never actually required (well, some external plugin might be using it). > > that way we can skip deactivation altogether (it's only relevant for > shared storages that require it for migration, and by locking the owning > VM and having a requirement for it to be on the same node at import-time, > no migration can happen in parallel anyway..). or we could deactivate if > an owning VM exists and is not running, like we do at the end of full > clones. Sounds good. We actually only deactivate at the end of full clone if the clone was to a different target node. Since the new config moves to a different node then, deactivating the cloned volumes is required of course, but I /think/ deactivating the source volumes is actually optional (why should it depend on whether there's a target or not?). > > 1b is 'all bets are off' territory anyway IMHO - there is no sane way to > handle all the edge cases.. > >> >>> + my $src_size = PVE::Storage::file_size_info($source); >>> + die "Could not get file size of $source" if !defined($src_size); >>> + >>> + my (undef, $dst_volid) = PVE::QemuServer::ImportDisk::do_import( >>