From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 1FF917982D for ; Wed, 27 Oct 2021 12:34:14 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 1DB2A11149 for ; Wed, 27 Oct 2021 12:34:14 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 9D16A1113A for ; Wed, 27 Oct 2021 12:34:12 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 7696845F43 for ; Wed, 27 Oct 2021 12:34:12 +0200 (CEST) To: pve-devel@lists.proxmox.com, Aaron Lauterer References: <20211021113030.2649985-1-a.lauterer@proxmox.com> <20211021113030.2649985-3-a.lauterer@proxmox.com> From: Fabian Ebner Message-ID: Date: Wed, 27 Oct 2021 12:34:10 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: <20211021113030.2649985-3-a.lauterer@proxmox.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.363 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_SHORT 0.001 Use of a URL Shortener for very short URL NICE_REPLY_A -0.215 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [lvmthinplugin.pm, zfspoolplugin.pm, lvmplugin.pm, gnu.org, storage.pm, rbdplugin.pm, plugin.pm] Subject: Re: [pve-devel] [PATCH v3 storage 2/9] add disk rename feature X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Oct 2021 10:34:14 -0000 Am 21.10.21 um 13:30 schrieb Aaron Lauterer: > Functionality has been added for the following storage types: > > * directory ones, based on the default implementation: > * directory > * NFS > * CIFS > * gluster > * ZFS > * (thin) LVM > * Ceph > > A new feature `rename` has been introduced to mark which storage > plugin supports the feature. > > Version API and AGE have been bumped. There also is a rather new ApiChangeLog file, where the changes should be documented. Can also be done by the release team or in a follow-up. > > The storage gets locked and each plugin checks if the target volume > already exists prior renaming. > This is done because there could be a race condition from the time the > external caller requests a new free disk name to the time the volume is > actually renamed. > > Signed-off-by: Aaron Lauterer > --- > v2 -> v3: > * dropped exists() check > * fixed base image handling > * fixed smaller code style issues > > v1 -> v2: > * many small fixes and improvements > * rename_volume now accepts $source_volname instead of $source_volid > helps us to avoid to parse the volid a second time > > rfc -> v1: > * reduced number of parameters to minimum needed, plugins infer needed > information themselves > * added storage locking and checking if volume already exists > * parse target_volname prior to renaming to check if valid > > old dedicated API endpoint -> rfc: > only do rename now but the rename function handles templates and returns > the new volid as this can be differently handled on some storages. > > PVE/Storage.pm | 20 +++++++++++++++-- > PVE/Storage/LVMPlugin.pm | 24 ++++++++++++++++++++ > PVE/Storage/LvmThinPlugin.pm | 1 + > PVE/Storage/Plugin.pm | 43 ++++++++++++++++++++++++++++++++++++ > PVE/Storage/RBDPlugin.pm | 28 +++++++++++++++++++++++ > PVE/Storage/ZFSPoolPlugin.pm | 23 +++++++++++++++++++ > 6 files changed, 137 insertions(+), 2 deletions(-) > > diff --git a/PVE/Storage.pm b/PVE/Storage.pm > index 4fbeaea..40d0d39 100755 > --- a/PVE/Storage.pm > +++ b/PVE/Storage.pm > @@ -41,11 +41,11 @@ use PVE::Storage::PBSPlugin; > use PVE::Storage::BTRFSPlugin; > > # Storage API version. Increment it on changes in storage API interface. > -use constant APIVER => 9; > +use constant APIVER => 10; > # Age is the number of versions we're backward compatible with. > # This is like having 'current=APIVER' and age='APIAGE' in libtool, > # see https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html > -use constant APIAGE => 0; > +use constant APIAGE => 1; > > # load standard plugins > PVE::Storage::DirPlugin->register(); > @@ -358,6 +358,7 @@ sub volume_snapshot_needs_fsfreeze { > # snapshot - taking a snapshot is possible > # sparseinit - volume is sparsely initialized > # template - conversion to base image is possible > +# rename - renaming volumes is possible > # $snap - check if the feature is supported for a given snapshot > # $running - if the guest owning the volume is running > # $opts - hash with further options: > @@ -1865,6 +1866,21 @@ sub complete_volume { > return $res; > } > > +sub rename_volume { > + my ($cfg, $source_volid, $target_volname) = @_; > + > + my ($storeid, $source_volname) = parse_volume_id($source_volid); > + > + activate_storage($cfg, $storeid); > + > + my $scfg = storage_config($cfg, $storeid); > + my $plugin = PVE::Storage::Plugin->lookup($scfg->{type}); > + > + return $plugin->cluster_lock_storage($storeid, $scfg->{shared}, undef, sub { > + return $plugin->rename_volume($scfg, $storeid, $source_volname, $target_volname); > + }); > +} > + > # Various io-heavy operations require io/bandwidth limits which can be > # configured on multiple levels: The global defaults in datacenter.cfg, and > # per-storage overrides. When we want to do a restore from storage A to storage > diff --git a/PVE/Storage/LVMPlugin.pm b/PVE/Storage/LVMPlugin.pm > index 139d391..f28b818 100644 > --- a/PVE/Storage/LVMPlugin.pm > +++ b/PVE/Storage/LVMPlugin.pm > @@ -339,6 +339,15 @@ sub lvcreate { > run_command($cmd, errmsg => "lvcreate '$vg/$name' error"); > } > > +sub lvrename { > + my ($vg, $oldname, $newname) = @_; > + > + run_command( > + ['/sbin/lvrename', $vg, $oldname, $newname], > + errmsg => "lvrename '${vg}/${oldname}' to '${newname}' error", > + ); > +} > + > sub alloc_image { > my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_; > > @@ -584,6 +593,7 @@ sub volume_has_feature { > > my $features = { > copy => { base => 1, current => 1}, > + rename => {current => 1}, > }; > > my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = > @@ -692,4 +702,18 @@ sub volume_import_write { > input => '<&'.fileno($input_fh)); > } > > +sub rename_volume { > + my ($class, $scfg, $storeid, $source_volname, $target_volname) = @_; > + > + $class->parse_volname($target_volname); > + > + my $vg = $scfg->{vgname}; > + my $lvs = lvm_list_volumes($vg); > + die "target volume '${target_volname}' already exists\n" > + if ($lvs->{$vg}->{$target_volname}); > + > + lvrename($vg, $source_volname, $target_volname); > + return "${storeid}:${target_volname}"; > +} > + > 1; > diff --git a/PVE/Storage/LvmThinPlugin.pm b/PVE/Storage/LvmThinPlugin.pm > index 4ba6f90..c24af22 100644 > --- a/PVE/Storage/LvmThinPlugin.pm > +++ b/PVE/Storage/LvmThinPlugin.pm > @@ -355,6 +355,7 @@ sub volume_has_feature { > template => { current => 1}, > copy => { base => 1, current => 1, snap => 1}, > sparseinit => { base => 1, current => 1}, > + rename => {current => 1}, > }; > > my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = > diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm > index 26465fd..e3e91af 100644 > --- a/PVE/Storage/Plugin.pm > +++ b/PVE/Storage/Plugin.pm > @@ -1019,6 +1019,7 @@ sub volume_has_feature { > snap => {qcow2 => 1} }, > sparseinit => { base => {qcow2 => 1, raw => 1, vmdk => 1}, > current => {qcow2 => 1, raw => 1, vmdk => 1} }, > + rename => { current => {qcow2 => 1, raw => 1, vmdk => 1} }, > }; > > # clone_image creates a qcow2 volume > @@ -1026,6 +1027,14 @@ sub volume_has_feature { > defined($opts->{valid_target_formats}) && > !(grep { $_ eq 'qcow2' } @{$opts->{valid_target_formats}}); > > + if ( > + $feature eq 'rename' > + && $class->can('api') > + && $class->api() < 10 Style nit: expression fits in one line. > + ) { > + return 0; > + } > + > my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = > $class->parse_volname($volname); > > @@ -1542,4 +1551,38 @@ sub volume_import_formats { > return (); > } > > +sub rename_volume { > + my ($class, $scfg, $storeid, $source_volname, $target_volname) = @_; > + die "no path found\n" if (!$scfg->{path}); Style nit: parentheses could be omitted. > + die "not implemented in storage plugin '$class'\n" if $class->can('api') && $class->api() < 10; > + > + my (undef, undef, $target_vmid) = $class->parse_volname($target_volname); > + > + my ( > + undef, > + $source_image, > + $source_vmid, > + $base_name, > + $base_vmid, > + undef, > + $format > + ) = $class->parse_volname($source_volname); > + > + my $basedir = $class->get_subdir($scfg, 'images'); > + > + mkpath "${basedir}/${target_vmid}"; > + > + my $old_path = "${basedir}/${source_vmid}/${source_image}"; > + my $new_path = "${basedir}/${target_volname}"; > + > + die "target volume '${target_volname}' already exists\n" if -e $new_path; > + > + my $base = $base_name ? "${base_vmid}/${base_name}/" : ''; > + > + rename($old_path, $new_path) || > + die "rename '$old_path' to '$new_path' failed - $!\n"; > + > + return "${storeid}:${base}${target_volname}"; > +} > + > 1; > diff --git a/PVE/Storage/RBDPlugin.pm b/PVE/Storage/RBDPlugin.pm > index 4bd43d5..e42b094 100644 > --- a/PVE/Storage/RBDPlugin.pm > +++ b/PVE/Storage/RBDPlugin.pm > @@ -742,6 +742,7 @@ sub volume_has_feature { > template => { current => 1}, > copy => { base => 1, current => 1, snap => 1}, > sparseinit => { base => 1, current => 1}, > + rename => {current => 1}, > }; > > my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = $class->parse_volname($volname); > @@ -757,4 +758,31 @@ sub volume_has_feature { > return undef; > } > > +sub rename_volume { > + my ($class, $scfg, $storeid, $source_volname, $target_volname) = @_; > + > + $class->parse_volname($target_volname); > + > + my (undef, $source_image, $source_vmid, $base_name) = $class->parse_volname($source_volname); > + > + eval { > + my $cmd = $rbd_cmd->($scfg, $storeid, 'info', $target_volname); > + run_rbd_command($cmd, errmsg => "exist check", quiet => 1, noerr => 1); Nit: noerr should not be set here. AFAICT noerr has no effect when there is an errmsg, but it would otherwise prevent run_command (and hence run_rbd_command) from dying which is not what you want? > + }; > + my $err = $@; > + die "target volume '${target_volname}' already exists\n" > + if !$err; Style nit: doesn't need to be three lines ;) Ideally, the existence check would be able to distinguish between "error because it doesn't exist" and some other error, but I suppose that's a very minor issue in practice. Especially, because 'rbd rename' also complains when trying to rename to an existing name. > + > + my $cmd = $rbd_cmd->($scfg, $storeid, 'rename', $source_image, $target_volname); > + > + run_rbd_command( > + $cmd, > + errmsg => "could not rename image '${source_volname}' to '${target_volname}'", Nit: for consistency, use $source_image in the error message too, because $target_volname doesn't ever include the base, but $source_volname can. > + ); > + > + $base_name = $base_name ? "${base_name}/" : ''; > + > + return "${storeid}:${base_name}${target_volname}"; > +} > + > 1; > diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm > index 660b3d9..c5d2e67 100644 > --- a/PVE/Storage/ZFSPoolPlugin.pm > +++ b/PVE/Storage/ZFSPoolPlugin.pm > @@ -690,6 +690,7 @@ sub volume_has_feature { > copy => { base => 1, current => 1}, > sparseinit => { base => 1, current => 1}, > replicate => { base => 1, current => 1}, > + rename => {current => 1}, > }; > > my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = > @@ -792,4 +793,26 @@ sub volume_import_formats { > return $class->volume_export_formats($scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots); > } > > +sub rename_volume { > + my ($class, $scfg, $storeid, $source_volname, $target_volname) = @_; > + > + $class->parse_volname($target_volname); > + > + my (undef, $source_image, $source_vmid, $base_name) = $class->parse_volname($source_volname); > + > + my $pool = $scfg->{pool}; > + my $source_zfspath = "${pool}/${source_image}"; > + my $target_zfspath = "${pool}/${target_volname}"; > + > + my $exists = 0 == run_command(['zfs', 'get', '-H', 'name', $target_zfspath], > + noerr => 1, quiet => 1); > + die "target volume '${target_volname}' already exists\n" if $exists; > + > + $class->zfs_request($scfg, 5, 'rename', ${source_zfspath}, ${target_zfspath}); > + > + $base_name = $base_name ? "${base_name}/" : ''; > + > + return "${storeid}:${base_name}${target_volname}"; > +} > + > 1; >