From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 17F147DBEA for ; Tue, 9 Nov 2021 15:55:52 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E2525F86B for ; Tue, 9 Nov 2021 15:55:51 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 878D8F5A4 for ; Tue, 9 Nov 2021 15:55:43 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 6660A42D3E for ; Tue, 9 Nov 2021 15:55:43 +0100 (CET) From: Aaron Lauterer To: pve-devel@lists.proxmox.com Date: Tue, 9 Nov 2021 15:55:32 +0100 Message-Id: <20211109145540.1422104-2-a.lauterer@proxmox.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211109145540.1422104-1-a.lauterer@proxmox.com> References: <20211109145540.1422104-1-a.lauterer@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.090 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_SHORT 0.001 Use of a URL Shortener for very short URL SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [plugin.pm, lvmplugin.pm, lvmthinplugin.pm, rbdplugin.pm, gnu.org, zfspoolplugin.pm, storage.pm] Subject: [pve-devel] [PATCH v5 storage 1/9] add disk rename feature X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Nov 2021 14:55:52 -0000 Functionality has been added for the following storage types: * directory ones, based on the default implementation: * directory * NFS * CIFS * gluster * ZFS * (thin) LVM * Ceph A new feature `rename` has been introduced to mark which storage plugin supports the feature. Version API and AGE have been bumped. Signed-off-by: Aaron Lauterer --- v4 -> v5: Since the previous first patch which added the 'find_free_volname' on a Storage.pm and Plugin.pm level was completely dropped, the 'rename_volume' functions have gotten a little more logic to check if a target_volname has been given or if one needs to get requested from 'find_free_diskname'. Since 'find_free_diskname' does not handle VMID subdirectories, the Plugin.pm also adds them now where needed. This was previously handled by the now gone 'find_free_volname'. v3 -> v4: * add notes to ApiChangeLog * fix style nits v2 -> v3: * dropped exists() check * fixed base image handling * fixed smaller code style issues v1 -> v2: * many small fixes and improvements * rename_volume now accepts $source_volname instead of $source_volid helps us to avoid to parse the volid a second time rfc -> v1: * reduced number of parameters to minimum needed, plugins infer needed information themselves * added storage locking and checking if volume already exists * parse target_volname prior to renaming to check if valid old dedicated API endpoint -> rfc: only do rename now but the rename function handles templates and returns the new volid as this can be differently handled on some storages. ApiChangeLog | 10 ++++++++++ PVE/Storage.pm | 25 ++++++++++++++++++++++-- PVE/Storage/LVMPlugin.pm | 34 ++++++++++++++++++++++++++++++++ PVE/Storage/LvmThinPlugin.pm | 1 + PVE/Storage/Plugin.pm | 38 ++++++++++++++++++++++++++++++++++++ PVE/Storage/RBDPlugin.pm | 34 ++++++++++++++++++++++++++++++++ PVE/Storage/ZFSPoolPlugin.pm | 31 +++++++++++++++++++++++++++++ 7 files changed, 171 insertions(+), 2 deletions(-) diff --git a/ApiChangeLog b/ApiChangeLog index 8c119c5..5b572e3 100644 --- a/ApiChangeLog +++ b/ApiChangeLog @@ -6,6 +6,16 @@ without breaking anything unaware of it.) Future changes should be documented in here. +## Version 10: (AGE 1): + +* a new `find_free_volname` method has been added + It exposes the functionality to request a new, not yet used, volname for a storage + to outside callers + +* a new `rename_volume` method has been added + Storage plugins with rename support need to enable the `rename` feature flag; e.g. in the + `volume_has_feature` method. + ## Version 9: (AGE resets to 0): * volume_import_formats gets a new parameter *inserted*: diff --git a/PVE/Storage.pm b/PVE/Storage.pm index 71d6ad7..7c2ceab 100755 --- a/PVE/Storage.pm +++ b/PVE/Storage.pm @@ -41,11 +41,11 @@ use PVE::Storage::PBSPlugin; use PVE::Storage::BTRFSPlugin; # Storage API version. Increment it on changes in storage API interface. -use constant APIVER => 9; +use constant APIVER => 10; # Age is the number of versions we're backward compatible with. # This is like having 'current=APIVER' and age='APIAGE' in libtool, # see https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html -use constant APIAGE => 0; +use constant APIAGE => 1; # load standard plugins PVE::Storage::DirPlugin->register(); @@ -349,6 +349,7 @@ sub volume_snapshot_needs_fsfreeze { # snapshot - taking a snapshot is possible # sparseinit - volume is sparsely initialized # template - conversion to base image is possible +# rename - renaming volumes is possible # $snap - check if the feature is supported for a given snapshot # $running - if the guest owning the volume is running # $opts - hash with further options: @@ -1856,6 +1857,26 @@ sub complete_volume { return $res; } +sub rename_volume { + my ($cfg, $source_volid, $target_vmid, $target_volname) = @_; + + die "no source volid provided\n" if !$source_volid; + die "no target VMID or target volname provided\n" if !$target_vmid && !$target_volname; + + my ($storeid, $source_volname) = parse_volume_id($source_volid); + + activate_storage($cfg, $storeid); + + my $scfg = storage_config($cfg, $storeid); + my $plugin = PVE::Storage::Plugin->lookup($scfg->{type}); + + $target_vmid = ($plugin->parse_volname($source_volname))[3] if !$target_vmid; + + return $plugin->cluster_lock_storage($storeid, $scfg->{shared}, undef, sub { + return $plugin->rename_volume($scfg, $storeid, $source_volname, $target_vmid, $target_volname); + }); +} + # Various io-heavy operations require io/bandwidth limits which can be # configured on multiple levels: The global defaults in datacenter.cfg, and # per-storage overrides. When we want to do a restore from storage A to storage diff --git a/PVE/Storage/LVMPlugin.pm b/PVE/Storage/LVMPlugin.pm index 139d391..40c1613 100644 --- a/PVE/Storage/LVMPlugin.pm +++ b/PVE/Storage/LVMPlugin.pm @@ -339,6 +339,15 @@ sub lvcreate { run_command($cmd, errmsg => "lvcreate '$vg/$name' error"); } +sub lvrename { + my ($vg, $oldname, $newname) = @_; + + run_command( + ['/sbin/lvrename', $vg, $oldname, $newname], + errmsg => "lvrename '${vg}/${oldname}' to '${newname}' error", + ); +} + sub alloc_image { my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_; @@ -584,6 +593,7 @@ sub volume_has_feature { my $features = { copy => { base => 1, current => 1}, + rename => {current => 1}, }; my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = @@ -692,4 +702,28 @@ sub volume_import_write { input => '<&'.fileno($input_fh)); } +sub rename_volume { + my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_; + + my ( + undef, + $source_image, + $source_vmid, + $base_name, + $base_vmid, + undef, + $format + ) = $class->parse_volname($source_volname); + $target_volname = $class->find_free_diskname($storeid, $scfg, $target_vmid, $format) + if !$target_volname; + + my $vg = $scfg->{vgname}; + my $lvs = lvm_list_volumes($vg); + die "target volume '${target_volname}' already exists\n" + if ($lvs->{$vg}->{$target_volname}); + + lvrename($vg, $source_volname, $target_volname); + return "${storeid}:${target_volname}"; +} + 1; diff --git a/PVE/Storage/LvmThinPlugin.pm b/PVE/Storage/LvmThinPlugin.pm index 4ba6f90..c24af22 100644 --- a/PVE/Storage/LvmThinPlugin.pm +++ b/PVE/Storage/LvmThinPlugin.pm @@ -355,6 +355,7 @@ sub volume_has_feature { template => { current => 1}, copy => { base => 1, current => 1, snap => 1}, sparseinit => { base => 1, current => 1}, + rename => {current => 1}, }; my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm index aeb4fff..5350a62 100644 --- a/PVE/Storage/Plugin.pm +++ b/PVE/Storage/Plugin.pm @@ -1008,6 +1008,7 @@ sub volume_has_feature { snap => {qcow2 => 1} }, sparseinit => { base => {qcow2 => 1, raw => 1, vmdk => 1}, current => {qcow2 => 1, raw => 1, vmdk => 1} }, + rename => { current => {qcow2 => 1, raw => 1, vmdk => 1} }, }; # clone_image creates a qcow2 volume @@ -1015,6 +1016,8 @@ sub volume_has_feature { defined($opts->{valid_target_formats}) && !(grep { $_ eq 'qcow2' } @{$opts->{valid_target_formats}}); + return 0 if $feature eq 'rename' && $class->can('api') && $class->api() < 10; + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = $class->parse_volname($volname); @@ -1531,4 +1534,39 @@ sub volume_import_formats { return (); } +sub rename_volume { + my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_; + die "not implemented in storage plugin '$class'\n" if $class->can('api') && $class->api() < 10; + die "no path found\n" if !$scfg->{path}; + + my ( + undef, + $source_image, + $source_vmid, + $base_name, + $base_vmid, + undef, + $format + ) = $class->parse_volname($source_volname); + + $target_volname = $class->find_free_diskname($storeid, $scfg, $target_vmid, $format, 1) + if !$target_volname; + + my $basedir = $class->get_subdir($scfg, 'images'); + + mkpath "${basedir}/${target_vmid}"; + + my $old_path = "${basedir}/${source_vmid}/${source_image}"; + my $new_path = "${basedir}/${target_vmid}/${target_volname}"; + + die "target volume '${target_volname}' already exists\n" if -e $new_path; + + my $base = $base_name ? "${base_vmid}/${base_name}/" : ''; + + rename($old_path, $new_path) || + die "rename '$old_path' to '$new_path' failed - $!\n"; + + return "${storeid}:${base}${target_vmid}/${target_volname}"; +} + 1; diff --git a/PVE/Storage/RBDPlugin.pm b/PVE/Storage/RBDPlugin.pm index 613d32b..2607d25 100644 --- a/PVE/Storage/RBDPlugin.pm +++ b/PVE/Storage/RBDPlugin.pm @@ -742,6 +742,7 @@ sub volume_has_feature { template => { current => 1}, copy => { base => 1, current => 1, snap => 1}, sparseinit => { base => 1, current => 1}, + rename => {current => 1}, }; my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = $class->parse_volname($volname); @@ -757,4 +758,37 @@ sub volume_has_feature { return undef; } +sub rename_volume { + my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_; + + my ( + undef, + $source_image, + $source_vmid, + $base_name, + $base_vmid, + undef, + $format + ) = $class->parse_volname($source_volname); + $target_volname = $class->find_free_diskname($storeid, $scfg, $target_vmid, $format) + if !$target_volname; + + eval { + my $cmd = $rbd_cmd->($scfg, $storeid, 'info', $target_volname); + run_rbd_command($cmd, errmsg => "exist check", quiet => 1); + }; + die "target volume '${target_volname}' already exists\n" if !$@; + + my $cmd = $rbd_cmd->($scfg, $storeid, 'rename', $source_image, $target_volname); + + run_rbd_command( + $cmd, + errmsg => "could not rename image '${source_image}' to '${target_volname}'", + ); + + $base_name = $base_name ? "${base_name}/" : ''; + + return "${storeid}:${base_name}${target_volname}"; +} + 1; diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm index 660b3d9..83220ec 100644 --- a/PVE/Storage/ZFSPoolPlugin.pm +++ b/PVE/Storage/ZFSPoolPlugin.pm @@ -690,6 +690,7 @@ sub volume_has_feature { copy => { base => 1, current => 1}, sparseinit => { base => 1, current => 1}, replicate => { base => 1, current => 1}, + rename => {current => 1}, }; my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = @@ -792,4 +793,34 @@ sub volume_import_formats { return $class->volume_export_formats($scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots); } +sub rename_volume { + my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_; + + my ( + undef, + $source_image, + $source_vmid, + $base_name, + $base_vmid, + undef, + $format + ) = $class->parse_volname($source_volname); + $target_volname = $class->find_free_diskname($storeid, $scfg, $target_vmid, $format) + if !$target_volname; + + my $pool = $scfg->{pool}; + my $source_zfspath = "${pool}/${source_image}"; + my $target_zfspath = "${pool}/${target_volname}"; + + my $exists = 0 == run_command(['zfs', 'get', '-H', 'name', $target_zfspath], + noerr => 1, quiet => 1); + die "target volume '${target_volname}' already exists\n" if $exists; + + $class->zfs_request($scfg, 5, 'rename', ${source_zfspath}, ${target_zfspath}); + + $base_name = $base_name ? "${base_name}/" : ''; + + return "${storeid}:${base_name}${target_volname}"; +} + 1; -- 2.30.2