From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id A4738906F6 for ; Thu, 9 Mar 2023 10:41:41 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 816EB891D for ; Thu, 9 Mar 2023 10:41:41 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Thu, 9 Mar 2023 10:41:40 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 155E342CF5 for ; Thu, 9 Mar 2023 10:41:40 +0100 (CET) From: Christian Ebner To: pve-devel@lists.proxmox.com Date: Thu, 9 Mar 2023 10:41:23 +0100 Message-Id: <20230309094123.564813-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.119 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] [PATCH v3 storage] api: fix get content call for volumes X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Mar 2023 09:41:41 -0000 `pvesh get /nodes/{node}/storage/{storage}/content/{volume}` failed for several storage types, because the respective storage plugins returned only the volumes `size` on `volume_size_info` calls, while also the format is required. This patch fixes the issue by returning also `format` and where possible `used`. The issue was reported in the forum: https://forum.proxmox.com/threads/pvesh-get-nodes-node-storage-storage-content-volume-returns-error.123747/ Signed-off-by: Christian Ebner --- Changes since v1: * Remove errous check for $used being set, rely on fallback to 0 if undef * Return `parent` for RBD and ZFS * Return `used` for ZFS Changes since v2: * Add conditional call to `rbd du` to get `used` for RBD based volumes * Get `usedbydataset` instead of `used` for ZFS volumes, refactor zfs_get_properties call Note: The file_size_info for iscsi direct targets unfortunately does not return anything usefull for `used` storage size, so it stayed as is. PVE/Storage/ISCSIDirectPlugin.pm | 2 +- PVE/Storage/RBDPlugin.pm | 44 ++++++++++++++++++++++++++++++-- PVE/Storage/ZFSPoolPlugin.pm | 11 +++++--- 3 files changed, 50 insertions(+), 7 deletions(-) diff --git a/PVE/Storage/ISCSIDirectPlugin.pm b/PVE/Storage/ISCSIDirectPlugin.pm index 9777969..eb329d4 100644 --- a/PVE/Storage/ISCSIDirectPlugin.pm +++ b/PVE/Storage/ISCSIDirectPlugin.pm @@ -208,7 +208,7 @@ sub volume_size_info { my $vollist = iscsi_ls($scfg,$storeid); my $info = $vollist->{$storeid}->{$volname}; - return $info->{size}; + return wantarray ? ($info->{size}, 'raw', 0, undef) : $info->{size}; } sub volume_resize { diff --git a/PVE/Storage/RBDPlugin.pm b/PVE/Storage/RBDPlugin.pm index 9047504..35b2372 100644 --- a/PVE/Storage/RBDPlugin.pm +++ b/PVE/Storage/RBDPlugin.pm @@ -308,6 +308,45 @@ sub rbd_volume_info { return $volume->@{qw(size parent format protected features)}; } +sub rbd_volume_du { + my ($scfg, $storeid, $volname) = @_; + + my @options = ('du', $volname, '--format', 'json'); + my $cmd = $rbd_cmd->($scfg, $storeid, @options); + + my $raw = ''; + my $parser = sub { $raw .= shift }; + + run_rbd_command($cmd, errmsg => "rbd error", errfunc => sub {}, outfunc => $parser); + + my $volume; + if ($raw eq '') { + $volume = {}; + } elsif ($raw =~ m/^(\{.*\})$/s) { # untaint + $volume = JSON::decode_json($1); + } else { + die "got unexpected data from rbd du: '$raw'\n"; + } + + if (!defined($volume->{images})) { + die "got no images from rbd du\n"; + } + + # `rbd du` returns array of images for name matching `volname`, + # including snapshots. + my $images = $volume->{images}; + foreach my $image (@$images) { + next if defined($image->{snapshot}); + next if !defined($image->{used_size}) || !defined($image->{name}); + + # Return `used_size` of first volume with matching name which + # is not a snapshot. + return $image->{used_size} if $image->{name} eq $volname; + } + + die "got no matching image from rbd du\n"; +} + # Configuration sub type { @@ -729,8 +768,9 @@ sub volume_size_info { my ($class, $scfg, $storeid, $volname, $timeout) = @_; my ($vtype, $name, $vmid) = $class->parse_volname($volname); - my ($size, undef) = rbd_volume_info($scfg, $storeid, $name); - return $size; + my ($size, $parent) = rbd_volume_info($scfg, $storeid, $name); + my $used = wantarray ? rbd_volume_du($scfg, $storeid, $name) : 0; + return wantarray ? ($size, 'raw', $used, $parent) : $size; } sub volume_resize { diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm index 9fbd149..54dd2ae 100644 --- a/PVE/Storage/ZFSPoolPlugin.pm +++ b/PVE/Storage/ZFSPoolPlugin.pm @@ -446,13 +446,16 @@ sub status { sub volume_size_info { my ($class, $scfg, $storeid, $volname, $timeout) = @_; - my (undef, $vname, undef, undef, undef, undef, $format) = + my (undef, $vname, undef, $parent, undef, undef, $format) = $class->parse_volname($volname); my $attr = $format eq 'subvol' ? 'refquota' : 'volsize'; - my $value = $class->zfs_get_properties($scfg, $attr, "$scfg->{pool}/$vname"); - if ($value =~ /^(\d+)$/) { - return $1; + my ($size, $used) = $class->zfs_get_properties($scfg, "$attr,usedbydataset", "$scfg->{pool}/$vname"); + + $used = ($used =~ /^(\d+)$/) ? $1 : 0; + + if ($size =~ /^(\d+)$/) { + return wantarray ? ($1, $format, $used, $parent) : $1; } die "Could not get zfs volume size\n"; -- 2.30.2