From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <c.ebner@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id A4738906F6
for <pve-devel@lists.proxmox.com>; Thu, 9 Mar 2023 10:41:41 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 816EB891D
for <pve-devel@lists.proxmox.com>; Thu, 9 Mar 2023 10:41:41 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Thu, 9 Mar 2023 10:41:40 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 155E342CF5
for <pve-devel@lists.proxmox.com>; Thu, 9 Mar 2023 10:41:40 +0100 (CET)
From: Christian Ebner <c.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Thu, 9 Mar 2023 10:41:23 +0100
Message-Id: <20230309094123.564813-1-c.ebner@proxmox.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.119 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
POISEN_SPAM_PILL 0.1 Meta: its spam
POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes
POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH v3 storage] api: fix get content call for volumes
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Thu, 09 Mar 2023 09:41:41 -0000
`pvesh get /nodes/{node}/storage/{storage}/content/{volume}` failed for
several storage types, because the respective storage plugins returned
only the volumes `size` on `volume_size_info` calls, while also the format
is required.
This patch fixes the issue by returning also `format` and where possible `used`.
The issue was reported in the forum:
https://forum.proxmox.com/threads/pvesh-get-nodes-node-storage-storage-content-volume-returns-error.123747/
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since v1:
* Remove errous check for $used being set, rely on fallback to 0 if undef
* Return `parent` for RBD and ZFS
* Return `used` for ZFS
Changes since v2:
* Add conditional call to `rbd du` to get `used` for RBD based volumes
* Get `usedbydataset` instead of `used` for ZFS volumes, refactor
zfs_get_properties call
Note: The file_size_info for iscsi direct targets unfortunately does
not return anything usefull for `used` storage size, so it stayed as
is.
PVE/Storage/ISCSIDirectPlugin.pm | 2 +-
PVE/Storage/RBDPlugin.pm | 44 ++++++++++++++++++++++++++++++--
PVE/Storage/ZFSPoolPlugin.pm | 11 +++++---
3 files changed, 50 insertions(+), 7 deletions(-)
diff --git a/PVE/Storage/ISCSIDirectPlugin.pm b/PVE/Storage/ISCSIDirectPlugin.pm
index 9777969..eb329d4 100644
--- a/PVE/Storage/ISCSIDirectPlugin.pm
+++ b/PVE/Storage/ISCSIDirectPlugin.pm
@@ -208,7 +208,7 @@ sub volume_size_info {
my $vollist = iscsi_ls($scfg,$storeid);
my $info = $vollist->{$storeid}->{$volname};
- return $info->{size};
+ return wantarray ? ($info->{size}, 'raw', 0, undef) : $info->{size};
}
sub volume_resize {
diff --git a/PVE/Storage/RBDPlugin.pm b/PVE/Storage/RBDPlugin.pm
index 9047504..35b2372 100644
--- a/PVE/Storage/RBDPlugin.pm
+++ b/PVE/Storage/RBDPlugin.pm
@@ -308,6 +308,45 @@ sub rbd_volume_info {
return $volume->@{qw(size parent format protected features)};
}
+sub rbd_volume_du {
+ my ($scfg, $storeid, $volname) = @_;
+
+ my @options = ('du', $volname, '--format', 'json');
+ my $cmd = $rbd_cmd->($scfg, $storeid, @options);
+
+ my $raw = '';
+ my $parser = sub { $raw .= shift };
+
+ run_rbd_command($cmd, errmsg => "rbd error", errfunc => sub {}, outfunc => $parser);
+
+ my $volume;
+ if ($raw eq '') {
+ $volume = {};
+ } elsif ($raw =~ m/^(\{.*\})$/s) { # untaint
+ $volume = JSON::decode_json($1);
+ } else {
+ die "got unexpected data from rbd du: '$raw'\n";
+ }
+
+ if (!defined($volume->{images})) {
+ die "got no images from rbd du\n";
+ }
+
+ # `rbd du` returns array of images for name matching `volname`,
+ # including snapshots.
+ my $images = $volume->{images};
+ foreach my $image (@$images) {
+ next if defined($image->{snapshot});
+ next if !defined($image->{used_size}) || !defined($image->{name});
+
+ # Return `used_size` of first volume with matching name which
+ # is not a snapshot.
+ return $image->{used_size} if $image->{name} eq $volname;
+ }
+
+ die "got no matching image from rbd du\n";
+}
+
# Configuration
sub type {
@@ -729,8 +768,9 @@ sub volume_size_info {
my ($class, $scfg, $storeid, $volname, $timeout) = @_;
my ($vtype, $name, $vmid) = $class->parse_volname($volname);
- my ($size, undef) = rbd_volume_info($scfg, $storeid, $name);
- return $size;
+ my ($size, $parent) = rbd_volume_info($scfg, $storeid, $name);
+ my $used = wantarray ? rbd_volume_du($scfg, $storeid, $name) : 0;
+ return wantarray ? ($size, 'raw', $used, $parent) : $size;
}
sub volume_resize {
diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm
index 9fbd149..54dd2ae 100644
--- a/PVE/Storage/ZFSPoolPlugin.pm
+++ b/PVE/Storage/ZFSPoolPlugin.pm
@@ -446,13 +446,16 @@ sub status {
sub volume_size_info {
my ($class, $scfg, $storeid, $volname, $timeout) = @_;
- my (undef, $vname, undef, undef, undef, undef, $format) =
+ my (undef, $vname, undef, $parent, undef, undef, $format) =
$class->parse_volname($volname);
my $attr = $format eq 'subvol' ? 'refquota' : 'volsize';
- my $value = $class->zfs_get_properties($scfg, $attr, "$scfg->{pool}/$vname");
- if ($value =~ /^(\d+)$/) {
- return $1;
+ my ($size, $used) = $class->zfs_get_properties($scfg, "$attr,usedbydataset", "$scfg->{pool}/$vname");
+
+ $used = ($used =~ /^(\d+)$/) ? $1 : 0;
+
+ if ($size =~ /^(\d+)$/) {
+ return wantarray ? ($1, $format, $used, $parent) : $1;
}
die "Could not get zfs volume size\n";
--
2.30.2