From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id 50D871FF17C
for <inbox@lore.proxmox.com>; Wed, 14 May 2025 10:23:04 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 43471459;
Wed, 14 May 2025 10:23:24 +0200 (CEST)
Message-ID: <9248a2f1-64be-4c7d-85c8-2cc31dde7133@proxmox.com>
Date: Wed, 14 May 2025 10:22:46 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Fiona Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
References: <20250513133122.94322-1-f.ebner@proxmox.com>
<20250513133122.94322-2-f.ebner@proxmox.com>
Content-Language: en-US
In-Reply-To: <20250513133122.94322-2-f.ebner@proxmox.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.034 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: Re: [pve-devel] [PATCH storage 2/2] rbd plugin: status: explain why
percentage value can be different from Ceph
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Am 13.05.25 um 15:31 schrieb Fiona Ebner:
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
> src/PVE/Storage/RBDPlugin.pm | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/src/PVE/Storage/RBDPlugin.pm b/src/PVE/Storage/RBDPlugin.pm
> index 154fa00..b56f8e4 100644
> --- a/src/PVE/Storage/RBDPlugin.pm
> +++ b/src/PVE/Storage/RBDPlugin.pm
> @@ -703,6 +703,12 @@ sub status {
>
> # max_avail -> max available space for data w/o replication in the pool
> # stored -> amount of user data w/o replication in the pool
> + # NOTE These values are used because they are most natural from a user perspective.
> + # However, the %USED/percent_used value in Ceph is calculated from values before factoring out
> + # replication, namely 'bytes_used / (bytes_used + avail_raw)'. In certain setups, e.g. with LZ4
> + # compression, this percentage can be noticeably different form the percentage
> + # 'stored / (stored + max_avail)' shown in the Proxmox VE CLI/UI. See also src/mon/PGMap.cc from
> + # the Ceph source code, which also mentions that 'stored' is an approximation.
> my $free = $d->{stats}->{max_avail};
> my $used = $d->{stats}->{stored};
> my $total = $used + $free;
Thinking about this again, I don't think continuing to use 'stored' is
best after all, because that is before compression. And this is where
the mismatch really comes from AFAICT. For highly compressible data, the
mismatch between actual usage on the storage and 'stored' can be very
big (in a quick test using the 'yes' command to fill an RBD image, I got
stored = 2 * (used / replication_count)). And here in the storage stats
we are interested in the usage on the storage, not the actual amount of
data written by the user. For ZFS we also don't use 'logicalused', but
'used'.
>From src/osd/osd_types.h:
> int64_t data_stored = 0; ///< Bytes actually stored by the user
> int64_t data_compressed = 0; ///< Bytes stored after compression
> int64_t data_compressed_allocated = 0; ///< Bytes allocated for compressed data
> int64_t data_compressed_original = 0; ///< Bytes that were compressed
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel