From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 032B01FF18C
for <inbox@lore.proxmox.com>; Mon, 12 May 2025 16:36:33 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 7EAEF7B25;
Mon, 12 May 2025 16:36:52 +0200 (CEST)
Message-ID: <73e9ad92-622a-4f16-ad6d-b28acc992ac9@proxmox.com>
Date: Mon, 12 May 2025 16:36:18 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Fiona Ebner <f.ebner@proxmox.com>
To: "DERUMIER, Alexandre" <alexandre.derumier@groupe-cyllene.com>,
"pve-devel@lists.proxmox.com" <pve-devel@lists.proxmox.com>
References: <20250509141532.111458-1-f.ebner@proxmox.com>
<20250509141532.111458-2-f.ebner@proxmox.com>
<dfc78aa17b9c1c8496fa74cb6e6d2517337b65c0.camel@groupe-cyllene.com>
<214d7881-9fc3-4713-ac21-40f35199672f@proxmox.com>
<330ddb6da2469b425acda6ceb9cdaf5a510a854f.camel@groupe-cyllene.com>
Content-Language: en-US
In-Reply-To: <330ddb6da2469b425acda6ceb9cdaf5a510a854f.camel@groupe-cyllene.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.035 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [proxmox.com]
Subject: Re: [pve-devel] [RFC qemu 1/1] block/rbd: add @keyring-file option
to BlockdevOptionsRbd
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Am 12.05.25 um 15:39 schrieb DERUMIER, Alexandre:
> Am 12.05.25 um 12:57 schrieb DERUMIER, Alexandre:
>> for blockdev, do we still use a ceph config file in /var/run for
>> potential others rbd client options ?
>
>>> Not currently, but we can add that later if we consider it worth it.
>>> We
>>> would need to merge with the storage's already existing ceph.conf and
>>> not only write the new options. For now, users can adapt their
>>> storage's
>>> ceph.conf as desired.
>
> they still are this rbd_cache_policy for efidisk to fix
> https://bugzilla.proxmox.com/show_bug.cgi?id=3329
>
>
> # SPI flash does lots of read-modify-write OPs, without writeback this
> gets really slow #3329
> if ($path =~ m/^rbd:/) {
> $var_drive_str .= ',cache=writeback';
> $path .= ':rbd_cache_policy=writeback'; # avoid write-around,
> we *need* to cache writes too
> }
>
>
>
> I'm not sure, but maybe it's fixed in qemu , the biggest problem was
> that every single byte write was push to the storage without any buffer
> (so it was pretty slow with rbd crush).
> but maybe it ok now with:
> https://github.com/qemu/qemu/commit/284a7ee2e290e0c9b8cd3ea6164d92386933054f
>
> (I don't have tested it)
Good point!
Unfortunately, it's still very slow without the additional options in
current QEMU 9.2 (i.e. even after that commit).
I suppose this does require us to have a per-drive configuration already.
It's not ideal that qemu-server knows about storage-internal details
though and would need to re-write the Ceph config, I might abstract that
away by passing an additional $hints parameter or something (e.g.
'writeback-cache' => 1, for EFI disk).
We do have a similar situation (but with KRBD):
https://lore.proxmox.com/pve-devel/20241025111304.99680-1-f.weber@proxmox.com/
Replying to stuff from your other mail here too:
> They are interesting rbd client option that we could add later
> https://bugzilla.proxmox.com/show_bug.cgi?id=6290
> crush_location=host:myhost|datacenter:mydc
> read_from_replica=localize
Those can/should simply be set in the storage's ceph.conf, or do they
need to be different per-volume or per-VM?
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel