From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 978F21FF142 for ; Fri, 03 Jul 2026 19:29:15 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 0C4C5213A3; Fri, 03 Jul 2026 19:29:14 +0200 (CEST) Message-ID: Date: Fri, 3 Jul 2026 19:29:09 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: superseded: [PATCH storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview To: Dietmar Maurer References: <20260626121000.2095591-1-t.lamprecht@proxmox.com> <8873d3a2-0ad5-4ed4-9ef7-7c29c67ec62b@proxmox.com> Content-Language: en-US From: Thomas Lamprecht In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1783099741845 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.018 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: RK5T7VAL5TR7KFRB6RX7LJI2DNUPWEDY X-Message-ID-Hash: RK5T7VAL5TR7KFRB6RX7LJI2DNUPWEDY X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: pve-devel@lists.proxmox.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Am 03.07.26 um 18:13 schrieb Dietmar Maurer: > I just want to mention that NVMeOF has internal multipathing, so adding > a new storage plugin for multipathing may adds some confusion (iSCSI > need it, NVMe don't). Yeah, I agree. That is a big part of why the type is marked as provisional. I actually started to plan out the layered variant already before submitting the v1: basically the transport plugins get a multipath option (iSCSI first) and the path handling then lives behind the transport, dm-multipath for SCSI and native ANA for (future) NVMe-oF, so users never have to pick a separate multipath storage at all. But as that is quite a bit more work I basically just sent out v1 with the simple standalone type to get early feedback on the whole direction, and v2 to address the most grave feedback v1 got. The cluster config and the health matrix could stay as they currently are either way, they do not really care about the transport. A small standalone provider might only be needed for FC, as IIRC there the LUNs just show up on their own without any transport plugin involved. > And I guess bad things happens if you add such nvme drive to the > multipath storage plugin? Nothing bad will really happen as far as I can tell, it would be just confusing: With native NVMe multipathing (the kernel default, ours has it built-in) multipathd never even sees the single paths, so no dm map can get assembled for such a WWID. One then "just" gets a 'missing' row in the health matrix view I add here, and the storage never shows a volume for it as it only lists actually assembled maps. >>From some basic research it seems like dm-multipath for NVMe would only do something when explicitly booting with the nvme_core.multipath=0 on purpose, and not sure if that's worth catering for. I'll see if I can pick up the other approach on the weekend, but I still think it makes sense to rebase anything from my series here, if we actually want something from it, then on top of your work then in any case.