From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pve-devel-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 3339B1FF16F for <inbox@lore.proxmox.com>; Tue, 15 Apr 2025 16:11:11 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 85DE3A5C8; Tue, 15 Apr 2025 16:11:08 +0200 (CEST) In-Reply-To: <b19fc3b6-563c-4d0e-a766-78dd3bb28804@proxmox.com> Date: Tue, 15 Apr 2025 16:10:58 +0200 References: <mailman.879.1744219341.359.pve-devel@lists.proxmox.com> <b19fc3b6-563c-4d0e-a766-78dd3bb28804@proxmox.com> To: Mira Limbeck <m.limbeck@proxmox.com> MIME-Version: 1.0 Message-ID: <mailman.1019.1744726267.359.pve-devel@lists.proxmox.com> List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Post: <mailto:pve-devel@lists.proxmox.com> From: Timo Veith via pve-devel <pve-devel@lists.proxmox.com> Precedence: list Cc: Timo Veith <timo.veith@uni-tuebingen.de>, Proxmox VE development discussion <pve-devel@lists.proxmox.com> X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> Subject: Re: [pve-devel] iscsi and multipathing Content-Type: multipart/mixed; boundary="===============3308364387960768862==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com> --===============3308364387960768862== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: <timo.veith@uni-tuebingen.de> X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id BC99CD556E for <pve-devel@lists.proxmox.com>; Tue, 15 Apr 2025 16:11:06 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 9D4D9A5A6 for <pve-devel@lists.proxmox.com>; Tue, 15 Apr 2025 16:11:06 +0200 (CEST) Received: from mx03.uni-tuebingen.de (mx03.uni-tuebingen.de [IPv6:2001:7c0:300c:3105::8602:5d5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for <pve-devel@lists.proxmox.com>; Tue, 15 Apr 2025 16:11:05 +0200 (CEST) Received: from smtpclient.apple (u-006-s131.v276.uni-tuebingen.de [134.2.6.131]) by mx03.uni-tuebingen.de (Postfix) with ESMTPSA id DB92120C68EF; Tue, 15 Apr 2025 16:10:58 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.11.0 mx03.uni-tuebingen.de DB92120C68EF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uni-tuebingen.de; s=20211202prod; t=1744726258; bh=qXKOy31iwIE6sBkvbt31YMfCuUhYq8aBmp6VxIuwNOY=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From; b=JawITxgyThnYe5D/Xc5uxYn6F7B40W61F2kDlMGHwtmNB5KN26VeslsGT1poGkazJ c7EJ+eagiar5QhWz9zw9Uqv7DsTOI0BpsF+C4pZ2ZIWfoyibHhagI70SOiel9ABI/w 3pbGc/aPxOW7ajHwjHHxeOJIgdMEfXxlDwzwyg96CvHdotLtlJlkhEAwrt4483Pabg R/KNVRzpn/7SmZymaXMBtdNR00GKYf4Xs55YmJl8sv/NSaFKwXt0WAm9PSk0wits5K opsJUk4XerKknBkVCBAzoiHr67Gfzo+s3rTq+rXnnBMy6jHxOBKbTt6DrbbGbqyeZM BZWdLwbEI95dA== Content-Type: multipart/signed; boundary="Apple-Mail=_25ABFC43-91DA-4942-A984-F8CCDC60B1DE"; protocol="application/pkcs7-signature"; micalg=sha-256 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Re: [pve-devel] iscsi and multipathing From: Timo Veith <timo.veith@uni-tuebingen.de> In-Reply-To: <b19fc3b6-563c-4d0e-a766-78dd3bb28804@proxmox.com> Date: Tue, 15 Apr 2025 16:10:58 +0200 Cc: Proxmox VE development discussion <pve-devel@lists.proxmox.com> Content-Transfer-Encoding: quoted-printable Message-Id: <BE392B97-179A-4168-A3F0-B8ED4EF46907@uni-tuebingen.de> References: <mailman.879.1744219341.359.pve-devel@lists.proxmox.com> <b19fc3b6-563c-4d0e-a766-78dd3bb28804@proxmox.com> To: Mira Limbeck <m.limbeck@proxmox.com> X-Mailer: Apple Mail (2.3826.500.181.1.5) X-SPAM-LEVEL: Spam detection results: 0 AWL 0.388 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com,uni-tuebingen.de,rescan-scsi-bus.sh] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 --Apple-Mail=_25ABFC43-91DA-4942-A984-F8CCDC60B1DE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hello Mira, thank you very much for your reply. > Am 15.04.2025 um 11:09 schrieb Mira Limbeck <m.limbeck@proxmox.com>: >=20 > Hi Timo, >=20 > At the moment I'm working on storage mapping support for iSCSI. > This would allow one to configure different portals on each of the = hosts > that are logically the same storage. >=20 > If you tried setting up a storage via iSCSI where each host can only > access a part of the portals which are announced, you probably noticed > some higher pvestatd update times. > The storage mapping implementation will alleviate those issues. >=20 > Other than that I'm not aware of anyone working on iSCSI improvements = at > the moment. > We do have some open enhancement requests in our bug tracker [0]. One = of > which is yours [1]. =46rom the list [0] you mentioned iSCSI CHAP credentials in the GUI is = something we are interested in too.=20 >=20 > Regarding multipath handling via the GUI there hasn't been much of a > discussion on how we could tackle that yet. It is quite easy to set up > [2] the usual way. I know that it is easy, because otherwise I wouldn=E2=80=99t have been = able to configure it ;) >=20 >=20 > Sorry, I might have missed your bug report previously, so I'll go into = a > bit more detail here. (I'll add that information to the enhancement > request as well) >=20 >> When adding iscsi storage to the data center there could possiblity = to >> do a iscsi discovery multiple times against different portal ips and >> thus get multiple path to a iscsi san. >=20 > That's already the default. For each target we run the discovery on at > least one portal since it should announce all other portals. We = haven't > encountered a setup where that is not the case. I am dealing only with setups that do not announce their portals. I have = to do iscsi discovery for every portal ip address. That are mostly = Infortrend iSCSI SAN systems but also from Huawei. But I think I know = what you mean. Some storage devices give you all portals when you do a = discovery against one of their ip adresses. However, it would be great to have a possibility to enter multiple = portal ip addresses in the web ui. Together with chap credentials. =20 >=20 >> multipathd should be updated with the path to the luns. The user >> would/could only need to have to add vendor specific device configs >> like alua or multibus settings. >=20 > For now that has to be done manually. There exists a multipath.conf > setting that automatically creates a multipath mapping for devices = that > have at least 2 paths available: `find_multipaths yes` [3]. I will test `find_multipaths yes`. If I understand you correctly then = the command `multipath -a <wwid>` will not be necessary. Just like = written in the multipath wiki article [2]. >=20 >> Then when adding a certain disk to a vm, it would be good if it's wwn >> would be displayed instead of the "CH 00 ID0 LUN0" e.g. So it would = be >> easier to identify the right one. >=20 > That would be a nice addition. And shouldn't be too hard to extract = that > information in the ISCSIPlugin and provide it as additional = information > via the API. > That information could also be listed in the `VM Disks` page of iSCSI > storages. > Would you like to tackle that? Are you asking me to provide the code for that?=20 >=20 >> Also when changing lun size would have been grown on the storage = side, >> it would be handy to have a button in pve web gui to "refresh" the >> disk in the vm. The new size should be reflected in the hardware >> details of the vm. And the qemu prozess should be informed of the new >> disk size so the vm would not have to be shutdown and restarted. >=20 > Based on experience, I doubt it would be that easy. Refreshing of the > LUN sizes involves the SAN, the client, multipath and QEMU. There's > always at least one place where it doesn't update even with > `rescan-scsi-bus.sh`, `multipath -r`, etc. > If you have a reliable way to make all sides agree on the new size, > please let us know. Don=E2=80=99t get me wrong, I didn=E2=80=99t meant that it should be = possible to resize a iscsi disk right from the PVE web gui. I meant that = if one has changed the size of a LUN on SAN side with the measures that = are necessary to that there (e.g. with Infortrend you need to login to = the management software there, find the LUN and then resize it) then the = refreshing of that new size could be triggered by a button in the PVE = web gui. When pressing the button an iscsi rescan of the corresponding = iscsi session would have to be done and then a multipath map rescan like = you wrote and eventually a qemu block device refresh. (And/Or the = equvialent for the lxc container)=20 Even if I do all that manually then the size of the LUN in the hardware = details of the vm is not beeing updated.=20 I personally do not know how but at least I know that it is possible in = ovirt/RHV.=20 Regards, Timo >=20 >=20 >=20 > [0] > = https://bugzilla.proxmox.com/buglist.cgi?bug_severity=3Denhancement&list_i= d=3D50969&resolution=3D---&short_desc=3Discsi&short_desc_type=3Dallwordssu= bstr > [1] https://bugzilla.proxmox.com/show_bug.cgi?id=3D6133 > [2] https://pve.proxmox.com/wiki/Multipath > [3] > = https://manpages.debian.org/bookworm/multipath-tools/multipath.conf.5.en.h= tml >=20 --Apple-Mail=_25ABFC43-91DA-4942-A984-F8CCDC60B1DE-- --===============3308364387960768862== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============3308364387960768862==--