From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5ED8B663B5 for ; Sat, 25 Jul 2020 18:04:57 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4891711E10 for ; Sat, 25 Jul 2020 18:04:27 +0200 (CEST) Received: from srv.wewi.de (mail.wewi.de [IPv6:2a02:c205:3000:2707::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id AAD0111E03 for ; Sat, 25 Jul 2020 18:04:26 +0200 (CEST) Received: from [10.3.3.6] (p57bbe701.dip0.t-ipconnect.de [87.187.231.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: wwwehner) by srv.wewi.de (Postfix) with ESMTPSA id 863448032D for ; Sat, 25 Jul 2020 18:04:19 +0200 (CEST) To: PVE User List References: <8e7395ee-c3ab-03fc-aa2d-9bf2e0781375@hw.wewi.de> <20200630102125.nnrf3okkiwtdmojw@zeha.at> From: Hermann Message-ID: Date: Sat, 25 Jul 2020 18:04:19 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20200630102125.nnrf3okkiwtdmojw@zeha.at> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-SPAM-LEVEL: Spam detection results: 0 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods NICE_REPLY_A -0.35 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Subject: Re: [PVE-User] FC-Luns only local devices? X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jul 2020 16:04:57 -0000 Hello Chris, (Sorry for not answering to the list, PM was sent erroneously.) Thank you for the eye-opener. I managed to configure multipath and can see all the luns and was able to create physical volumes and so on. I assume it is correct to configure LVM as shared, because proxmox itself handles the locking. At least a test with an ubuntu-vm went well. Only problem was that I could not stop the vm from the GUI, because a lock could not be aquired. Could you explain in a short sentence, why you avoid partitioning? I remember running into difficulties with an image I had created this way in another setup. I could not change the size when it was necessary an had to delete and recreate everything, which was quite painful, because I had to move around a TB of Files. Have a nice weekend, Hermann. Am 30.06.20 um 12:21 schrieb Chris Hofstaedtler | Deduktiva: > Hi Hermann , > > * Hermann [200630 10:51]: >> I would really appreciate being steered in the right direction as to the >> connection of Fibre-Channel-Luns in Proxmox. >> >> As far as I can see, FC-LUNs only appear als local blockdevices in PVE. >> If I have several LWL-Cables between my Cluster and these bloody >> expensive Storages, do I have to set up multipath manually in debian? > With most storages you need to configure multipath itself manually, > with the settings your storage vendor hands you. > > Our setup for this is: > > 1. Manual multipath setup, we tend to enable find_multipaths "smart" > to avoid configuring all WWIDs everywhere and so on. > > 2. The LVM PVs go directly on the mpathXX devices (no partitioning). > > 3. One VG per mpath device. The VGs are then seen by Proxmox just > like always. > > You have to take great care when removing block devices again, so > all PVE nodes release the VGs, PVs, all underlying device mapper > devices, and remove the physical sdXX devices, before removing the > exports from the storage side. > Often it's easier to reboot, and during the reboot fence access to > the to-be-removed LUN for the currently rebooting host. > > Chris >