From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 4D289697B9 for ; Mon, 31 Aug 2020 16:01:58 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 3964C15015 for ; Mon, 31 Aug 2020 16:01:58 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 56DA715005 for ; Mon, 31 Aug 2020 16:01:56 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 1C21A44503 for ; Mon, 31 Aug 2020 16:01:56 +0200 (CEST) To: pve-user@lists.proxmox.com References: From: Dominik Csapak Message-ID: Date: Mon, 31 Aug 2020 16:01:55 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:80.0) Gecko/20100101 Thunderbird/80.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.739 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.207 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [PVE-User] Check existing logical volume on iscsi before vm creation X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Aug 2020 14:01:58 -0000 Hi, On 8/31/20 12:26 PM, Amin Vakil via pve-user wrote: > Proxmox doesn't check existing logical volume on iscsi if the vmid isn't > present. > > We have multiple proxmoxs which aren't clustered in a location and they > all use the same nas storage which we have created an iscsi on it. this is always a bad idea, because the different installations do not communicate and do not know about the others, so there is nothing that tells one installation/cluster that some disk belongs to another installation. do not let multiple clusters access the same storage (see also https://pve.proxmox.com/wiki/Storage for that info) if that happens and you have vmid collisions, every cluster thinks that all disks belong to its own vm (which obviously does not work) this is particularly bad for e.g. lvm, since the way we lock the storage is based around a cluster so multiple lvm operations from multiple cluster will conflict with each other > > There was a vm with vmid 100 on prox1 which its disk was on iscsi storage. > > I created another vm with vmid 100 on another prox on iscsi and proxmox > didn't complain about it, therefore vm with vmid 100 disk on prox1 got > corrupted. > > Shouldn't proxmox check existing logical volumes before creating, > migrating, moving disk to iscsi storage? btw. afaics from your config, this has nothing to do with iscsi, but the actual storage is lvm? or do you use the iscsi luns directly? this should not happen if you have selected the same vgname since the second installation should have listed the disks and created a new one can you post both storage cfgs from both installations? (and tell us which is which?) if you used the luns directly, there is no mechanism to limit the use of a disk at all, and the admin is responsible for handling the assignment you can even assign the same lun to multiple vms in the same cluster (there are use cases for this, but needs special file systems, e.g. ocfs2 or gfs2) which is why most users choose to use lvm over iscsi, it is then managed (and locked) by our cluster mechanism, but only in a single cluster > > cat /etc/pve/storage.cfg > ... > iscsi: nas02iscsi > portal a.b.c.d > target iqn.2004-04.com.qnap:ts-431x2:iscsi.proxmox.36fa44 > content images > > lvm: nas02iscsilvm > vgname vg_nas02 > content rootdir,images > shared 1 > > qm config 100 > > ... > scsi0: > nas02iscsilvm:vm-100-disk-0,backup=0,cache=writeback,discard=on,size=300G > ... > > And guess what, backup wasn't enabled on the disk, so we don't have any > backup, too. > > P.S. > I'm having my worst day in office becuase of this corrupted vm.