Re: [PVE-User] Check existing logical volume on iscsi before vm creation

Re: [PVE-User] Check existing logical volume on iscsi before vm creation

[Dominik Csapak]

Hi,


On 8/31/20 12:26 PM, Amin Vakil via pve-user wrote:
> Proxmox doesn't check existing logical volume on iscsi if the vmid isn't
> present.
> 
> We have multiple proxmoxs which aren't clustered in a location and they
> all use the same nas storage which we have created an iscsi on it.

this is always a bad idea, because the different installations do not
communicate and do not know about the others, so
there is nothing that tells one installation/cluster that some
disk belongs to another installation.

do not let multiple clusters access the same storage
(see also https://pve.proxmox.com/wiki/Storage for that info)

if that happens and you have vmid collisions, every cluster thinks
that all disks belong to its own vm (which obviously does not work)

this is particularly bad for e.g. lvm, since the way we lock
the storage is based around a cluster so multiple lvm operations
from multiple cluster will conflict with each other

> 
> There was a vm with vmid 100 on prox1 which its disk was on iscsi storage.
> 
> I created another vm with vmid 100 on another prox on iscsi and proxmox
> didn't complain about it, therefore vm with vmid 100 disk on prox1 got
> corrupted.


> 
> Shouldn't proxmox check existing logical volumes before creating,
> migrating, moving disk to iscsi storage?

btw. afaics from your config, this has nothing to do with
iscsi, but the actual storage is lvm?

or do you use the iscsi luns directly?

this should not happen if you have selected the same vgname
since the second installation should have listed the disks and
created a new one

can you post both storage cfgs from both installations?
(and tell us which is which?)

if you used the luns directly, there is no mechanism to
limit the use of a disk at all, and the admin is responsible
for handling the assignment

you can even assign the same lun to multiple vms in the same cluster
(there are use cases for this, but needs special file systems, e.g. 
ocfs2 or gfs2)

which is why most users choose to use lvm over iscsi, it
is then managed (and locked) by our cluster mechanism,
but only in a single cluster

> 
> cat /etc/pve/storage.cfg
> ...
> iscsi: nas02iscsi
> 	portal a.b.c.d
> 	target iqn.2004-04.com.qnap:ts-431x2:iscsi.proxmox.36fa44
> 	content images
> 
> lvm: nas02iscsilvm
> 	vgname vg_nas02
> 	content rootdir,images
> 	shared 1
> 
> qm config 100
> 
> ...
> scsi0:
> nas02iscsilvm:vm-100-disk-0,backup=0,cache=writeback,discard=on,size=300G
> ...
> 
> And guess what, backup wasn't enabled on the disk, so we don't have any
> backup, too.
> 
> P.S.
> I'm having my worst day in office becuase of this corrupted vm.