Re: [PVE-User] FC-Luns only local devices?

Re: [PVE-User] FC-Luns only local devices?

[Hermann]

Hello Chris,
(Sorry for not answering to the list, PM was sent erroneously.)

Thank you for the eye-opener.

I managed to configure multipath and can see all the luns and was able
to create physical volumes and so on.
I assume it is correct to configure LVM as shared, because proxmox
itself handles the locking. At least a test with an ubuntu-vm went well.
Only problem was that I could not stop the vm from the GUI, because a
lock could not be aquired.

Could you explain in a short sentence, why you avoid partitioning? I
remember running into difficulties with an image I had created this way
in another setup. I could not change the size when it was necessary an
had to delete and recreate everything, which was quite painful, because
I had to move around a TB of Files.

Have a nice weekend,

Hermann.



Am 30.06.20 um 12:21 schrieb Chris Hofstaedtler | Deduktiva:
> Hi Hermann <NoLastName>,
>
> * Hermann <pve@hw.wewi.de> [200630 10:51]:
>> I would really appreciate being steered in the right direction as to the
>> connection of Fibre-Channel-Luns in Proxmox.
>>
>> As far as I can see, FC-LUNs only appear als local blockdevices in PVE.
>> If I have several LWL-Cables between my Cluster and these bloody
>> expensive Storages, do I have to set up multipath manually in debian?
> With most storages you need to configure multipath itself manually,
> with the settings your storage vendor hands you.
>
> Our setup for this is:
>
> 1. Manual multipath setup, we tend to enable find_multipaths "smart"
> to avoid configuring all WWIDs everywhere and so on.
>
> 2. The LVM PVs go directly on the mpathXX devices (no partitioning).
>
> 3. One VG per mpath device. The VGs are then seen by Proxmox just
> like always.
>
> You have to take great care when removing block devices again, so
> all PVE nodes release the VGs, PVs, all underlying device mapper
> devices, and remove the physical sdXX devices, before removing the
> exports from the storage side.
> Often it's easier to reboot, and during the reboot fence access to
> the to-be-removed LUN for the currently rebooting host.
>
> Chris
>

Re: [PVE-User] FC-Luns only local devices?

[Chris Hofstaedtler | Deduktiva]

Hi,

* Hermann <pve@hw.wewi.de> [200725 18:05]:
> Could you explain in a short sentence, why you avoid partitioning? I
> remember running into difficulties with an image I had created this way
> in another setup. I could not change the size when it was necessary an
> had to delete and recreate everything, which was quite painful, because
> I had to move around a TB of Files.

I don't know about images, but we were talking about LVM PVs on top
of LUNs (SCSI disks from a Linux point of view).

Now, if the only thing that goes on the LUN is an LVM PV, why add an
extra layer, an MBR or GPT partitioning layer?

From my experience this adds two things:
1) another size definition, which you have to edit when resizing the LUN
2) another size definition, which is also read and cached by the
Linux kernel, and - at least in the past - was often impossible to
get updated without closing all open handles to the block device.
Effectively, this meant rebooting.

Also, local staff isn't too happy to fiddle with fdisk: delete and
recreate a partition to resize it. Or use parted and find the magic
flags so it doesn't actually do something unexpected.

Chris

-- 
Chris Hofstaedtler / Deduktiva GmbH (FN 418592 b, HG Wien)
www.deduktiva.com / +43 1 353 1707