Re: [PVE-User] Intel RST / VROC support in latest Proxmox?

[Aaron Lauterer <a.lauterer@proxmox.com>]

On 6/28/23 16:55, Philippe Andersson wrote:
> Hello Aaron,
> 
> Many thanks for your reply, first of all.
> 
> On 27/06/2023 10:33, Aaron Lauterer wrote:
>> AFAIK, VROC has two variants, actual HW RAID and so-called fake RAID. If
>> it is the fake RAID variant, it will be ignored and the single disks
>> should be shown in the installer.
> I don't know whether this "fake RAID vs real RAID" still applies to the
> recent BIOS/UEFI firmwares.
> 
> We recently tested (at work) an HP Z4 Gen5 workstation with 2 NVMEs
> configured in RAID-1 through VROC, and the openSUSE 15.4 we installed
> there had no issue seeing the resulting LUN. Even better, we could use
> regular 'mdadm' to monitor the state of the array (although 'mdadm'
> could not modify its configuration, obviously -- that can only be done
> through the BIOS/UEFI utilities).

And that is fake RAID AFAIU. The BIOS telling the OS how the RAID should be 
configured. On Linux, that usually means MD RAID. On Windows, a special driver 
might be necessary. So in the end, a BIOS configured software RAID running on 
the OS level.

On a real RAID, the OS would only see a single disk and would not have any idea 
about it. Except that the disk might not provide infos like a physical disk 
would in regard to vendor, model and serial number.

Proxmox VE does not come with mdadm. The main reasoning behind this decision can 
be found in the links here: https://bugzilla.proxmox.com/show_bug.cgi?id=4573#c6

> 
> But for this to work, you need some so-called "hardware enablement"
> module loaded in the kernel, that I believe is called 'intel_rst' (not
> 100% sure -- we no longer have the test workstation, unfortunately). If
> that module is also bundled with the kernel that ships with Proxmox 8, I
> don't see why it should not work there as well.

On all the reports we have seen over the last years, users reported that the 
disks were shown separately even though they configured the RAID in the BIOS. 
The intel_rst kernel module is not loaded by default on Proxmox VE.

> 
>> If it is an actual HW RAID, it should only show a single block device
>> for each RAID to the OS.
> On the HP Z4 Gen5 workstation we evaluated, that's exactly what happened.

I don't know the intel_rst kernel module. Maybe it hides the actual disks used 
for the MD raid. I would have been interested in the output of the following 
commands to confirm that suspicion:
mdadm -D /dev/mdX
lsblk
ls -l /dev/disk/by-*/

> 
>> In consumer boards, the chances of VROC being the fake RAID variant are
>> quite high.
> Maybe or maybe not, but I'd rather make sure before buying :- >
>> But why not ZFS with mirrored (maybe even multiple mirrored disk, RAID
>> 10 like)?
> I'm already quite familiar with md-raid + LVM (not on Proxmox,
> obviously, but on openSUSE), but I never used ZFS. Although the server
> we're discussing is a home server, it's still a "production" home server
> and I don't want to face the learning curve when things go south. Based
> on what I read, ZFS sounds interesting, but I'd rather experiment with
> it on a test machine first.

As long as you go with ZFS pools made up of a mirror, or multiple mirrors, there 
isn't much that can go wrong or result in unpleasant surprises when used as VM 
storage. For more details, you can check out 
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_zfs_raid_considerations

Cheers, Aaron

> 
>> In case of a disk failure, the handling of it is much nicer and if you
>> can hotplug the disks, you don't even need to power down the machine.
> The budget I plan to spend on the replacement server won't allow for
> hotplug, I'm afraid, but in terms of disk failure handling, it's hard to
> beat true HW RAID in my experience.
> 
> Cheers,
> 
> Ph. A.
> 
>>
>>
>> Cheers,
>> Aaron
>>
>> On 6/27/23 09:50, Philippe Andersson wrote:
>>> Hello List,
>>>
>>> Quick question. I'm currently looking into upgrading my home Proxmox
>>> server (using PC components meant for the home market, for cost
>>> reasons), and getting a "real" hardware RAID proposition under these
>>> conditions turns out to be quite an ordeal. Intel RST/VROC seems to be
>>> the only available option.
>>>
>>> Can anyone on this list please confirm whether these technologies are
>>> supported by the latest Proxmox VE installer/kernel ?
>>>
>>> Thanks in advance.
>>>
>>> Ph. A.
>>>
>>> -- 
>>>
>>> *Philippe Andersson*
>>> Unix System Administrator
>>> IBA Particle Therapy |
>>> Tel: +32-10-475.983
>>> Fax: +32-10-487.707
>>> eMail: pan@iba-group.com
>>> <http://www.iba-worldwide.com>
>>>
>>>
>>>
>>> Disclaimer | Use of IBA
>>> e-communication<https://iba-worldwide.com/disclaimer>
>>>
>>> The contents of this e-mail message and any attachments are intended
>>> solely for the recipient (s) named above. This communication is
>>> intended to be and to remain confidential and may be protected by
>>> intellectual property rights. Any use of the information contained
>>> herein (including but not limited to, total or partial reproduction,
>>> communication or distribution of any form) by persons other than the
>>> designated recipient(s) is prohibited. Please notify the sender
>>> immediately by e-mail if you have received this e-mail by mistake and
>>> delete this e-mail from your system. E-mail transmission cannot be
>>> guaranteed to be secure or error-free. Ion Beam Applications does not
>>> accept liability for any such errors. Thank you for your cooperation.
>>>
>>> _______________________________________________
>>> pve-user mailing list
>>> pve-user@lists.proxmox.com
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.proxmox.com_cgi-2Dbin_mailman_listinfo_pve-2Duser&d=DwICaQ&c=thdlSQUKcAunxaltNxOnpA&r=rQMCMl4AQx8LkqcV6Z_KaNDuQW75wf_RzKbpnczVnKo&m=1fA5OoIIBMGSqFthga-vsoSusk_dmVO6CrqTSRr_0NzBVgYDwxWF03x-Dfkhd3nI&s=TXlgJFn9uGZNfu6JaVpOmcnGnvLL2buHiv-_Gtz7K0A&e=
>>>
>>
> 
> -- 
> 
> *Philippe Andersson*
> Unix System Administrator
> IBA Particle Therapy |
> Tel: +32-10-475.983
> Fax: +32-10-487.707
> eMail: pan@iba-group.com
> <http://www.iba-worldwide.com>
> 
> 
> 
> Disclaimer | Use of IBA e-communication<https://iba-worldwide.com/disclaimer>
> 
> The contents of this e-mail message and any attachments are intended solely for 
> the recipient (s) named above. This communication is intended to be and to 
> remain confidential and may be protected by intellectual property rights. Any 
> use of the information contained herein (including but not limited to, total or 
> partial reproduction, communication or distribution of any form) by persons 
> other than the designated recipient(s) is prohibited. Please notify the sender 
> immediately by e-mail if you have received this e-mail by mistake and delete 
> this e-mail from your system. E-mail transmission cannot be guaranteed to be 
> secure or error-free. Ion Beam Applications does not accept liability for any 
> such errors. Thank you for your cooperation.
> 
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 
>