[PVE-User] Intel RST / VROC support in latest Proxmox?

[PVE-User] Intel RST / VROC support in latest Proxmox?

[Philippe Andersson]

Hello List,

Quick question. I'm currently looking into upgrading my home Proxmox
server (using PC components meant for the home market, for cost
reasons), and getting a "real" hardware RAID proposition under these
conditions turns out to be quite an ordeal. Intel RST/VROC seems to be
the only available option.

Can anyone on this list please confirm whether these technologies are
supported by the latest Proxmox VE installer/kernel ?

Thanks in advance.

Ph. A.

--

*Philippe Andersson*
Unix System Administrator
IBA Particle Therapy |
Tel: +32-10-475.983
Fax: +32-10-487.707
eMail: pan@iba-group.com
<http://www.iba-worldwide.com>



Disclaimer | Use of IBA e-communication<https://iba-worldwide.com/disclaimer>

The contents of this e-mail message and any attachments are intended solely for the recipient (s) named above. This communication is intended to be and to remain confidential and may be protected by intellectual property rights. Any use of the information contained herein (including but not limited to, total or partial reproduction, communication or distribution of any form) by persons other than the designated recipient(s) is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free. Ion Beam Applications does not accept liability for any such errors. Thank you for your cooperation.

Re: [PVE-User] Intel RST / VROC support in latest Proxmox?

[Aaron Lauterer]

AFAIK, VROC has two variants, actual HW RAID and so-called fake RAID. If it is 
the fake RAID variant, it will be ignored and the single disks should be shown 
in the installer.

If it is an actual HW RAID, it should only show a single block device for each 
RAID to the OS.

In consumer boards, the chances of VROC being the fake RAID variant are quite high.

But why not ZFS with mirrored (maybe even multiple mirrored disk, RAID 10 like)?
In case of a disk failure, the handling of it is much nicer and if you can 
hotplug the disks, you don't even need to power down the machine.


Cheers,
Aaron

On 6/27/23 09:50, Philippe Andersson wrote:
> Hello List,
> 
> Quick question. I'm currently looking into upgrading my home Proxmox
> server (using PC components meant for the home market, for cost
> reasons), and getting a "real" hardware RAID proposition under these
> conditions turns out to be quite an ordeal. Intel RST/VROC seems to be
> the only available option.
> 
> Can anyone on this list please confirm whether these technologies are
> supported by the latest Proxmox VE installer/kernel ?
> 
> Thanks in advance.
> 
> Ph. A.
> 
> -- 
> 
> *Philippe Andersson*
> Unix System Administrator
> IBA Particle Therapy |
> Tel: +32-10-475.983
> Fax: +32-10-487.707
> eMail: pan@iba-group.com
> <http://www.iba-worldwide.com>
> 
> 
> 
> Disclaimer | Use of IBA e-communication<https://iba-worldwide.com/disclaimer>
> 
> The contents of this e-mail message and any attachments are intended solely for 
> the recipient (s) named above. This communication is intended to be and to 
> remain confidential and may be protected by intellectual property rights. Any 
> use of the information contained herein (including but not limited to, total or 
> partial reproduction, communication or distribution of any form) by persons 
> other than the designated recipient(s) is prohibited. Please notify the sender 
> immediately by e-mail if you have received this e-mail by mistake and delete 
> this e-mail from your system. E-mail transmission cannot be guaranteed to be 
> secure or error-free. Ion Beam Applications does not accept liability for any 
> such errors. Thank you for your cooperation.
> 
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 
> 

Re: [PVE-User] Intel RST / VROC support in latest Proxmox?

[Konold, Martin]

Hi,

if you love your data do not go for hw raid but ZFS instead.

Proxmox has native support for ZFS.

---
Regards
ppa. Martin Konold

--
Martin Konold - Prokurist, CTO
KONSEC GmbH -⁠ make things real
Amtsgericht Stuttgart, HRB 23690
Geschäftsführer: Andreas Mack
Im Köller 3, 70794 Filderstadt, Germany

On 2023-06-27 10:33, Aaron Lauterer wrote:
> AFAIK, VROC has two variants, actual HW RAID and so-called fake RAID.
> If it is the fake RAID variant, it will be ignored and the single
> disks should be shown in the installer.
> 
> If it is an actual HW RAID, it should only show a single block device
> for each RAID to the OS.
> 
> In consumer boards, the chances of VROC being the fake RAID variant
> are quite high.
> 
> But why not ZFS with mirrored (maybe even multiple mirrored disk, RAID 
> 10 like)?
> In case of a disk failure, the handling of it is much nicer and if you
> can hotplug the disks, you don't even need to power down the machine.
> 
> 
> Cheers,
> Aaron
> 
> On 6/27/23 09:50, Philippe Andersson wrote:
>> Hello List,
>> 
>> Quick question. I'm currently looking into upgrading my home Proxmox
>> server (using PC components meant for the home market, for cost
>> reasons), and getting a "real" hardware RAID proposition under these
>> conditions turns out to be quite an ordeal. Intel RST/VROC seems to be
>> the only available option.
>> 
>> Can anyone on this list please confirm whether these technologies are
>> supported by the latest Proxmox VE installer/kernel ?
>> 
>> Thanks in advance.
>> 
>> Ph. A.
>> 
>> -- *Philippe Andersson*
>> Unix System Administrator
>> IBA Particle Therapy |
>> Tel: +32-10-475.983
>> Fax: +32-10-487.707
>> eMail: pan@iba-group.com
>> <http://www.iba-worldwide.com>
>> 
>> 
>> 
>> Disclaimer | Use of IBA 
>> e-communication<https://iba-worldwide.com/disclaimer>
>> 
>> The contents of this e-mail message and any attachments are intended 
>> solely for the recipient (s) named above. This communication is 
>> intended to be and to remain confidential and may be protected by 
>> intellectual property rights. Any use of the information contained 
>> herein (including but not limited to, total or partial reproduction, 
>> communication or distribution of any form) by persons other than the 
>> designated recipient(s) is prohibited. Please notify the sender 
>> immediately by e-mail if you have received this e-mail by mistake and 
>> delete this e-mail from your system. E-mail transmission cannot be 
>> guaranteed to be secure or error-free. Ion Beam Applications does not 
>> accept liability for any such errors. Thank you for your cooperation.
>> 
>> _______________________________________________
>> pve-user mailing list
>> pve-user@lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>> 
>> 
> 
> 
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] Intel RST / VROC support in latest Proxmox?

[Philippe Andersson]

Hello Aaron,

Many thanks for your reply, first of all.

On 27/06/2023 10:33, Aaron Lauterer wrote:
> AFAIK, VROC has two variants, actual HW RAID and so-called fake RAID. If
> it is the fake RAID variant, it will be ignored and the single disks
> should be shown in the installer.
I don't know whether this "fake RAID vs real RAID" still applies to the
recent BIOS/UEFI firmwares.

We recently tested (at work) an HP Z4 Gen5 workstation with 2 NVMEs
configured in RAID-1 through VROC, and the openSUSE 15.4 we installed
there had no issue seeing the resulting LUN. Even better, we could use
regular 'mdadm' to monitor the state of the array (although 'mdadm'
could not modify its configuration, obviously -- that can only be done
through the BIOS/UEFI utilities).

But for this to work, you need some so-called "hardware enablement"
module loaded in the kernel, that I believe is called 'intel_rst' (not
100% sure -- we no longer have the test workstation, unfortunately). If
that module is also bundled with the kernel that ships with Proxmox 8, I
don't see why it should not work there as well.

> If it is an actual HW RAID, it should only show a single block device
> for each RAID to the OS.
On the HP Z4 Gen5 workstation we evaluated, that's exactly what happened.

> In consumer boards, the chances of VROC being the fake RAID variant are
> quite high.
Maybe or maybe not, but I'd rather make sure before buying :-/

> But why not ZFS with mirrored (maybe even multiple mirrored disk, RAID
> 10 like)?
I'm already quite familiar with md-raid + LVM (not on Proxmox,
obviously, but on openSUSE), but I never used ZFS. Although the server
we're discussing is a home server, it's still a "production" home server
and I don't want to face the learning curve when things go south. Based
on what I read, ZFS sounds interesting, but I'd rather experiment with
it on a test machine first.

> In case of a disk failure, the handling of it is much nicer and if you
> can hotplug the disks, you don't even need to power down the machine.
The budget I plan to spend on the replacement server won't allow for
hotplug, I'm afraid, but in terms of disk failure handling, it's hard to
beat true HW RAID in my experience.

Cheers,

Ph. A.

>
>
> Cheers,
> Aaron
>
> On 6/27/23 09:50, Philippe Andersson wrote:
>> Hello List,
>>
>> Quick question. I'm currently looking into upgrading my home Proxmox
>> server (using PC components meant for the home market, for cost
>> reasons), and getting a "real" hardware RAID proposition under these
>> conditions turns out to be quite an ordeal. Intel RST/VROC seems to be
>> the only available option.
>>
>> Can anyone on this list please confirm whether these technologies are
>> supported by the latest Proxmox VE installer/kernel ?
>>
>> Thanks in advance.
>>
>> Ph. A.
>>
>> --
>>
>> *Philippe Andersson*
>> Unix System Administrator
>> IBA Particle Therapy |
>> Tel: +32-10-475.983
>> Fax: +32-10-487.707
>> eMail: pan@iba-group.com
>> <http://www.iba-worldwide.com>
>>
>>
>>
>> Disclaimer | Use of IBA
>> e-communication<https://iba-worldwide.com/disclaimer>
>>
>> The contents of this e-mail message and any attachments are intended
>> solely for the recipient (s) named above. This communication is
>> intended to be and to remain confidential and may be protected by
>> intellectual property rights. Any use of the information contained
>> herein (including but not limited to, total or partial reproduction,
>> communication or distribution of any form) by persons other than the
>> designated recipient(s) is prohibited. Please notify the sender
>> immediately by e-mail if you have received this e-mail by mistake and
>> delete this e-mail from your system. E-mail transmission cannot be
>> guaranteed to be secure or error-free. Ion Beam Applications does not
>> accept liability for any such errors. Thank you for your cooperation.
>>
>> _______________________________________________
>> pve-user mailing list
>> pve-user@lists.proxmox.com
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.proxmox.com_cgi-2Dbin_mailman_listinfo_pve-2Duser&d=DwICaQ&c=thdlSQUKcAunxaltNxOnpA&r=rQMCMl4AQx8LkqcV6Z_KaNDuQW75wf_RzKbpnczVnKo&m=1fA5OoIIBMGSqFthga-vsoSusk_dmVO6CrqTSRr_0NzBVgYDwxWF03x-Dfkhd3nI&s=TXlgJFn9uGZNfu6JaVpOmcnGnvLL2buHiv-_Gtz7K0A&e=
>>
>

--

*Philippe Andersson*
Unix System Administrator
IBA Particle Therapy |
Tel: +32-10-475.983
Fax: +32-10-487.707
eMail: pan@iba-group.com
<http://www.iba-worldwide.com>



Disclaimer | Use of IBA e-communication<https://iba-worldwide.com/disclaimer>

The contents of this e-mail message and any attachments are intended solely for the recipient (s) named above. This communication is intended to be and to remain confidential and may be protected by intellectual property rights. Any use of the information contained herein (including but not limited to, total or partial reproduction, communication or distribution of any form) by persons other than the designated recipient(s) is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free. Ion Beam Applications does not accept liability for any such errors. Thank you for your cooperation.

Re: [PVE-User] Intel RST / VROC support in latest Proxmox?

[Aaron Lauterer]

On 6/28/23 16:55, Philippe Andersson wrote:
> Hello Aaron,
> 
> Many thanks for your reply, first of all.
> 
> On 27/06/2023 10:33, Aaron Lauterer wrote:
>> AFAIK, VROC has two variants, actual HW RAID and so-called fake RAID. If
>> it is the fake RAID variant, it will be ignored and the single disks
>> should be shown in the installer.
> I don't know whether this "fake RAID vs real RAID" still applies to the
> recent BIOS/UEFI firmwares.
> 
> We recently tested (at work) an HP Z4 Gen5 workstation with 2 NVMEs
> configured in RAID-1 through VROC, and the openSUSE 15.4 we installed
> there had no issue seeing the resulting LUN. Even better, we could use
> regular 'mdadm' to monitor the state of the array (although 'mdadm'
> could not modify its configuration, obviously -- that can only be done
> through the BIOS/UEFI utilities).

And that is fake RAID AFAIU. The BIOS telling the OS how the RAID should be 
configured. On Linux, that usually means MD RAID. On Windows, a special driver 
might be necessary. So in the end, a BIOS configured software RAID running on 
the OS level.

On a real RAID, the OS would only see a single disk and would not have any idea 
about it. Except that the disk might not provide infos like a physical disk 
would in regard to vendor, model and serial number.

Proxmox VE does not come with mdadm. The main reasoning behind this decision can 
be found in the links here: https://bugzilla.proxmox.com/show_bug.cgi?id=4573#c6

> 
> But for this to work, you need some so-called "hardware enablement"
> module loaded in the kernel, that I believe is called 'intel_rst' (not
> 100% sure -- we no longer have the test workstation, unfortunately). If
> that module is also bundled with the kernel that ships with Proxmox 8, I
> don't see why it should not work there as well.

On all the reports we have seen over the last years, users reported that the 
disks were shown separately even though they configured the RAID in the BIOS. 
The intel_rst kernel module is not loaded by default on Proxmox VE.

> 
>> If it is an actual HW RAID, it should only show a single block device
>> for each RAID to the OS.
> On the HP Z4 Gen5 workstation we evaluated, that's exactly what happened.

I don't know the intel_rst kernel module. Maybe it hides the actual disks used 
for the MD raid. I would have been interested in the output of the following 
commands to confirm that suspicion:
mdadm -D /dev/mdX
lsblk
ls -l /dev/disk/by-*/

> 
>> In consumer boards, the chances of VROC being the fake RAID variant are
>> quite high.
> Maybe or maybe not, but I'd rather make sure before buying :- >
>> But why not ZFS with mirrored (maybe even multiple mirrored disk, RAID
>> 10 like)?
> I'm already quite familiar with md-raid + LVM (not on Proxmox,
> obviously, but on openSUSE), but I never used ZFS. Although the server
> we're discussing is a home server, it's still a "production" home server
> and I don't want to face the learning curve when things go south. Based
> on what I read, ZFS sounds interesting, but I'd rather experiment with
> it on a test machine first.

As long as you go with ZFS pools made up of a mirror, or multiple mirrors, there 
isn't much that can go wrong or result in unpleasant surprises when used as VM 
storage. For more details, you can check out 
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_zfs_raid_considerations

Cheers, Aaron

> 
>> In case of a disk failure, the handling of it is much nicer and if you
>> can hotplug the disks, you don't even need to power down the machine.
> The budget I plan to spend on the replacement server won't allow for
> hotplug, I'm afraid, but in terms of disk failure handling, it's hard to
> beat true HW RAID in my experience.
> 
> Cheers,
> 
> Ph. A.
> 
>>
>>
>> Cheers,
>> Aaron
>>
>> On 6/27/23 09:50, Philippe Andersson wrote:
>>> Hello List,
>>>
>>> Quick question. I'm currently looking into upgrading my home Proxmox
>>> server (using PC components meant for the home market, for cost
>>> reasons), and getting a "real" hardware RAID proposition under these
>>> conditions turns out to be quite an ordeal. Intel RST/VROC seems to be
>>> the only available option.
>>>
>>> Can anyone on this list please confirm whether these technologies are
>>> supported by the latest Proxmox VE installer/kernel ?
>>>
>>> Thanks in advance.
>>>
>>> Ph. A.
>>>
>>> -- 
>>>
>>> *Philippe Andersson*
>>> Unix System Administrator
>>> IBA Particle Therapy |
>>> Tel: +32-10-475.983
>>> Fax: +32-10-487.707
>>> eMail: pan@iba-group.com
>>> <http://www.iba-worldwide.com>
>>>
>>>
>>>
>>> Disclaimer | Use of IBA
>>> e-communication<https://iba-worldwide.com/disclaimer>
>>>
>>> The contents of this e-mail message and any attachments are intended
>>> solely for the recipient (s) named above. This communication is
>>> intended to be and to remain confidential and may be protected by
>>> intellectual property rights. Any use of the information contained
>>> herein (including but not limited to, total or partial reproduction,
>>> communication or distribution of any form) by persons other than the
>>> designated recipient(s) is prohibited. Please notify the sender
>>> immediately by e-mail if you have received this e-mail by mistake and
>>> delete this e-mail from your system. E-mail transmission cannot be
>>> guaranteed to be secure or error-free. Ion Beam Applications does not
>>> accept liability for any such errors. Thank you for your cooperation.
>>>
>>> _______________________________________________
>>> pve-user mailing list
>>> pve-user@lists.proxmox.com
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.proxmox.com_cgi-2Dbin_mailman_listinfo_pve-2Duser&d=DwICaQ&c=thdlSQUKcAunxaltNxOnpA&r=rQMCMl4AQx8LkqcV6Z_KaNDuQW75wf_RzKbpnczVnKo&m=1fA5OoIIBMGSqFthga-vsoSusk_dmVO6CrqTSRr_0NzBVgYDwxWF03x-Dfkhd3nI&s=TXlgJFn9uGZNfu6JaVpOmcnGnvLL2buHiv-_Gtz7K0A&e=
>>>
>>
> 
> -- 
> 
> *Philippe Andersson*
> Unix System Administrator
> IBA Particle Therapy |
> Tel: +32-10-475.983
> Fax: +32-10-487.707
> eMail: pan@iba-group.com
> <http://www.iba-worldwide.com>
> 
> 
> 
> Disclaimer | Use of IBA e-communication<https://iba-worldwide.com/disclaimer>
> 
> The contents of this e-mail message and any attachments are intended solely for 
> the recipient (s) named above. This communication is intended to be and to 
> remain confidential and may be protected by intellectual property rights. Any 
> use of the information contained herein (including but not limited to, total or 
> partial reproduction, communication or distribution of any form) by persons 
> other than the designated recipient(s) is prohibited. Please notify the sender 
> immediately by e-mail if you have received this e-mail by mistake and delete 
> this e-mail from your system. E-mail transmission cannot be guaranteed to be 
> secure or error-free. Ion Beam Applications does not accept liability for any 
> such errors. Thank you for your cooperation.
> 
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 
> 

Re: [PVE-User] Intel RST / VROC support in latest Proxmox?

[Philippe Andersson]

Disclaimer | Use of IBA e-communication<https://iba-worldwide.com/disclaimer>

The contents of this e-mail message and any attachments are intended solely for the recipient (s) named above. This communication is intended to be and to remain confidential and may be protected by intellectual property rights. Any use of the information contained herein (including but not limited to, total or partial reproduction, communication or distribution of any form) by persons other than the designated recipient(s) is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free. Ion Beam Applications does not accept liability for any such errors. Thank you for your cooperation.

Re: [PVE-User] Intel RST / VROC support in latest Proxmox?

[Philippe Andersson]

Bonjour Alain 😉

On 28/06/2023 10:58, Alain Péan via pve-user wrote:
 > Hi Philippe,
 >
 > Why do you want to use an hadware Raid ?
Well, the original reason is because it's supposed to be transparent to
the OS above (although, as I explained in my reply to Aaron, this may
not be entirely true for VROC).

 > On a home PC, I would use an
 > NVME SSD of the desired size (prices have decreased a lot in the last
 > times), and if you want a secure one, look for enterprise grade NVME
 > (for example Read intensive), even they are more costly,
Sure.

 > and indeed,
 > you can use ZFS on top of that, to have snapshot for exemple, or
 > LVMthin...
I will look into ZFS, indeed. At the very least, it means I have a
fallback solution if I go for the VROC option and it turns out to be
unworkable.

Thanks to all who responded.

Ph. A.

--

*Philippe Andersson*
Unix System Administrator
IBA Particle Therapy |
Tel: +32-10-475.983
Fax: +32-10-487.707
eMail: pan@iba-group.com
<http://www.iba-worldwide.com>


Disclaimer | Use of IBA e-communication<https://iba-worldwide.com/disclaimer>

The contents of this e-mail message and any attachments are intended solely for the recipient (s) named above. This communication is intended to be and to remain confidential and may be protected by intellectual property rights. Any use of the information contained herein (including but not limited to, total or partial reproduction, communication or distribution of any form) by persons other than the designated recipient(s) is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free. Ion Beam Applications does not accept liability for any such errors. Thank you for your cooperation.