Proxmox VE stretched cluster with NetApp MetroCluster over Fibre Channel

Proxmox VE stretched cluster with NetApp MetroCluster over Fibre Channel

[Iván Méndez Cairós]

Hello,
We are evaluating the replacement of an existing VMware vSphere infrastructure with Proxmox VE and would like to confirm the feasibility and support considerations of the proposed architecture.
The current environment is distributed across two data centers and uses a NetApp MetroCluster storage system. The new design would consist of:

  *   One Proxmox VE cluster distributed across both data centers.
  *   Approximately 12 Proxmox nodes per site.
  *   NetApp MetroCluster providing synchronously replicated shared storage.
  *   Fibre Channel connectivity between the Proxmox hosts and NetApp.
  *   Two independent FC fabrics based on Brocade G720 switches.
  *   Dual-port FC HBAs in each Proxmox host.
  *   Linux multipath/ALUA configured on every host.
  *   Shared LVM storage on top of the FC LUNs.
  *   Proxmox HA enabled for the virtual machines.
  *   Redundant Corosync links between both sites, with latency below 5 ms.
  *   A QDevice located in a third failure domain.
NetApp documentation explicitly lists MetroCluster support for Proxmox VE using SAN protocols, including Fibre Channel:
https://docs.netapp.com/us-en/netapp-solutions-virtualization/proxmox/proxmox-ontap-architecture.html#solution-architecture
However, we would like to confirm the following points from the Proxmox side:

  1.  Is a stretched Proxmox VE cluster using shared FC storage provided by NetApp MetroCluster considered a supported and recommended configuration?
  2.  Is shared LVM over a multipath FC device the appropriate storage configuration for this design?
  3.  Are there any specific multipath, ALUA or LVM configuration requirements for NetApp ONTAP/MetroCluster?
  4.  During a MetroCluster switchover, SAN I/O may be temporarily interrupted. Are there any Proxmox, multipath or HA timeout parameters that should be adjusted to tolerate this process and prevent unnecessary fencing or VM failures?
  5.  In the event of a complete site failure, would Proxmox HA be able to restart the affected VMs on the surviving site once the MetroCluster switchover has completed and the FC LUNs are accessible again?
  6.  Is the proposed QDevice design sufficient to preserve quorum if communication between the two data centers is lost?
  7.  Are there any known reference architectures, deployment guides or production experiences involving Proxmox VE and NetApp MetroCluster over Fibre Channel?
We understand that the exact server, HBA, switch, ONTAP and firmware versions must also be validated through the corresponding vendor interoperability matrices.
Any recommendations or experiences with a similar architecture would be greatly appreciated.
Best regards,
Iván Méndez

RE: Proxmox VE stretched cluster with NetApp MetroCluster over Fibre Channel

[Lund Svein-Erik]

> Hello,
> We are evaluating the replacement of an existing VMware vSphere infrastructure with Proxmox VE and would like to confirm the feasibility and support considerations of the proposed  architecture.
> The current environment is distributed across two data centers and uses a NetApp MetroCluster storage system. The new design would consist of:
> 
>   *   One Proxmox VE cluster distributed across both data centers.
>   *   Approximately 12 Proxmox nodes per site.
>   *   NetApp MetroCluster providing synchronously replicated shared storage.
>   *   Fibre Channel connectivity between the Proxmox hosts and NetApp.
>   *   Two independent FC fabrics based on Brocade G720 switches.
>   *   Dual-port FC HBAs in each Proxmox host.
>   *   Linux multipath/ALUA configured on every host.
>   *   Shared LVM storage on top of the FC LUNs.
>   *   Proxmox HA enabled for the virtual machines.
>   *   Redundant Corosync links between both sites, with latency below 5 ms.
>   *   A QDevice located in a third failure domain.
> NetApp documentation explicitly lists MetroCluster support for Proxmox VE using SAN protocols, including Fibre Channel:
> https://docs.netapp.com/us-en/netapp-solutions-virtualization/proxmox/proxmox-ontap-architecture.html#solution-architecture
> However, we would like to confirm the following points from the Proxmox side:

>  1.  Is a stretched Proxmox VE cluster using shared FC storage provided by NetApp MetroCluster considered a supported and recommended configuration?
If it's supported or not, I'm not sure but the described solution sounds sane from what I've worked on previously. I suspect you're trying to move away from VMWare with your current hardware here. 

>  2.  Is shared LVM over a multipath FC device the appropriate storage configuration for this design?
It works, but it's not what I would have chosen if I was to design a Proxmox setup from scratch. Having run Proxmox with Ceph for a few years now I must say that I like that much better. 
One thing to keep an eye on here is your lvm config, Make really sure that you _only_ access your multipah devices, not the sdX devices. 

>  3.  Are there any specific multipath, ALUA or LVM configuration requirements for NetApp ONTAP/MetroCluster?
>  4.  During a MetroCluster switchover, SAN I/O may be temporarily interrupted. Are there any Proxmox, multipath or HA timeout parameters that should be adjusted to tolerate this process and prevent unnecessary fencing or VM failures?

Make sure that you tune the scsi timeout settings here, Having scsi devices dissapearing in any hypervisor is really a pain. Had the "plesure" of that problem on both vmware and proxmox my self... 

>  5.  In the event of a complete site failure, would Proxmox HA be able to restart the affected VMs on the surviving site once the MetroCluster switchover has completed and the FC LUNs are accessible again?

It's a bit more work in Proxmox, You have to add vm's to a HA group to make sure the vm is started in such cases. 

>  6.  Is the proposed QDevice design sufficient to preserve quorum if communication between the two data centers is lost?
A QDevice is only a vote, that's it. Nothing else. If one site falls down, and the qdevice is down, then HA wont recover. Only way I know that _might_ help here is to have three qdevices, but then they would need to be on tree offsite location, I've never seen that done. Nor am I 100% sure that it actually would work. 

>  7.  Are there any known reference architectures, deployment guides or production experiences involving Proxmox VE and NetApp MetroCluster over Fibre Channel?
>We understand that the exact server, HBA, switch, ONTAP and firmware versions must also be validated through the corresponding vendor interoperability matrices.
>Any recommendations or experiences with a similar architecture would be greatly appreciated.

I don't know  your company, but I suspect that you've been paying for VMWare support. I highly suggest that you subscribe to Proxmox as well. Yes it's opensource, and you can use it freely, but given the questions you have I suspect that you don't have that much inhouse experience with Proxmox 
My experience with Proxmox go all the way back to the first public beta of Proxmox, and I must say that it's been a bumpy ride at times, but It's always been possible to find solutions to problems with Proxmox. Vmware on the other hand is another story...

--
Svein-Erik 

Proxmox VE stretched cluster with NetApp MetroCluster over Fibre Channel

[zzservers]

Message: 1
Date: Thu, 30 Jul 2026 15:42:20 +0000
From: Iván Méndez Cairós <imendez@integratyc.com>
Subject: Proxmox VE stretched cluster with NetApp MetroCluster over
	Fibre Channel
To: Proxmox VE user list <pve-user@lists.proxmox.com>
Message-ID: <DU0PR03MB925343602EE3FA5AD9EA265CCDC92@DU0PR03MB9253.eurp
	rd03.prod.outlook.com>
Content-Type: text/plain; charset="iso-8859-1"

Hello,
We are evaluating the replacement of an existing VMware vSphere infrastructure with Proxmox VE and would like to confirm the feasibility and support considerations of the proposed architecture.
The current environment is distributed across two data centers and uses a NetApp MetroCluster storage system. The new design would consist of:

  *   One Proxmox VE cluster distributed across both data centers.
  *   Approximately 12 Proxmox nodes per site.
  *   NetApp MetroCluster providing synchronously replicated shared storage.
  *   Fibre Channel connectivity between the Proxmox hosts and NetApp.
  *   Two independent FC fabrics based on Brocade G720 switches.
  *   Dual-port FC HBAs in each Proxmox host.
  *   Linux multipath/ALUA configured on every host.
  *   Shared LVM storage on top of the FC LUNs.
  *   Proxmox HA enabled for the virtual machines.
  *   Redundant Corosync links between both sites, with latency below 5 ms.
  *   A QDevice located in a third failure domain.
NetApp documentation explicitly lists MetroCluster support for Proxmox VE using SAN protocols, including Fibre Channel:
https://docs.netapp.com/us-en/netapp-solutions-virtualization/proxmox/proxmox-ontap-architecture.html#solution-architecture
However, we would like to confirm the following points from the Proxmox side:

  1.  Is a stretched Proxmox VE cluster using shared FC storage provided by NetApp MetroCluster considered a supported and recommended configuration?
  2.  Is shared LVM over a multipath FC device the appropriate storage configuration for this design?
  3.  Are there any specific multipath, ALUA or LVM configuration requirements for NetApp ONTAP/MetroCluster?
  4.  During a MetroCluster switchover, SAN I/O may be temporarily interrupted. Are there any Proxmox, multipath or HA timeout parameters that should be adjusted to tolerate this process and prevent unnecessary fencing or VM failures?
  5.  In the event of a complete site failure, would Proxmox HA be able to restart the affected VMs on the surviving site once the MetroCluster switchover has completed and the FC LUNs are accessible again?
  6.  Is the proposed QDevice design sufficient to preserve quorum if communication between the two data centers is lost?
  7.  Are there any known reference architectures, deployment guides or production experiences involving Proxmox VE and NetApp MetroCluster over Fibre Channel?
We understand that the exact server, HBA, switch, ONTAP and firmware versions must also be validated through the corresponding vendor interoperability matrices.
Any recommendations or experiences with a similar architecture would be greatly appreciated.
Best regards,
Iván Méndez
------------------------------

This is free, peer support. You know, three mini-PCs in a HomeLab.

For professional usage, you should be working with a knowledgeable VAR.

-----Burton