[PVE-User] ZFS configurations, performance and optimization...

[PVE-User] ZFS configurations, performance and optimization...

[Marco Gaiarin]

Consider two rather equal servers, both with equal disks (2TB
rotational drives); one server have a ZFS mirror (two disk), the other
have a ZFS RAIDZ1, with 3 disks.

Both server are unload, we are moving data in, via single
Gigabit connection.


Moving data into the ZFS mirror lead to a impressive 120 Mbyte/s
writing speed, with little or no iodelay (1-2 %).

Moving data into ZRAID1 lead to an 60-70 Mbyte/s writing speed, no
more, and a decent iodelay on the system (8-10 %).

Both seems to me configured equally (eg, for example, no compression).


I've read:

	https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_raid_considerations

but really i'm got a bit lost... there's somewhere a more clear
explanation of the 'effect', and i hope some parameters to tackle with?


Thanks.


PS: it is not a problem 'per se', the RAIDZ1 pool could be 'slow', but
    i've the doubt to have done somthing wrong...

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)

Re: [PVE-User] ZFS configurations, performance and optimization...

[Yanni M.]

In my opinion, you are not doing something wrong but likely are the
limitations of relatively small raidz1 array (3 disks) and the parity
calculation overhead which both are leading to lower performance compared
to that of the mirrored array. Wherever possible, choose multiple mirrored
vdev configuration (raid10) rather than raidz, as VMs in general tend to be
more interested in IOPS rather than bandwidth (streaming) performance.
If you decide to keep raidz1, then try adding more disks onto it. If you
use zvols for the VMs, try increasing volblocksize from 8k which is the
default, to 16k or 32k.
Personally, I'd add one more disk into the mix and go for 2 vdev mirrored
setup (raid10) and keep the zfs defaults.



On Mon, 26 Jul 2021 at 10:37, Marco Gaiarin <gaio@sv.lnf.it> wrote:

>
> Consider two rather equal servers, both with equal disks (2TB
> rotational drives); one server have a ZFS mirror (two disk), the other
> have a ZFS RAIDZ1, with 3 disks.
>
> Both server are unload, we are moving data in, via single
> Gigabit connection.
>
>
> Moving data into the ZFS mirror lead to a impressive 120 Mbyte/s
> writing speed, with little or no iodelay (1-2 %).
>
> Moving data into ZRAID1 lead to an 60-70 Mbyte/s writing speed, no
> more, and a decent iodelay on the system (8-10 %).
>
> Both seems to me configured equally (eg, for example, no compression).
>
>
> I've read:
>
>
> https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_raid_considerations
>
> but really i'm got a bit lost... there's somewhere a more clear
> explanation of the 'effect', and i hope some parameters to tackle with?
>
>
> Thanks.
>
>
> PS: it is not a problem 'per se', the RAIDZ1 pool could be 'slow', but
>     i've the doubt to have done somthing wrong...
>
> --
> dott. Marco Gaiarin                                     GNUPG Key ID:
> 240A3D66
>   Associazione ``La Nostra Famiglia''
> http://www.lanostrafamiglia.it/
>   Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento
> (PN)
>   marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f
> +39-0434-842797
>
>                 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
>       http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
>         (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)
>
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>
>

Re: [PVE-User] ZFS configurations, performance and optimization...

[Marco Gaiarin]

Mandi! Yanni M.
  In chel di` si favelave...

> In my opinion, you are not doing something wrong but likely are the
> limitations of relatively small raidz1 array (3 disks) and the parity
> calculation overhead which both are leading to lower performance compared
> to that of the mirrored array.

Thanks, Gianni.

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)