[PVE-User] PBS datastore

[PVE-User] PBS datastore

[Tonči Stipičević]

I'm planning to use SUPERMICRO 721 box (4 x 2T sata) as PBS

So, if I create zfs10 raid as boot drive , I have no more disks left for 
datastore, right ?

BUT, is it possible to make directory on this / boot drive i.e. 
/mnt/datastore   and attach it as datastore ?   Is that a way-to-go  and 
is there any disadvantages against separate 2 x 240g ssd as boot and 4 x 
2T sata (zfs raid 10 ) for datastore?

Actually I'ma having hard time finding 2 x 2,5" small  sata drives for 
boot ....

   Thank you very much in advance

BR

Tonci

srdačan pozdrav / best regards

Tonči Stipičević, dipl. ing. elektr.
direktor / manager

SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263

Podrška / Upravljanje IT sustavima za male i srednje tvrtke
Small & Medium Business IT Support / Management

mob: 091 1234003
www.suma-informatika.hr

Re: [PVE-User] PBS datastore

[Aaron Lauterer]

A PBS datastore is just a directory somewhere.

If you set up the system on a RAID 10 and want to use that as well for the datastore, I would create a new ZFS dataset for the datastore. This allows you to change certain ZFS properties on that dataset specifically.

There are a few things to consider if you want to have the OS on the same disks as the datastore.
How fast are the disks? HDDs or SSDs?
If they are HDDs, then having the OS on separate disks is a good idea because HDDs will already be having a hard time to provide the IOPS for decent performance. If they are *decent* SSDs, you should be fine.

No idea how easy it is to get stuff in Croatia, but this should give you some ideas: https://geizhals.eu/?cat=hdssd&xf=2028_256~4643_Power-Loss+Protection~4832_1~4836_2&sort=r

250 GiB is plenty of the OS, but anything smaller is hard to find or of dubious quality ;)

Cheers,
Aaron


On 2/7/22 01:10, Tonči Stipičević wrote:
> I'm planning to use SUPERMICRO 721 box (4 x 2T sata) as PBS
> 
> So, if I create zfs10 raid as boot drive , I have no more disks left for datastore, right ?
> 
> BUT, is it possible to make directory on this / boot drive i.e. /mnt/datastore   and attach it as datastore ?   Is that a way-to-go  and is there any disadvantages against separate 2 x 240g ssd as boot and 4 x 2T sata (zfs raid 10 ) for datastore?
> 
> Actually I'ma having hard time finding 2 x 2,5" small  sata drives for boot ....
> 
>    Thank you very much in advance
> 
> BR
> 
> Tonci
> 
> srdačan pozdrav / best regards
> 
> Tonči Stipičević, dipl. ing. elektr.
> direktor / manager
> 
> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
> 
> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
> Small & Medium Business IT Support / Management
> 
> mob: 091 1234003
> www.suma-informatika.hr
> 
> 
> 
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PBS datastore

[nada]

hi Tonci
in case you do not have time and small disks
you may create 3 ZFS mirrors
so there will be for example
rpool for PVE
ctpool for CT/QM
datapool for backups

e.g. in your case
rpool and ctpool may be on 2disks (sda and sdb)
datapool will be on 2 disks (sdc and sdd)

during PVE installation you may select ZFS
allocate 2 disks in mirror and limit volume space to 300G
after installation you may see status
zpool list -v
fdisk -l /dev/sda
fdisk -l /dev/sdb
...
PVE will be probably installed at sda1,sda2,sda3 and sdb1,sdb2,sdb3 
respectively
and rpool will be probably ZFS mirror over the partitions sda3 and sdb3

after that you may create partitions sda4 and sdb4 (type Solaris /usr & 
Apple ZFS)
with zpool mirror over it
zpool create -f -o ashift=12 ctpool mirror /dev/sda4 /dev/sdb4

after installation you may dedicate the rest 2 disks for datastore
e.g.
zpool create -f -o ashift=12 datapool mirror /dev/sdc /dev/sdd

finally you will declare your PVE storage and you may start to play ;-)
hope it helps
Nada


On 2022-02-07 01:10, Tonči Stipičević wrote:
> I'm planning to use SUPERMICRO 721 box (4 x 2T sata) as PBS
> 
> So, if I create zfs10 raid as boot drive , I have no more disks left
> for datastore, right ?
> 
> BUT, is it possible to make directory on this / boot drive i.e.
> /mnt/datastore   and attach it as datastore ?   Is that a way-to-go 
> and is there any disadvantages against separate 2 x 240g ssd as boot
> and 4 x 2T sata (zfs raid 10 ) for datastore?
> 
> Actually I'ma having hard time finding 2 x 2,5" small  sata drives for 
> boot ....
> 
>   Thank you very much in advance
> 
> BR
> 
> Tonci
> 
> srdačan pozdrav / best regards
> 
> Tonči Stipičević, dipl. ing. elektr.
> direktor / manager
> 
> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
> 
> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
> Small & Medium Business IT Support / Management
> 
> mob: 091 1234003
> www.suma-informatika.hr
> 
> 
> 
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PBS datastore

[Tonči Stipičević]

Thank you Aaron,

this atribut "small" is blocking point ... like you said it is hard to 
find sata disk under 250G

I was also thinking about 2 x sata-dom 32G/64G   for zfs boot ? Does it 
make any sense (performance/relayibilty-wise?

Ok , today I'm gonna install PBS on 4x2T (disks, not ssds) zfs raid10  
and create one new dataset in /rpool  -> /rpool/databck    ...  So , the 
PBS itself and datastore will reside on the same pool ...  Then I'll 
make some copmarison tests (backup/restore)  and report back some 
relevant results  ...  I'd like to avoid PBS on its own pool because 
this SuperMicro 721 case has only 4 disk slots , a additional ones (2) 
have to be connected directly to MB ...


'till then

BR

Tonci

srdačan pozdrav / best regards

Tonči Stipičević, dipl. ing. elektr.
direktor / manager

SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263

Podrška / Upravljanje IT sustavima za male i srednje tvrtke
Small & Medium Business IT Support / Management

mob: 091 1234003
www.suma-informatika.hr

On 07. 02. 2022. 09:44, Aaron Lauterer wrote:
> A PBS datastore is just a directory somewhere.
>
> If you set up the system on a RAID 10 and want to use that as well for 
> the datastore, I would create a new ZFS dataset for the datastore. 
> This allows you to change certain ZFS properties on that dataset 
> specifically.
>
> There are a few things to consider if you want to have the OS on the 
> same disks as the datastore.
> How fast are the disks? HDDs or SSDs?
> If they are HDDs, then having the OS on separate disks is a good idea 
> because HDDs will already be having a hard time to provide the IOPS 
> for decent performance. If they are *decent* SSDs, you should be fine.
>
> No idea how easy it is to get stuff in Croatia, but this should give 
> you some ideas: 
> https://geizhals.eu/?cat=hdssd&xf=2028_256~4643_Power-Loss+Protection~4832_1~4836_2&sort=r
>
> 250 GiB is plenty of the OS, but anything smaller is hard to find or 
> of dubious quality ;)
>
> Cheers,
> Aaron
>
>
> On 2/7/22 01:10, Tonči Stipičević wrote:
>> I'm planning to use SUPERMICRO 721 box (4 x 2T sata) as PBS
>>
>> So, if I create zfs10 raid as boot drive , I have no more disks left 
>> for datastore, right ?
>>
>> BUT, is it possible to make directory on this / boot drive i.e. 
>> /mnt/datastore   and attach it as datastore ?   Is that a way-to-go  
>> and is there any disadvantages against separate 2 x 240g ssd as boot 
>> and 4 x 2T sata (zfs raid 10 ) for datastore?
>>
>> Actually I'ma having hard time finding 2 x 2,5" small  sata drives 
>> for boot ....
>>
>>    Thank you very much in advance
>>
>> BR
>>
>> Tonci
>>
>> srdačan pozdrav / best regards
>>
>> Tonči Stipičević, dipl. ing. elektr.
>> direktor / manager
>>
>> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
>>
>> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
>> Small & Medium Business IT Support / Management
>>
>> mob: 091 1234003
>> www.suma-informatika.hr
>>
>>
>>
>> _______________________________________________
>> pve-user mailing list
>> pve-user@lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>

Re: [PVE-User] PBS datastore

[Aaron Lauterer]

On 2/7/22 12:24, Tonči Stipičević wrote:
> Thank you Aaron,
> 
> this atribut "small" is blocking point ... like you said it is hard to find sata disk under 250G
> 
> I was also thinking about 2 x sata-dom 32G/64G   for zfs boot ? Does it make any sense (performance/relayibilty-wise?

SATA DOMs need to be considered carefully. If they can last long enough, then why not. But if they are just cheap SSDs that will wear out fast... well.

> 
> Ok , today I'm gonna install PBS on 4x2T (disks, not ssds) zfs raid10 and create one new dataset in /rpool  -> /rpool/databck    ...  So , the PBS itself and datastore will reside on the same pool ...  Then I'll make some copmarison tests (backup/restore)  and report back some relevant results  ...  I'd like to avoid PBS on its own pool because this SuperMicro 721 case has only 4 disk slots , a additional ones (2) have to be connected directly to MB ...
> 
> 
> 'till then
> 
> BR
> 
> Tonci
> 
> srdačan pozdrav / best regards
> 
> Tonči Stipičević, dipl. ing. elektr.
> direktor / manager
> 
> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
> 
> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
> Small & Medium Business IT Support / Management
> 
> mob: 091 1234003
> www.suma-informatika.hr
> 
> On 07. 02. 2022. 09:44, Aaron Lauterer wrote:
>> A PBS datastore is just a directory somewhere.
>>
>> If you set up the system on a RAID 10 and want to use that as well for the datastore, I would create a new ZFS dataset for the datastore. This allows you to change certain ZFS properties on that dataset specifically.
>>
>> There are a few things to consider if you want to have the OS on the same disks as the datastore.
>> How fast are the disks? HDDs or SSDs?
>> If they are HDDs, then having the OS on separate disks is a good idea because HDDs will already be having a hard time to provide the IOPS for decent performance. If they are *decent* SSDs, you should be fine.
>>
>> No idea how easy it is to get stuff in Croatia, but this should give you some ideas: https://geizhals.eu/?cat=hdssd&xf=2028_256~4643_Power-Loss+Protection~4832_1~4836_2&sort=r
>>
>> 250 GiB is plenty of the OS, but anything smaller is hard to find or of dubious quality ;)
>>
>> Cheers,
>> Aaron
>>
>>
>> On 2/7/22 01:10, Tonči Stipičević wrote:
>>> I'm planning to use SUPERMICRO 721 box (4 x 2T sata) as PBS
>>>
>>> So, if I create zfs10 raid as boot drive , I have no more disks left for datastore, right ?
>>>
>>> BUT, is it possible to make directory on this / boot drive i.e. /mnt/datastore   and attach it as datastore ?   Is that a way-to-go and is there any disadvantages against separate 2 x 240g ssd as boot and 4 x 2T sata (zfs raid 10 ) for datastore?
>>>
>>> Actually I'ma having hard time finding 2 x 2,5" small  sata drives for boot ....
>>>
>>>    Thank you very much in advance
>>>
>>> BR
>>>
>>> Tonci
>>>
>>> srdačan pozdrav / best regards
>>>
>>> Tonči Stipičević, dipl. ing. elektr.
>>> direktor / manager
>>>
>>> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
>>>
>>> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
>>> Small & Medium Business IT Support / Management
>>>
>>> mob: 091 1234003
>>> www.suma-informatika.hr
>>>
>>>
>>>
>>> _______________________________________________
>>> pve-user mailing list
>>> pve-user@lists.proxmox.com
>>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
> 

Re: [PVE-User] PBS datastore

[Tonči Stipičević]

Yes, I'm aware of that too ...

Obviously this's  not been  good-practice and you do not have any 
recommended models/types ?

But they are very "practical" though :)

srdačan pozdrav / best regards

Tonči Stipičević, dipl. ing. elektr.
direktor / manager

SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263

Podrška / Upravljanje IT sustavima za male i srednje tvrtke
Small & Medium Business IT Support / Management

mob: 091 1234003
www.suma-informatika.hr

On 07. 02. 2022. 13:36, Aaron Lauterer wrote:
>
>
> On 2/7/22 12:24, Tonči Stipičević wrote:
>> Thank you Aaron,
>>
>> this atribut "small" is blocking point ... like you said it is hard 
>> to find sata disk under 250G
>>
>> I was also thinking about 2 x sata-dom 32G/64G   for zfs boot ? Does 
>> it make any sense (performance/relayibilty-wise?
>
> SATA DOMs need to be considered carefully. If they can last long 
> enough, then why not. But if they are just cheap SSDs that will wear 
> out fast... well.
>
>>
>> Ok , today I'm gonna install PBS on 4x2T (disks, not ssds) zfs raid10 
>> and create one new dataset in /rpool  -> /rpool/databck    ...  So , 
>> the PBS itself and datastore will reside on the same pool ...  Then 
>> I'll make some copmarison tests (backup/restore)  and report back 
>> some relevant results ...  I'd like to avoid PBS on its own pool 
>> because this SuperMicro 721 case has only 4 disk slots , a additional 
>> ones (2) have to be connected directly to MB ...
>>
>>
>> 'till then
>>
>> BR
>>
>> Tonci
>>
>> srdačan pozdrav / best regards
>>
>> Tonči Stipičević, dipl. ing. elektr.
>> direktor / manager
>>
>> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
>>
>> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
>> Small & Medium Business IT Support / Management
>>
>> mob: 091 1234003
>> www.suma-informatika.hr
>>
>> On 07. 02. 2022. 09:44, Aaron Lauterer wrote:
>>> A PBS datastore is just a directory somewhere.
>>>
>>> If you set up the system on a RAID 10 and want to use that as well 
>>> for the datastore, I would create a new ZFS dataset for the 
>>> datastore. This allows you to change certain ZFS properties on that 
>>> dataset specifically.
>>>
>>> There are a few things to consider if you want to have the OS on the 
>>> same disks as the datastore.
>>> How fast are the disks? HDDs or SSDs?
>>> If they are HDDs, then having the OS on separate disks is a good 
>>> idea because HDDs will already be having a hard time to provide the 
>>> IOPS for decent performance. If they are *decent* SSDs, you should 
>>> be fine.
>>>
>>> No idea how easy it is to get stuff in Croatia, but this should give 
>>> you some ideas: 
>>> https://geizhals.eu/?cat=hdssd&xf=2028_256~4643_Power-Loss+Protection~4832_1~4836_2&sort=r
>>>
>>> 250 GiB is plenty of the OS, but anything smaller is hard to find or 
>>> of dubious quality ;)
>>>
>>> Cheers,
>>> Aaron
>>>
>>>
>>> On 2/7/22 01:10, Tonči Stipičević wrote:
>>>> I'm planning to use SUPERMICRO 721 box (4 x 2T sata) as PBS
>>>>
>>>> So, if I create zfs10 raid as boot drive , I have no more disks 
>>>> left for datastore, right ?
>>>>
>>>> BUT, is it possible to make directory on this / boot drive i.e. 
>>>> /mnt/datastore   and attach it as datastore ?   Is that a way-to-go 
>>>> and is there any disadvantages against separate 2 x 240g ssd as 
>>>> boot and 4 x 2T sata (zfs raid 10 ) for datastore?
>>>>
>>>> Actually I'ma having hard time finding 2 x 2,5" small  sata drives 
>>>> for boot ....
>>>>
>>>>    Thank you very much in advance
>>>>
>>>> BR
>>>>
>>>> Tonci
>>>>
>>>> srdačan pozdrav / best regards
>>>>
>>>> Tonči Stipičević, dipl. ing. elektr.
>>>> direktor / manager
>>>>
>>>> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
>>>>
>>>> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
>>>> Small & Medium Business IT Support / Management
>>>>
>>>> mob: 091 1234003
>>>> www.suma-informatika.hr
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> pve-user mailing list
>>>> pve-user@lists.proxmox.com
>>>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>>
>>
>

Re: [PVE-User] PBS datastore

[Aaron Lauterer]

On 2/7/22 13:42, Tonči Stipičević wrote:
> Yes, I'm aware of that too ...
> 
> Obviously this's  not been  good-practice and you do not have any recommended models/types ?

No, I actually never used any, only looked into them a few times, but never found anything that I really liked.

Another thing though that could work, especially with newer boards, are NVMEs in m.2 format. The last server boards that I looked into and eventually also bought, have m.2 NVME slots.

> 
> But they are very "practical" though :)
> 
> srdačan pozdrav / best regards
> 
> Tonči Stipičević, dipl. ing. elektr.
> direktor / manager
> 
> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
> 
> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
> Small & Medium Business IT Support / Management
> 
> mob: 091 1234003
> www.suma-informatika.hr
> 
> On 07. 02. 2022. 13:36, Aaron Lauterer wrote:
>>
>>
>> On 2/7/22 12:24, Tonči Stipičević wrote:
>>> Thank you Aaron,
>>>
>>> this atribut "small" is blocking point ... like you said it is hard to find sata disk under 250G
>>>
>>> I was also thinking about 2 x sata-dom 32G/64G   for zfs boot ? Does it make any sense (performance/relayibilty-wise?
>>
>> SATA DOMs need to be considered carefully. If they can last long enough, then why not. But if they are just cheap SSDs that will wear out fast... well.
>>
>>>
>>> Ok , today I'm gonna install PBS on 4x2T (disks, not ssds) zfs raid10 and create one new dataset in /rpool  -> /rpool/databck    ...  So , the PBS itself and datastore will reside on the same pool ...  Then I'll make some copmarison tests (backup/restore)  and report back some relevant results ...  I'd like to avoid PBS on its own pool because this SuperMicro 721 case has only 4 disk slots , a additional ones (2) have to be connected directly to MB ...
>>>
>>>
>>> 'till then
>>>
>>> BR
>>>
>>> Tonci
>>>
>>> srdačan pozdrav / best regards
>>>
>>> Tonči Stipičević, dipl. ing. elektr.
>>> direktor / manager
>>>
>>> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
>>>
>>> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
>>> Small & Medium Business IT Support / Management
>>>
>>> mob: 091 1234003
>>> www.suma-informatika.hr
>>>
>>> On 07. 02. 2022. 09:44, Aaron Lauterer wrote:
>>>> A PBS datastore is just a directory somewhere.
>>>>
>>>> If you set up the system on a RAID 10 and want to use that as well for the datastore, I would create a new ZFS dataset for the datastore. This allows you to change certain ZFS properties on that dataset specifically.
>>>>
>>>> There are a few things to consider if you want to have the OS on the same disks as the datastore.
>>>> How fast are the disks? HDDs or SSDs?
>>>> If they are HDDs, then having the OS on separate disks is a good idea because HDDs will already be having a hard time to provide the IOPS for decent performance. If they are *decent* SSDs, you should be fine.
>>>>
>>>> No idea how easy it is to get stuff in Croatia, but this should give you some ideas: https://geizhals.eu/?cat=hdssd&xf=2028_256~4643_Power-Loss+Protection~4832_1~4836_2&sort=r
>>>>
>>>> 250 GiB is plenty of the OS, but anything smaller is hard to find or of dubious quality ;)
>>>>
>>>> Cheers,
>>>> Aaron
>>>>
>>>>
>>>> On 2/7/22 01:10, Tonči Stipičević wrote:
>>>>> I'm planning to use SUPERMICRO 721 box (4 x 2T sata) as PBS
>>>>>
>>>>> So, if I create zfs10 raid as boot drive , I have no more disks left for datastore, right ?
>>>>>
>>>>> BUT, is it possible to make directory on this / boot drive i.e. /mnt/datastore   and attach it as datastore ?   Is that a way-to-go and is there any disadvantages against separate 2 x 240g ssd as boot and 4 x 2T sata (zfs raid 10 ) for datastore?
>>>>>
>>>>> Actually I'ma having hard time finding 2 x 2,5" small  sata drives for boot ....
>>>>>
>>>>>    Thank you very much in advance
>>>>>
>>>>> BR
>>>>>
>>>>> Tonci
>>>>>
>>>>> srdačan pozdrav / best regards
>>>>>
>>>>> Tonči Stipičević, dipl. ing. elektr.
>>>>> direktor / manager
>>>>>
>>>>> SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263
>>>>>
>>>>> Podrška / Upravljanje IT sustavima za male i srednje tvrtke
>>>>> Small & Medium Business IT Support / Management
>>>>>
>>>>> mob: 091 1234003
>>>>> www.suma-informatika.hr
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> pve-user mailing list
>>>>> pve-user@lists.proxmox.com
>>>>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>>>
>>>
>>
>