[PVE-User] add disk back to raid10

[PVE-User] add disk back to raid10

[Tonči Stipičević]

Hello to all,

I'm playing with zfs arrays. Here you can see former RAID10 because 
there is mirror-0 missing as such , because I unintentionally detached 
all disk references from this mirror-0 that used to be 2-disk mirror 
vdev for this raid10.

1. raid10 was OK  .... this is also boot raid10

2.  I physically removed one disk disk and mirror-0 became degraded

3. after lot of steps in order to replace one disk, put another-new and 
get all boot staff needed  and after lot of unsuccessful tries I 
unintentionally ended up detaching all references of old "faulted" disk 
and mirror-0 was gone

4. Now I have unnormal raid with two striped vdevs (see below), but one 
of them is just one disk , and the other one is regular mirror(-1)

So, now I'm trying to attach/add new disk to this raid10 but in order to 
get mirror-0 like it was before  ....   How can I reference "add" or 
"attach" to make mirror-0 again (consisting of 
ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3  and the new disk )  ?


Thank you very much in advance

BR

Tonci


   pool: rpool
  state: ONLINE
   scan: resilvered 2.34G in 01:22:54 with 0 errors on Sun Feb 26 
14:13:11 2023
config:

         NAME STATE     READ WRITE CKSUM
         rpool ONLINE       0     0     0
           ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3 ONLINE       
0     0     0
           mirror-1 ONLINE       0     0     0
             ata-WDC_WD2000F9YZ-09N20L1_WD-WMC1P0F7KHZM-part3 
ONLINE       0     0     0
             ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D6WZT4-part3 
ONLINE       0     0     0
         special
           mirror-2 ONLINE       0     0     0
             sde ONLINE       0     0     0
             sdf ONLINE       0     0     0

errors: No known data errors

-- 
srdačan pozdrav / best regards

Tonči Stipičević, dipl. ing. elektr.
direktor / manager

SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263

Podrška / Upravljanje IT sustavima za male i srednje tvrtke
Small & Medium Business IT Support / Management

mob: 091 1234003
www.suma-informatika.hr

Re: [PVE-User] add disk back to raid10

[Aaron Lauterer]

I see that the zpool is on the third partition on the disk 
ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3.

If you created the raid 10 from the installer, please follow the steps in the 
admin guide [0] in the section "Changing a failed bootable device".

But instead of "zpool replace" you should run

zpool attach rpool ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3 
/dev/disk/by-id/{new disk}-part3

The man page for the 'zpool attach' command can be viewed with 'man zpool-attach'



[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs

On 2/27/23 15:45, Tonči Stipičević wrote:
> Hello to all,
> 
> I'm playing with zfs arrays. Here you can see former RAID10 because there is 
> mirror-0 missing as such , because I unintentionally detached all disk 
> references from this mirror-0 that used to be 2-disk mirror vdev for this raid10.
> 
> 1. raid10 was OK  .... this is also boot raid10
> 
> 2.  I physically removed one disk disk and mirror-0 became degraded
> 
> 3. after lot of steps in order to replace one disk, put another-new and get all 
> boot staff needed  and after lot of unsuccessful tries I unintentionally ended 
> up detaching all references of old "faulted" disk and mirror-0 was gone
> 
> 4. Now I have unnormal raid with two striped vdevs (see below), but one of them 
> is just one disk , and the other one is regular mirror(-1)
> 
> So, now I'm trying to attach/add new disk to this raid10 but in order to get 
> mirror-0 like it was before  ....   How can I reference "add" or "attach" to 
> make mirror-0 again (consisting of 
> ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3  and the new disk )  ?
> 
> 
> Thank you very much in advance
> 
> BR
> 
> Tonci
> 
> 
>    pool: rpool
>   state: ONLINE
>    scan: resilvered 2.34G in 01:22:54 with 0 errors on Sun Feb 26 14:13:11 2023
> config:
> 
>          NAME STATE     READ WRITE CKSUM
>          rpool ONLINE       0     0     0
>            ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3 ONLINE 0     0     0
>            mirror-1 ONLINE       0     0     0
>              ata-WDC_WD2000F9YZ-09N20L1_WD-WMC1P0F7KHZM-part3 ONLINE       0     
> 0     0
>              ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D6WZT4-part3 ONLINE       0     
> 0     0
>          special
>            mirror-2 ONLINE       0     0     0
>              sde ONLINE       0     0     0
>              sdf ONLINE       0     0     0
> 
> errors: No known data errors
> 

Re: [PVE-User] add disk back to raid10

[Tonči Stipičević]

Thank you very much .... I already did the preparation with sgdisk and 
after "attach" proxmox-boot-tool command did the rest

srdačan pozdrav / best regards

Tonči Stipičević, dipl. ing. elektr.
direktor / manager

SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263

Podrška / Upravljanje IT sustavima za male i srednje tvrtke
Small & Medium Business IT Support / Management

mob: 091 1234003
www.suma-informatika.hr

On 27. 02. 2023. 16:14, Aaron Lauterer wrote:
> I see that the zpool is on the third partition on the disk 
> ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3.
>
> If you created the raid 10 from the installer, please follow the steps 
> in the admin guide [0] in the section "Changing a failed bootable 
> device".
>
> But instead of "zpool replace" you should run
>
> zpool attach rpool ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3 
> /dev/disk/by-id/{new disk}-part3
>
> The man page for the 'zpool attach' command can be viewed with 'man 
> zpool-attach'
>
>
>
> [0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs
>
> On 2/27/23 15:45, Tonči Stipičević wrote:
>> Hello to all,
>>
>> I'm playing with zfs arrays. Here you can see former RAID10 because 
>> there is mirror-0 missing as such , because I unintentionally 
>> detached all disk references from this mirror-0 that used to be 
>> 2-disk mirror vdev for this raid10.
>>
>> 1. raid10 was OK  .... this is also boot raid10
>>
>> 2.  I physically removed one disk disk and mirror-0 became degraded
>>
>> 3. after lot of steps in order to replace one disk, put another-new 
>> and get all boot staff needed  and after lot of unsuccessful tries I 
>> unintentionally ended up detaching all references of old "faulted" 
>> disk and mirror-0 was gone
>>
>> 4. Now I have unnormal raid with two striped vdevs (see below), but 
>> one of them is just one disk , and the other one is regular mirror(-1)
>>
>> So, now I'm trying to attach/add new disk to this raid10 but in order 
>> to get mirror-0 like it was before  ....   How can I reference "add" 
>> or "attach" to make mirror-0 again (consisting of 
>> ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3  and the new disk )  ?
>>
>>
>> Thank you very much in advance
>>
>> BR
>>
>> Tonci
>>
>>
>>    pool: rpool
>>   state: ONLINE
>>    scan: resilvered 2.34G in 01:22:54 with 0 errors on Sun Feb 26 
>> 14:13:11 2023
>> config:
>>
>>          NAME STATE     READ WRITE CKSUM
>>          rpool ONLINE       0     0     0
>>            ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3 ONLINE 
>> 0     0     0
>>            mirror-1 ONLINE       0     0     0
>>              ata-WDC_WD2000F9YZ-09N20L1_WD-WMC1P0F7KHZM-part3 
>> ONLINE       0     0     0
>>              ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D6WZT4-part3 
>> ONLINE       0     0     0
>>          special
>>            mirror-2 ONLINE       0     0     0
>>              sde ONLINE       0     0     0
>>              sdf ONLINE       0     0     0
>>
>> errors: No known data errors
>>
>