From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <a.lauterer@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id 57AC794DA7
 for <pve-user@lists.proxmox.com>; Mon, 27 Feb 2023 16:14:59 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id 2EB602CDF3
 for <pve-user@lists.proxmox.com>; Mon, 27 Feb 2023 16:14:29 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [94.136.29.106])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS
 for <pve-user@lists.proxmox.com>; Mon, 27 Feb 2023 16:14:27 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id E6B70489A9;
 Mon, 27 Feb 2023 16:14:26 +0100 (CET)
Message-ID: <03515357-9a0b-e556-92aa-37decb9cb20c@proxmox.com>
Date: Mon, 27 Feb 2023 16:14:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.7.2
Content-Language: en-US
To: Proxmox VE user list <pve-user@lists.proxmox.com>,
 =?UTF-8?B?VG9uxI1pIFN0aXBpxI1ldmnEhw==?= <tonci@suma-informatika.hr>,
 PVE User List <pve-user@pve.proxmox.com>
References: <be9260f3-2f46-684f-5220-0b1e00477800@suma-informatika.hr>
From: Aaron Lauterer <a.lauterer@proxmox.com>
In-Reply-To: <be9260f3-2f46-684f-5220-0b1e00477800@suma-informatika.hr>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results:  0
 AWL -0.294 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years
 NICE_REPLY_A           -0.001 Looks like a legit reply (A)
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
Subject: Re: [PVE-User] add disk back to raid10
X-BeenThere: pve-user@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE user list <pve-user.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>, 
 <mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-user/>
List-Post: <mailto:pve-user@lists.proxmox.com>
List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>, 
 <mailto:pve-user-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Mon, 27 Feb 2023 15:14:59 -0000

I see that the zpool is on the third partition on the disk 
ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3.

If you created the raid 10 from the installer, please follow the steps in the 
admin guide [0] in the section "Changing a failed bootable device".

But instead of "zpool replace" you should run

zpool attach rpool ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3 
/dev/disk/by-id/{new disk}-part3

The man page for the 'zpool attach' command can be viewed with 'man zpool-attach'



[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs

On 2/27/23 15:45, Tonči Stipičević wrote:
> Hello to all,
> 
> I'm playing with zfs arrays. Here you can see former RAID10 because there is 
> mirror-0 missing as such , because I unintentionally detached all disk 
> references from this mirror-0 that used to be 2-disk mirror vdev for this raid10.
> 
> 1. raid10 was OK  .... this is also boot raid10
> 
> 2.  I physically removed one disk disk and mirror-0 became degraded
> 
> 3. after lot of steps in order to replace one disk, put another-new and get all 
> boot staff needed  and after lot of unsuccessful tries I unintentionally ended 
> up detaching all references of old "faulted" disk and mirror-0 was gone
> 
> 4. Now I have unnormal raid with two striped vdevs (see below), but one of them 
> is just one disk , and the other one is regular mirror(-1)
> 
> So, now I'm trying to attach/add new disk to this raid10 but in order to get 
> mirror-0 like it was before  ....   How can I reference "add" or "attach" to 
> make mirror-0 again (consisting of 
> ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3  and the new disk )  ?
> 
> 
> Thank you very much in advance
> 
> BR
> 
> Tonci
> 
> 
>    pool: rpool
>   state: ONLINE
>    scan: resilvered 2.34G in 01:22:54 with 0 errors on Sun Feb 26 14:13:11 2023
> config:
> 
>          NAME STATE     READ WRITE CKSUM
>          rpool ONLINE       0     0     0
>            ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D7E3DU-part3 ONLINE 0     0     0
>            mirror-1 ONLINE       0     0     0
>              ata-WDC_WD2000F9YZ-09N20L1_WD-WMC1P0F7KHZM-part3 ONLINE       0     
> 0     0
>              ata-WDC_WD2000F9YZ-09N20L1_WD-WMC5C0D6WZT4-part3 ONLINE       0     
> 0     0
>          special
>            mirror-2 ONLINE       0     0     0
>              sde ONLINE       0     0     0
>              sdf ONLINE       0     0     0
> 
> errors: No known data errors
>