From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id C075261539 for ; Sun, 13 Sep 2020 23:29:28 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id AC103AC86 for ; Sun, 13 Sep 2020 23:28:58 +0200 (CEST) Received: from mx0.it-functions.nl (mx0.it-functions.nl [178.32.167.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 19917AC78 for ; Sun, 13 Sep 2020 23:28:58 +0200 (CEST) Received: from [217.100.26.194] (helo=daruma-old.hachimitsu.nl) by mx0.it-functions.nl with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1kHZHk-0005sW-2W for pve-user@lists.proxmox.com; Sun, 13 Sep 2020 23:11:44 +0200 Received: from [192.168.254.32] by daruma-old.hachimitsu.nl with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1kHZHh-0005Qm-6Z for pve-user@lists.proxmox.com; Sun, 13 Sep 2020 23:11:41 +0200 To: Proxmox VE user list From: Stephan Leemburg Organization: IT Functions Message-ID: <35ff9a9f-60ba-5588-b5f7-d5c9b7483c84@it-functions.nl> Date: Sun, 13 Sep 2020 23:11:41 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: nl X-Scan-Signature: e42e3f506be2fbbba755f1d514225e3e X-GeoIP: NL X-Virus-Scanned: by clamav-new X-Scan-Signature: 1f0988f2cf46762dbdbdfb527b3004e3 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.590 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [PVE-User] zfs root after last update grub unknown filesystem X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Sep 2020 21:29:28 -0000 Hi All, I have a proxmox system at netcup that was clean installed about two weeks ago. It is full zfs, so root on zfs. I am migrating from one netcup system with less storage to this system. This system is using the pve-no-subscription repository, but after migration I will move the subscription from the 'old' system to this system. The rpool/data dataset is zfs encrypted. Today I did zfs send/recv from the 'old' system to this system to the rpool/data dataset. After that I did an apt update and noticed there where updates available. After the upgrade and the mandatory reboot, the system does not come up anymore. It is stuck in grub rescue. Grub mentions that it has a 'unknown filesystem'. Has anyone else experienced this same situation? If so and you could recover, what was the reason and fix? I am still researching what is causing this. If I boot a .iso then I can import the pool and see all datasets and subvolumes. So, it seems that the zpool itself and the datasets are ok, it is just that grub is unable to recognize them for some reason. I have read about other situations like this where large_dnode seemed to be the cause. I noticed that on the zpool large_dnode is enabled. And it is a creation only setting. It cannot be changed to disabled afterwards. This must have been done by the Proxmox installer. As booting is about the root filesystem, I guess the zfs send / recv to the rpool/data dataset would have nothing to do with it, but I could be wrong. I will continue my research tomorrow evening after some other obligations, but if anyone has an idea, please share it. If it is just how to get more debugging info out of the zfs module in grub. Because 'unknown filesystem', with the zfs module loaded is kind of not helping enough.. Best regards, Stephan