From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5A7DB620CC for ; Mon, 23 Nov 2020 08:49:40 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4D74724257 for ; Mon, 23 Nov 2020 08:49:40 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 41A3724243 for ; Mon, 23 Nov 2020 08:49:39 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 10CB443EFE for ; Mon, 23 Nov 2020 08:49:39 +0100 (CET) To: Proxmox VE development discussion , Pablo Ruiz , pve-devel@pve.proxmox.com, Fabian Ebner References: From: Thomas Lamprecht Message-ID: Date: Mon, 23 Nov 2020 08:49:37 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Thunderbird/83.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL -0.084 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] Allow dynamic pool name discovery on ZFSPoolPlugin X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Nov 2020 07:49:40 -0000 Hi, On 21.11.20 14:08, Pablo Ruiz wrote: > Hi, >=20 > I've just made a little custom storage plugin which basically > overrides/extends ZFSPoolPluging so the pool name instead of being fixe= d, > can be 'dynamically' guessed on each cluster node. >=20 > While adding some new nodes to one of our current clusters, due to hard= ware > differences, we ended up with new nodes having a different zpool's layo= ut > than the already existing nodes. Our existing nodes had a pool named > 'rpool/data' (a dataset nested into system's main pool). While the new > nodes have a dedicated pool (data) independent of 'rpool'. So we needed= a > way to have the same storage use different backing zfs pools on each se= rver > (ie. on old servers it should use 'rpool/data', while on newer ones it > should be using 'data'). >=20 > Currently proxmox does not allow overriding an storage.cfg's property > 'per-node'. So we came up with a simple custom plugin which basically l= ooks > up the pool with an attribute like 'pve:id=3D$storeid', and uses this p= ool > obtained dynamically. >=20 > We could have simply added a new store with a different id, but that wo= uld > break our orchestration and automatic deployment of machines, and add s= ome > additional management issues by having to track the store of each machi= ne, > specially if/when moving vms around. >=20 > That said, the plugin works, but I think this feature may be of use to > others. Would a patch against upstream ZFSPoolPlugin accepted, so we ca= n > avoid this custom plugin in the future? >=20 > I've posted the custom plugin as a gist: > https://gist.github.com/pruiz/5d7fbd75efb413ac15d2d0e3ef54f32a >=20 thanks for sharing your solution! It's an interesting way to solve this, the thing I do not really like, is= that the storage config is now not the only single source of truth for "which pool to use/import" anymore We could actually add a nodename to pool mapping as a property to the ZFS Pool storage config schema instead, that'd would keep this informatio= n contained to the configuration and make lookup a bit cheaper. cheers, Thomas