From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pablo.ruiz@gmail.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id AB227624FA for <pve-devel@lists.proxmox.com>; Mon, 23 Nov 2020 17:35:08 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A8DBA2B4B3 for <pve-devel@lists.proxmox.com>; Mon, 23 Nov 2020 17:35:08 +0100 (CET) Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id BF9E92B49E for <pve-devel@lists.proxmox.com>; Mon, 23 Nov 2020 17:35:07 +0100 (CET) Received: by mail-wm1-x32d.google.com with SMTP id a3so17821984wmb.5 for <pve-devel@lists.proxmox.com>; Mon, 23 Nov 2020 08:35:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=cplsY5QLit3wpPVLOCkC8EKu3WT4fhgQKbPxMF5qI3M=; b=LSim2m8P6wikyRza/4t3gJ2oIwTuL3w5YJso+/oYmE6c9sTyW4VLCAC1O26hYS8dDX XWmt3zQxP/js9OtTchyOZRJzdpsuT0g8QHkLjs8PkRGDxKXTvL/6hwUudT8mFdCNAvYW 7XMrSRUf6RWjvMplq68oDuU6ParMN4MsdO2UbvOZ3ZLoK1hiWe9sKrMZyVNYgJdp8s9G HB98kt2ZxfBiwQ9E477YnRxCY5Eir2OLGvHlWavR62ppfdSuS1XOslAplABGUuBtIk1k L8rM/CrsFq3rT+h26s8fthQ2q0deafWDXnwu97jM7cpMYYOcVISbik9Or+22ou2/MbWD sQsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cplsY5QLit3wpPVLOCkC8EKu3WT4fhgQKbPxMF5qI3M=; b=qbub0MckKDqX/1PxvXHC8Bpds7dLM0a+HaBacgoLNJpEkHgPX+Wv2cheYZPSVsiehe /O5ST4EySQkRj4kdkeGZVpgshEw+Ocwetk4t8xA67bYrKhpUog7O54ARMQcuFyJcBVQF d2QDSJ8CoSiVXgTWxEFBKYbjfpi7bu4rJxVLdjJuJrCjDerElpaYPR68jxVlJtqSB+4C YwS7Do9bmtayoIZI2g/mpsYNIswtCQh9+L/96G1JqctiT79vxzba/2JLvnmwWrucNWwL zmLaYIY4zhuhkraFfvsA1c4iLylbSm7URMoD9UD324HVwJfBSgU0B3HLrTPqELhCFOgW ruiQ== X-Gm-Message-State: AOAM533zEGac6CO9hwzDK/N0K24trjXeGzIsaBfli7Ai+x0htKp/tB5b UljmpD3okn6iI7JtnaFqRGEDuuuDq4ud8pP5oRLKA1V526xuDw== X-Google-Smtp-Source: ABdhPJz8tn1Y4mX8zm8zThu0z+bFyFgwSkR0MvOwEikHcVZZ1rA0CZmL2c423E7H5axiuGgKNaDVMX3VcDl8R2vXhXE= X-Received: by 2002:a1c:c309:: with SMTP id t9mr72371wmf.149.1606149307489; Mon, 23 Nov 2020 08:35:07 -0800 (PST) MIME-Version: 1.0 References: <CAHfDCQL4-CmvDGA3YynYjiHpEcukDDUpj7aDF+dHZ9XY-=GhFw@mail.gmail.com> <e6adfd7e-4424-e2ae-bbde-11af213832ca@proxmox.com> In-Reply-To: <e6adfd7e-4424-e2ae-bbde-11af213832ca@proxmox.com> From: Pablo Ruiz <pablo.ruiz@gmail.com> Date: Mon, 23 Nov 2020 17:34:56 +0100 Message-ID: <CAHfDCQK98GpCvAHGqz0VqdbB82=JN0f0va2xQnWWaOWmcG14-g@mail.gmail.com> To: Thomas Lamprecht <t.lamprecht@proxmox.com> Cc: Proxmox VE development discussion <pve-devel@lists.proxmox.com>, pve-devel@pve.proxmox.com, Fabian Ebner <f.ebner@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL 0.300 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider HTML_MESSAGE 0.001 HTML included in message RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [pve-devel] Allow dynamic pool name discovery on ZFSPoolPlugin X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> List-Post: <mailto:pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Mon, 23 Nov 2020 16:35:08 -0000 Hi Thomas, I could try that one, but I am not sure I understood what you mean. Can you provide an example of the syntax you envision for storage.cfg? Regards On Mon, Nov 23, 2020 at 8:49 AM Thomas Lamprecht <t.lamprecht@proxmox.com> wrote: > Hi, > > On 21.11.20 14:08, Pablo Ruiz wrote: > > Hi, > > > > I've just made a little custom storage plugin which basically > > overrides/extends ZFSPoolPluging so the pool name instead of being fixed, > > can be 'dynamically' guessed on each cluster node. > > > > While adding some new nodes to one of our current clusters, due to > hardware > > differences, we ended up with new nodes having a different zpool's layout > > than the already existing nodes. Our existing nodes had a pool named > > 'rpool/data' (a dataset nested into system's main pool). While the new > > nodes have a dedicated pool (data) independent of 'rpool'. So we needed a > > way to have the same storage use different backing zfs pools on each > server > > (ie. on old servers it should use 'rpool/data', while on newer ones it > > should be using 'data'). > > > > Currently proxmox does not allow overriding an storage.cfg's property > > 'per-node'. So we came up with a simple custom plugin which basically > looks > > up the pool with an attribute like 'pve:id=$storeid', and uses this pool > > obtained dynamically. > > > > We could have simply added a new store with a different id, but that > would > > break our orchestration and automatic deployment of machines, and add > some > > additional management issues by having to track the store of each > machine, > > specially if/when moving vms around. > > > > That said, the plugin works, but I think this feature may be of use to > > others. Would a patch against upstream ZFSPoolPlugin accepted, so we can > > avoid this custom plugin in the future? > > > > I've posted the custom plugin as a gist: > > https://gist.github.com/pruiz/5d7fbd75efb413ac15d2d0e3ef54f32a > > > > thanks for sharing your solution! > > It's an interesting way to solve this, the thing I do not really like, is > that the storage config is now not the only single source of truth for > "which pool to use/import" anymore > > We could actually add a nodename to pool mapping as a property to the > ZFS Pool storage config schema instead, that'd would keep this information > contained to the configuration and make lookup a bit cheaper. > > cheers, > Thomas > > >