Re: [pve-devel] Allow dynamic pool name discovery on ZFSPoolPlugin

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

Hi,

On 21.11.20 14:08, Pablo Ruiz wrote:
> Hi,
> 
> I've just made a little custom storage plugin which basically
> overrides/extends ZFSPoolPluging so the pool name instead of being fixed,
> can be 'dynamically' guessed on each cluster node.
> 
> While adding some new nodes to one of our current clusters, due to hardware
> differences, we ended up with new nodes having a different zpool's layout
> than the already existing nodes. Our existing nodes had a pool named
> 'rpool/data' (a dataset nested into system's main pool). While the new
> nodes have a dedicated pool (data) independent of 'rpool'. So we needed a
> way to have the same storage use different backing zfs pools on each server
> (ie. on old servers it should use 'rpool/data', while on newer ones it
> should be using 'data').
> 
> Currently proxmox does not allow overriding an storage.cfg's property
> 'per-node'. So we came up with a simple custom plugin which basically looks
> up the pool with an attribute like 'pve:id=$storeid', and uses this pool
> obtained dynamically.
> 
> We could have simply added a new store with a different id, but that would
> break our orchestration and automatic deployment of machines, and add some
> additional management issues by having to track the store of each machine,
> specially if/when moving vms around.
> 
> That said, the plugin works, but I think this feature may be of use to
> others. Would a patch against upstream ZFSPoolPlugin accepted, so we can
> avoid this custom plugin in the future?
> 
> I've posted the custom plugin as a gist:
> https://gist.github.com/pruiz/5d7fbd75efb413ac15d2d0e3ef54f32a
> 

thanks for sharing your solution!

It's an interesting way to solve this, the thing I do not really like, is
that the storage config is now not the only single source of truth for
"which pool to use/import" anymore

We could actually add a nodename to pool mapping as a property to the
ZFS Pool storage config schema instead, that'd would keep this information
contained to the configuration and make lookup a bit cheaper.

cheers,
Thomas