Re: [pve-devel] MooseFS storage plugin

Re: [pve-devel] MooseFS storage plugin

[Thomas Lamprecht]

Hi,

> On Tue, Apr 19, 2022 at 3:49 AM Benjamin via pve-devel
>> I'd like to try developing a MooseFS storage plugin for Proxmox, ideally
>> getting it good enough to be A) included by the project B) supported in
>> some way by the MooseFS folks.

For Storage plugins the basic rules to be considered included are:
* Plugin code itself must be AGPLv3, if it's self written that'd be already
  covered by submitting it for inclusion alongside the CLA.

* The underlying storage tech must be open source, as while some provider allow
  an open API or FOSS client, the backend needs to be FOSS too. This is no "legal
  issue" per se, but otherwise we cannot test it freely nor recommend to our
  users, and at that point its better to have it as external plugin.

* Must be actively developed

* Ideally brings some features/characteristics that the existing storage
  plugins don't have.

MooseFS seems to be split into the core GPLv2 licensed part and a proprietary
one, fwict. How big is the difference there, can someone use the free part to
build a enterprise-grade service without needing to get proprietary code?
(spending money is naturally fine, don't get me wrong).

Also, its feature set reminds a bit of ceph, which we already have support for
not only accessing but also managing. Is there an advantage of MooseFS over
ceph RBD and cephFS technology or is it more preference?

With that info it could be easier to decide if we'd include a MooseFS plugin
directly, for now I cannot answer that. But even if we wouldn't include it,
the external plugin mechanism makes it relatively easy to make PVE support it
nonetheless.

>> As a long-time user of Proxmox but a newbie to Proxmox development I'm not
>> completely sure where to start, but
>> https://pve.proxmox.com/wiki/Developer_Documentation says I should email
>> here before I do anything.
>>
>> I've looked at the code for the GlusterFS and CephFS plugins and they don't
>> look *too* complicated, although I'm fairly new to Perl so I can't say for
>> sure.

Yes, the plugin ABI isn't that complex, and we often avoid more complex or
unordinary perl language features; albeit I may be biased with that :)

On 19/04/2022 04:15, Benjamin wrote:
> I've hacked together a really basic first try.
> 
> https://github.com/Zorlin/pve-moosefs/blob/main/MooseFSPlugin.pm
> 
> I'm getting this error which I don't fully understand -
> 
> Can't locate object method "mkpath" via package "/mnt/mfs" (perhaps you
> forgot to load "/mnt/mfs"?) at
> /usr/share/perl5/PVE/Storage/Custom/MooseFSPlugin.pm line 111. (500)
> 

I guess that got now already fixed by your commit that actually imports
File::Path[0], which provides mkpath() and auto-exports it.

https://github.com/Zorlin/pve-moosefs/commit/a1621cfba41c88414e1bdc8fd857fa341d62bbf2

cheers,
Thomas

Re: [pve-devel] MooseFS storage plugin

[Fabian Grünbichler]

On April 19, 2022 4:15 am, Benjamin via pve-devel wrote:
> I've hacked together a really basic first try.
> 
> https://github.com/Zorlin/pve-moosefs/blob/main/MooseFSPlugin.pm
> 
> I'm getting this error which I don't fully understand -
> 
> Can't locate object method "mkpath" via package "/mnt/mfs" (perhaps you
> forgot to load "/mnt/mfs"?) at
> /usr/share/perl5/PVE/Storage/Custom/MooseFSPlugin.pm line 111. (500)

the current state seems to be basically equivalent to the following 
(which doesn't require a custom plugin ;)):
- ensure moosefs gets mounted (.mount unit, fstab, ..) at boot
- dir plugin entry with is_mountpoint set

the first part is handled by activate_storage in your plugin, the second 
part makes the dir plugin only active (or activatable) if something is 
mounted at the configured path (otherwise, failure to mount the moosefs 
would lead to using the local empty mountpoint dir, which is of course 
not desired!)

might be an avenue worth exploring if all the interaction between PVE 
and the storage besides the storage activation itself is via dir 
semantics/the dir plugin code ;) of course if there are more aspects 
that do require interacting with the moosefs server when doing certain 
storage operations, a custom plugin is the way to go (with all the 
caveats Thomas already mentioned).