[pve-devel] Vitastor block driver plugin

[pve-devel] Vitastor block driver plugin

[vitalif]

Hi!

You've added a block device driver whitelist in Proxmox 9.0.

Could you please add Vitastor there?

I mean this place: https://git.proxmox.com/?p=pve-storage.git;a=blob;f=src/PVE/Storage.pm;h=1dde2b751a766a28af8d40df7149936691cca772;hb=HEAD#l145

$allowed_qemu_blockdev_options in PVE/Storage.pm.

For Vitastor to work correctly, it needs to contain:

vitastor => { image => 1, 'config-path' => 1, 'etcd-host' => 1, 'etcd-prefix' => 1 }

Vitastor is a high-performance alternative to Ceph with a similar architecture (written totally from scratch) if you never heard of it. :-) I'm the author.

Current installation instructions are here:

https://vitastor.io/en/docs/installation/packages.html

https://vitastor.io/en/docs/installation/proxmox.html

It'd be also cool if you could upstream my block driver plugin, currently I'm releasing it as a QEMU patch + block/vitastor.c. It's not a problem for me to continue doing that just as before, but I'd of course appreciate if you added it to your QEMU packages. :-)

And feel free to ask any questions please :)

-- 
With best regards,
  Vitaliy Filippov

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Vitastor block driver plugin

[Thomas Lamprecht]

Hello,

On 26/08/2025 16:42, vitalif@yourcmc.ru wrote:
> You've added a block device driver whitelist in Proxmox 9.0.
It's more intended for specific driver options, but as it die's if a
QEMU block device driver is not in the list it indeed also acts as
allow list for those.
> Could you please add Vitastor there?
> 
> I mean this place: https://git.proxmox.com/?p=pve-storage.git;a=blob;f=src/PVE/Storage.pm;h=1dde2b751a766a28af8d40df7149936691cca772;hb=HEAD#l145
> 
> $allowed_qemu_blockdev_options in PVE/Storage.pm.
> 
> For Vitastor to work correctly, it needs to contain:
> 
> vitastor => { image => 1, 'config-path' => 1, 'etcd-host' => 1, 'etcd-prefix' => 1 }

Hmm, we could add that, but would prefer only doing so if the plugin
is directly in QEMU already. That said, this is rather internal, so
maintenance cost would be low, so could be still fine... 
I would like some other opinion on it though (CC @Fiona).

> Vitastor is a high-performance alternative to Ceph with a similar architecture (written totally from scratch) if you never heard of it. :-) I'm the author.
> 
> Current installation instructions are here:
> 
> https://vitastor.io/en/docs/installation/packages.html
> 
> https://vitastor.io/en/docs/installation/proxmox.html
> 
> It'd be also cool if you could upstream my block driver plugin, currently I'm releasing it as a QEMU patch + block/vitastor.c. It's not a problem for me to continue doing that just as before, but I'd of course appreciate if you added it to your QEMU packages. :-)


To avoid any false expectations upfront: As of now we won't take your
patches into our downstream build of QEMU.

For us the main reason for why we never evaluated Vitastore more closely
(yes, we heard of it) is that it seems there is a bus factor of 1, as
basically all contributions seen to be "just" from you, and while that
may even have some benefits w.r.t. a consistent and clean design it also
means that if anything should happen (let's really hope not!) that won't
allow (or want) you to continue working on Vitastore from one day to
another, we would need to provide quite a bit of developer power to be
able to understand the whole code base and maintain it further once we
adopted this. As while we're prepared for some bug fix (investigations)
and what not, like we are for every project we integrate, it's still a
big difference w.r.t. risk management when there are not at least 3
somewhat active devs upstream, at least for projects at this scale.

And while yes, reducing the complexity of Ceph while getting the same
(or even more) performance is certainly a great goal (kudos for working
on that and getting this far!), but for us Ceph is still manageable for
now and it's wide adoption is a major benefit that makes much of it's
pain tolerable. A smaller reason is C++, while I certainly do not want
to blame C++ for all problems of Ceph, IMO lots of pain and complexity
stems from that in Ceph, and while I cannot make any judgement for
Vitastore, as I did not check out the source in any relevant manner,
I'm worried that C++ needs a lot of attention and tooling to avoid such
growth of complexity and making maintenance harder, especially once more
developers are working on a project. While we got some expertise, most
of our Devs have no deep C++ experience besides for some drive-by fix
of a specific bug.

But that all said, why not upstream to QEMU directly? IMO that would
help much more people, improve willingness for adoption for users and
reduce overall maintenance cost for you and naturally also for us, if we
would (hypothetically) take this in. AFAICT it would not be _that_ much
change for QEMU.

Anyhow, I wish you the best of luck and I hope Vitastore can overcome
the initial "chicken and egg" inertia that such projects face. Maybe
some of our devs (or experienced users) get to do a closer evaluation
of your project in the context of Proxmox VE on a calmer day.

best regards
Thomas


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Vitastor block driver plugin

[vitalif]

Hi Thomas,

Thanks for your answer!

Regarding the "bus factor", I totally understand your concerns, but I guess it's more about popularity than about the number of developers. I mean if you take a startup with 10-20 devs it may run out of investor money and become abandoned as well as if there is only 1 developer. And I'd of course appreciate more popularity and if more people joined my project :) but after all, opensource is about indie developers too, not only about companies? So I don't think it should be a blocker. Vitastor has users, some of them use Proxmox by the way, there's also a partner company in Russia which offers commercial support for it.

C++ can be different :) the C++ I use is more like "C with classes" than modern idiomatic C++... I try to avoid complex features of the language as well as complex external libraries.

So yeah, try to evaluate it when you have spare time :) it's like 10x faster in terms of latency, has very similar tooling to ceph and PVE seems to be very easy to use Vitastor with. Also you're the only system which doesn't have hardcode and thus doesn't require patching core to install a 3rdparty block driver! It's really cool. OpenNebula, OpenStack, oVirt all require patches, both to their core and to libvirt. I really appreciate your libvirt-free approach. :)

I thought about upstreaming the QEMU driver, but then the first thing that I have to solve is getting my client library packaged in at least one of distros used in their CI: alpine, ubuntu, debian, fedora, centos or opensuse. Then they can consider accepting the driver. It seems I have to try to do it, at least at some point. :)

So to sum it up, at the moment it's perfectly fine for me if you just add that 1 line with vitastor options in that list. :-)

By the way, why did you add it in the first place? I thought these options could only contain "trusted" values coming from PVE code anyway? Or do some drivers really require filtering?

-- 
With best regards,
  Vitaliy Filippov

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Vitastor block driver plugin

[Fiona Ebner]

Hi,

Am 27.08.25 um 11:13 AM schrieb Thomas Lamprecht:
> Hello,
> 
> On 26/08/2025 16:42, vitalif@yourcmc.ru wrote:
>> You've added a block device driver whitelist in Proxmox 9.0.
> It's more intended for specific driver options, but as it die's if a
> QEMU block device driver is not in the list it indeed also acts as
> allow list for those.
>> Could you please add Vitastor there?
>>
>> I mean this place: https://git.proxmox.com/?p=pve-storage.git;a=blob;f=src/PVE/Storage.pm;h=1dde2b751a766a28af8d40df7149936691cca772;hb=HEAD#l145
>>
>> $allowed_qemu_blockdev_options in PVE/Storage.pm.
>>
>> For Vitastor to work correctly, it needs to contain:
>>
>> vitastor => { image => 1, 'config-path' => 1, 'etcd-host' => 1, 'etcd-prefix' => 1 }
> 
> Hmm, we could add that, but would prefer only doing so if the plugin
> is directly in QEMU already. That said, this is rather internal, so
> maintenance cost would be low, so could be still fine... 
> I would like some other opinion on it though (CC @Fiona).

I think it would also be nicer if it were an upstream QEMU block driver.
Personally, I wouldn't mind including it in the allow-list, with a
comment describing the situation, because the chances that another
driver also using 'vitastor' as its name lands upstream first are very low.

Am 28.08.25 um 12:58 AM schrieb vitalif@yourcmc.ru:
> By the way, why did you add it in the first place? I thought these
> options could only contain "trusted" values coming from PVE code
> anyway? Or do some drivers really require filtering?

It can also be third-party plugins. Some settings are handled via the VM
configuration file and should not be set by the plugins, like cache
mode, read-only, etc.. Many of the possible block drivers like
job-related drivers also don't really make sense for the basic building
block that qemu_blockdev_options() is supposed to return.

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel