Re: [pve-devel] [PATCH v2 pve-common 00/12] Introduce and Package PVE::Path & PVE::Filesystem

[Fiona Ebner <f.ebner@proxmox.com>]

Am 02.01.25 um 16:54 schrieb Max Carrara:
> On Thu Jan 2, 2025 at 2:53 PM CET, Fiona Ebner wrote:
>> Am 02.01.25 um 14:46 schrieb Fiona Ebner:
>>> Am 20.12.24 um 19:51 schrieb Max Carrara:
>>>> Introduce and Package PVE::Path & PVE::Filesystem - v2
>>>> ======================================================
>>>
>>> Just an idea, but I'd like to have a discussion about it: Instead of
>>> using Perl for such new general helper modules, would it be nicer to use
>>> Rust+perlmod?
>>>
>>> If our long-term goal is to migrate the Proxmox VE Perl code to Rust,
>>> then we need to switch these modules over at some point in any case (or
>>> drop them after switching over all users of the modules). Are there good
>>> reasons not to start out with Rust+perlmod already?
>>>
> 
> Depends on what you mean with nicer: I was reluctant to use perlmod
> here for a couple reasons:
> 
> 1. We appear to have everything in the pve-rs crate right now
> (libpve-rs-perl), so I had assumed that if I wanted to use perlmod here,
> then I'd have to put my implementations into that crate as well.
> 
> This in turn would mean that for a simple path op library I'd need to
> pull in pve-rs as dependency, which also contains a bunch of different
> things that aren't concerned with path op stuff.
> 
> Perhaps I'm misunderstanding the purpose of the pve-rs crate, but I
> decided against using perlmod here solely because I didn't want to add
> any additional dependencies to this library unless otherwise necessary.
> 
> Right now as of this series, no additional dependencies besides some
> Perl core modules are needed; the library can exist on its own.
> 
> 2. I'm uncertain whether we actually want to have multiple repositories
> or packages using perlmod (instead of having just pve-rs).
> 
> If we can use perlmod for individual modules, as in, add perlmod *alone*
> as a dependency for packages like this one, then implement features and
> add dependencies selectively, I'd be open to it.
> 
> Perhaps as an example, what I'd ideally prefer is something like
> Python's cryptography is using PyO3 -- there's a Rust part and then
> there's a Python part that's using the things implemented in Rust; only
> whatever's necessary is pulled in [1].


That is a good point. I agree this is also worth discussing. Do we want
to put all bindings for PVE there or do we want to have multiple
libraries for binding? Moving forward, more and more Perl packages will
need to depend on pve-rs or the other binding libraries in any case, so
I wouldn't consider that to be a blocker for the path library here.

> 3. Related to 1. and 2., there isn't any clear indication / guide / rule
> of thumb / etc. on how perlmod ought to be used and in which contexts it
> should be used.

Yes, that's why I wanted to have a discussion :)

> 4. Should we decide to use perlmod here eventually, individual functions
> can still be implemented in Rust separately. Right now, there wasn't
> really a need to use Rust, because PVE::Path works at most with strings
> and a couple arrays here and there; there are no complex data structures
> that need to be made typesafe.

The motivation for my suggestion is not about type-safety, but about
doing the groundwork for the future. Since we can call Rust from Perl
but not the other way around, we need to start with the low-level
modules to use Rust. If we go for it, I'd rather have all functions in
the new libraries be wrappers and as transparent as possible, so that a
module that uses the path functions can be converted to Rust more easily
in the future.

>>> You state that you (also) took inspiration from Rust's `std::path` so
>>> could we just use that itself, wrapping via perlmod? Or would the
>>> wrapping be too ugly here or lead to performance issues?
> 
> 5. I'm not sure about the performance overhead, but it would certainly
> be somewhat ugly, because all which PVE::Path essentially does consists
> of string and array operations. If we used perlmod here hypothetically,
> all that we'd be doing is give the Rust side a string or an array,
> convert that to a PathBuf / Path or an iterable, perform the requested
> operation and give the result back to Perl. It just seems a little
> unnecessary to me.
>
> 6. I reckon that the places in which those two little libraries here
> will be used will most likely be replaced by a pure Rust implementation
> as a whole -- IMO there's no need to use perlmod for every single
> smaller library if the Perl code using them gets replaced by Rust.

The question is how much easier does converting modules using these get
if we start out with wrappers? Because sure, right now it is limited in
scope, but if we start out with Perl, things will get added on top and
this might result in more work in the future.

> In other words, IMO a top-down approach such as replacing higher-level
> subroutines or entire API calls would probably yield better results
> rather than a bottom-up approach. (I believe there's a pattern for this
> -- strangler pattern? I'd have to look it up tbh)

See above, we can't too easily do that. We need to have all the parts an
API call needs already in Rust or we won't be able to implement it. Of
course path modification could be converted to Rust "on-the-fly", but if
we already have transparent wrappers it could even be trivial.

>> Or depending on whether it's nicer, also wrapping helpers from
>> proxmox-sys and friends where we already have similar functionality in
>> our Rust code.
> 
> 7. While I'm a big fan of re-using existing code, I don't think it
> applies here -- I think it's fine to keep *certain* things separate and
> decoupled from one another until we actually find that there's a lot of
> common functionality between two or more things (speaking generally
> here). For PVE::Path and PVE::Filesystem in particular, we can always
> bridge over to Rust via perlmod for individual functions if needed (4.)
> nevertheless, if that even ends up being necessary (6.).
> 
> With all that being said, I hope I could convey my reasoning here and
> shine some light on my design decisions -- please let me know what you
> think! And thanks for having a look :)
> 
> [1]: https://github.com/pyca/cryptography/tree/7fd5f95354e33d9ca90ba854e9cbda958968043a/src

All that said, yes, it might be a too small library in this case. OTOH,
it might be a good place to start.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel