Re: [pbs-devel] [RFC proxmox{, -backup} 0/6] add support for configuring max

["Fabian Grünbichler" <f.gruenbichler@proxmox.com>]

On September 4, 2025 4:37 pm, Hannes Laimer wrote:
> This uses POSIX semaphores to control how many backups run at any given
> time. Semaphores seemed like a fitting choice given the problem, so I 
> added a basic wrapper for named ones to proxmox-sys. Also, the changes
> needed to add concurrency limits in PBS were minimal.
> 
> A few questions I still have and would be great to get some feedback on,
>  - do we want to use semaphores like proposed here? Are there any good
>    reasons not to?

they are rather primitive, and their file based nature means extra care
is required to not accidentally reset or mishandle them. reconfiguring
is also a bit of a pain, as you need to know the old value for certain
to do so..

we already have shared memory based synchronization mechanisms in our
Rust code, which would give us a lot more features.. e.g., allow things
like registering queued backup sessions if there are no slots available,
and notifying the oldest one if a slot becomes available.. this is
currently used by the config version cache and the rate limiting code.

or if we want to keep it really primitive, we already have a file-based
operations tracking that we could extend, although that would probably
require quite a bit of refactoring to make sense, since we currently do
a lookup/write-operation registration as part of the backup session
opening/connection upgrade..

>  - should we include things like verify/gc/prune? Would an extra sem for
>    reading make sense(so separate limits for r and w, idk if there's a
>    use-case)?
> 
> Nothing UI-wise is included, and for changes to apply the proxy has to
> be restarted. Not sure if restarting the proxy is "ok-ish" ux wise, I
> guess not :P But changing it involves re-creating the semaphore, which
> is fine, just that there won't exist one for a really short time...

what about reloading? we might still have sessions running in the old
worker that are now no longer accounted for?

> 
> 
> proxmox:
> 
> Hannes Laimer (2):
>   sys: add wrapper for POSIX semaphores
>   pbs-api-types: add concurrency_limit to DataStoreConfig
> 
>  pbs-api-types/src/datastore.rs |   5 +
>  proxmox-sys/Cargo.toml         |   1 +
>  proxmox-sys/src/lib.rs         |   2 +
>  proxmox-sys/src/semaphore.rs   | 164 +++++++++++++++++++++++++++++++++
>  4 files changed, 172 insertions(+)
>  create mode 100644 proxmox-sys/src/semaphore.rs
> 
> 
> proxmox-backup:
> 
> Hannes Laimer (4):
>   api: config: update/delete concurrency_limit on datastore
>   Cargo.toml: add 'semaphore' feature to proxmox-sys dep
>   bin: proxy: initialize concurrency semaphores for datastores
>   api: backup: wait for semaphore if one exists
> 
>  Cargo.toml                      |  2 +-
>  src/api2/backup/mod.rs          | 24 ++++++++++++++++++++++--
>  src/api2/config/datastore.rs    |  9 +++++++++
>  src/bin/proxmox-backup-proxy.rs | 11 +++++++++++
>  4 files changed, 43 insertions(+), 3 deletions(-)
> 
> 
> Summary over all repositories:
>   8 files changed, 215 insertions(+), 3 deletions(-)
> 
> -- 
> Generated by git-murpp 0.8.1
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
> 


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel