[pbs-devel] [PATCH proxmox-backup v3] etc: raise nofile soft limit to hard limit for proxmox-backup-proxy

[pbs-devel] [PATCH proxmox-backup v3] etc: raise nofile soft limit to hard limit for proxmox-backup-proxy

[Christian Ebner]

Since commit 86d5d073 ("GC: fix race with chunk upload/insert on s3
backends"), per-chunk file locks are acquired during phase 2 of
garbage collection for datastores backed by s3 object stores. This
however means that up to 1000 file locks might be held at once, which
can result in the limit of open file handles to be reached.

Therefore, bump the NOFILE soft limit for the proxmox-backup-proxy in
the systemd service unit, while keeping the hard limit as defined in
/etc/systemd/system.conf.

This is acceptable since PBS does not directly depend on problematic
select() calls as verified via `nm` and does not use it in linked
libraries to the best of my knowledge.

Occurrences of the symbol according to `nm -D <shared-object>` are:

/lib/x86_64-linux-gnu/libapt-pkg.so.7.0
                 U select@GLIBC_2.2.5
/lib/x86_64-linux-gnu/libpam.so.0
                 U select@GLIBC_2.2.5
/lib/x86_64-linux-gnu/libc.so.6
000000000010e140 W select@@GLIBC_2.2.5
/lib/x86_64-linux-gnu/libcrypto.so.3
                 U select@GLIBC_2.2.5

[0] https://github.com/systemd/systemd/blob/main/NEWS#L12044

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
Changes since version 2:
- But soft to hard limit
- Extend commit message with respect to select()

 etc/proxmox-backup-proxy.service.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/etc/proxmox-backup-proxy.service.in b/etc/proxmox-backup-proxy.service.in
index 7ca806aa4..8e4bbc197 100644
--- a/etc/proxmox-backup-proxy.service.in
+++ b/etc/proxmox-backup-proxy.service.in
@@ -10,6 +10,7 @@ Type=notify
 ExecStart=%LIBEXECDIR%/proxmox-backup/proxmox-backup-proxy
 ExecReload=/bin/kill -HUP $MAINPID
 PIDFile=/run/proxmox-backup/proxy.pid
+LimitNOFILE=524288
 Restart=on-failure
 User=%PROXY_USER%
 Group=%PROXY_USER%
-- 
2.47.3



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH proxmox-backup v3] etc: raise nofile soft limit to hard limit for proxmox-backup-proxy

[Thomas Lamprecht]

Am 20.11.25 um 15:32 schrieb Christian Ebner:
> This is acceptable since PBS does not directly depend on problematic
> select() calls as verified via `nm` and does not use it in linked
> libraries to the best of my knowledge.
> 

Isn't above and 

> Occurrences of the symbol according to `nm -D <shared-object>` are:
> 
> /lib/x86_64-linux-gnu/libapt-pkg.so.7.0
>                  U select@GLIBC_2.2.5
> /lib/x86_64-linux-gnu/libpam.so.0
>                  U select@GLIBC_2.2.5
> /lib/x86_64-linux-gnu/libc.so.6
> 000000000010e140 W select@@GLIBC_2.2.5
> /lib/x86_64-linux-gnu/libcrypto.so.3
>                  U select@GLIBC_2.2.5

above a contradiction? Or do I just misinterpret this?
As it would seem to me that the usage of select symbols would in fact
show that this might not be safe, or?

If the API calls into any function of those libs, that might might then create
a FD >= 1024 inside which then could get passed down to any of their select
calls?


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH proxmox-backup v3] etc: raise nofile soft limit to hard limit for proxmox-backup-proxy

[Christian Ebner]

On 11/20/25 4:05 PM, Thomas Lamprecht wrote:
> Am 20.11.25 um 15:32 schrieb Christian Ebner:
>> This is acceptable since PBS does not directly depend on problematic
>> select() calls as verified via `nm` and does not use it in linked
>> libraries to the best of my knowledge.
>>
> 
> Isn't above and

With above I intended to state that the PBS code itself does not call 
into select(), while below are dependencies on shared objects which 
might call into select() according to their symbols.

> 
>> Occurrences of the symbol according to `nm -D <shared-object>` are:
>>
>> /lib/x86_64-linux-gnu/libapt-pkg.so.7.0
>>                   U select@GLIBC_2.2.5
>> /lib/x86_64-linux-gnu/libpam.so.0
>>                   U select@GLIBC_2.2.5
>> /lib/x86_64-linux-gnu/libc.so.6
>> 000000000010e140 W select@@GLIBC_2.2.5
>> /lib/x86_64-linux-gnu/libcrypto.so.3
>>                   U select@GLIBC_2.2.5
> 
> above a contradiction? Or do I just misinterpret this?
> As it would seem to me that the usage of select symbols would in fact
> show that this might not be safe, or?
> 
> If the API calls into any function of those libs, that might might then create
> a FD >= 1024 inside which then could get passed down to any of their select
> calls?


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH proxmox-backup v3] etc: raise nofile soft limit to hard limit for proxmox-backup-proxy

[Thomas Lamprecht]

Am 20.11.25 um 16:12 schrieb Christian Ebner:
> On 11/20/25 4:05 PM, Thomas Lamprecht wrote:
>> Am 20.11.25 um 15:32 schrieb Christian Ebner:
>>> This is acceptable since PBS does not directly depend on problematic
>>> select() calls as verified via `nm` and does not use it in linked
>>> libraries to the best of my knowledge.
>>>
>>
>> Isn't above and
> 
> With above I intended to state that the PBS code itself does not call into select(), while below are dependencies on shared objects which might call into select() according to their symbols.
> 

And the systemd news entry you link to in the commit message clearly states:

----8<----
Programs that want to take benefit of the increased limit have to "opt-in" into
high file descriptors explicitly by raising their soft limit. Of course, when
they do that they must acknowledge that they cannot use select() anymore (and
**neither can any shared library they use — or any shared library used by any
shared library they use and so on**).
---->8----

I just checked the apt repo, and it includes various select calls. Most seem
to center around downloading packages and such, but I'd not bet on it that
no such select is anywhere in the code paths we use.

PAM uses select in the pam_loginuid, which might be part of the login call,
albeit it uses it only if require_auditd is enabled (which I don't think it is).
I did not yet checked the others out.

I mean, one option might be to provide our own select wrapper preloaded
overriding the glibc one and keep some FDs below 1024 resereved for that, but
I really really dislike doing such things. Similar in spirit would be providing
a select compatible implementation using poll and ld_preload that, but also far
from great..

Moving either GC, or all the things that might call select as per your list,
into a dedicated process might be the nicer thing to do. But as mentioned offlist
I'll try to walk through the problem and code again tomorrow and see if I can
find some other viable options (or you/fabian got some ideas), as of my current
knowledge I cannot really accept doing this bump.




_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel