[RFC] Centralized management for standalone PBS file clients

[RFC] Centralized management for standalone PBS file clients

[Ciro Iriarte]

Hi all,

This is a proposal for discussion (not a patch). Currently
proxmox-backup-client is a stateless, outbound-only CLI, and PBS operates
purely as a passive storage endpoint. For non-PVE / standalone Linux
hosts, backups must be scheduled locally per-host (e.g. cron or systemd
timers). This creates operational gaps:

 - No central control or unified view of backup schedules across hosts.
 - No central task log visibility for non-PVE client backups.
 - No proactive alerting when a host fails to run an expected backup
   (a crashed host or a dead timer produces no signal at all).

Proposed shared component: client agent
---------------------------------------
Both options below rely on a small, optional agent service running on
target Linux hosts. The agent authenticates an incoming request from a
central manager and invokes proxmox-backup-client locally. The existing
CLI remains the execution engine; the agent only dispatches jobs and
reports status and logs back to the manager.

Option A: manager integrated into PBS / PDM
-------------------------------------------
Extend PBS (or PDM) to act as the central manager.

 - PBS/PDM stores job objects for standalone clients (target host,
   source paths, schedule, retention, namespace, notifications).
 - PBS/PDM triggers jobs on remote agents and ingests results.
 - Task results and logs surface via the existing PBS/PDM API and UI.

Option B: separate standalone manager
-------------------------------------
Keep the PBS/PDM codebase unchanged and build a separate manager.

 - Uses the identical client agent from Option A on target hosts.
 - Handles scheduling, central log aggregation (troubleshooting), and
   compliance (were the last tasks successful / did every host meet its
   SLA window).
 - Interacts with PBS strictly via existing public APIs.

The architectural decision is strictly where the manager lives (PBS/PDM
integration vs. a separate product); the client agent is identical in
both.

Questions for the list
----------------------
 1. Is central scheduling for standalone clients desirable inside
    PBS/PDM (Option A), or is a separate manager preferred (Option B)?
 2. Does an optional lightweight client agent align with Proxmox's
    architectural vision for standalone host backups?
 3. Are there existing design initiatives or preferred patterns for
    remote execution in the ecosystem this should align with?

Happy to prototype whichever direction the maintainers consider viable.

Thanks,
Ciro Iriarte

Re: [RFC] Centralized management for standalone PBS file clients

[Christian Ebner]

Hi,

thanks for reaching out for discussion.

On 8/9/26 9:41 PM, Ciro Iriarte wrote:
> Hi all,
> 
> This is a proposal for discussion (not a patch). Currently
> proxmox-backup-client is a stateless, outbound-only CLI, and PBS operates
> purely as a passive storage endpoint. For non-PVE / standalone Linux
> hosts, backups must be scheduled locally per-host (e.g. cron or systemd
> timers). This creates operational gaps:
> 
>   - No central control or unified view of backup schedules across hosts.

PBS itself is never aware of backup schedules, not even for PVE (jobs 
managed by each individual PVE cluster/standalone host). I think this 
should remain this way, the PBS server should be agnostic to clients.

>   - No central task log visibility for non-PVE client backups.

PBS itself does track each backup task in its log, but I see that this 
is not what you might intend here, just mentioning for completeness.

>   - No proactive alerting when a host fails to run an expected backup
>     (a crashed host or a dead timer produces no signal at all).

In my opinion this is again something to be managed by a different 
entity, not PBS itself.

> Proposed shared component: client agent
> ---------------------------------------
> Both options below rely on a small, optional agent service running on
> target Linux hosts. The agent authenticates an incoming request from a
> central manager and invokes proxmox-backup-client locally. The existing
> CLI remains the execution engine; the agent only dispatches jobs and
> reports status and logs back to the manager.

Rather than a dedicated agent, such agent functionality could maybe be 
integrated in the client itself, providing a command to connect to a 
scheduler instance and setup a systemd unit and timer to periodically 
connect to said scheduler instance. Most of your clients might not be 
reachable from your scheduling instance (e.g. blocked firewall ports, 
NATs, ...) and configuration might not always be possible.

Since the client has to reach out to the PBS instance anyways to perform 
backups, it might make sense to have such a control instance living 
there, but this could be a completely different management host as well, 
e.g. a PDM instance. I would however strictly keep such a scheduling and 
notification/monitoring separate from PBS, for improved flexibility.

CC'ing @Thomas and @Fabian for opinions.

> 
> Option A: manager integrated into PBS / PDM
> -------------------------------------------
> Extend PBS (or PDM) to act as the central manager.
> 
>   - PBS/PDM stores job objects for standalone clients (target host,
>     source paths, schedule, retention, namespace, notifications).
>   - PBS/PDM triggers jobs on remote agents and ingests results.
>   - Task results and logs surface via the existing PBS/PDM API and UI.
> 
> Option B: separate standalone manager
> -------------------------------------
> Keep the PBS/PDM codebase unchanged and build a separate manager.
> 
>   - Uses the identical client agent from Option A on target hosts.
>   - Handles scheduling, central log aggregation (troubleshooting), and
>     compliance (were the last tasks successful / did every host meet its
>     SLA window).
>   - Interacts with PBS strictly via existing public APIs.
> 
> The architectural decision is strictly where the manager lives (PBS/PDM
> integration vs. a separate product); the client agent is identical in
> both.

IMHO it makes most sense to have this independent from PBS (not sure 
about PDM?, CC'ing @Lukas) in the sense that the scheduler might live as 
independent service on a PBS instance as well as being hosted on a 
totally different host.

> 
> Questions for the list
> ----------------------
>   1. Is central scheduling for standalone clients desirable inside
>      PBS/PDM (Option A), or is a separate manager preferred (Option B)?

 From my perspective it makes mostly sense to be independent from PBS, 
not sure if it would make sense to integrate this into PDM or completely 
standalone. Opinions from other devs welcome.

>   2. Does an optional lightweight client agent align with Proxmox's
>      architectural vision for standalone host backups?

As stated above i think this could live within the client itself, the 
client fetching and persisting schedules as provided by the centralized 
scheduler instance.

>   3. Are there existing design initiatives or preferred patterns for
>      remote execution in the ecosystem this should align with?

This remains still open for further discussion, but IMO the client 
should act based on it's own, only asking for updated/changes schedules 
and doing status reporting to the scheduler instance.
  > Happy to prototype whichever direction the maintainers consider viable.
> 
> Thanks,
> Ciro Iriarte
> 
> 
> 

Re: [RFC] Centralized management for standalone PBS file clients

[Ciro Iriarte]

[-- Attachment #1: Type: text/plain, Size: 7646 bytes --]

On Mon, Aug 10, 2026, 04:45 Christian Ebner <c.ebner@proxmox.com> wrote:

> Hi,
>
> thanks for reaching out for discussion.
>

Happy to collaborate.


> On 8/9/26 9:41 PM, Ciro Iriarte wrote:
> > Hi all,
> >
> > This is a proposal for discussion (not a patch). Currently
> > proxmox-backup-client is a stateless, outbound-only CLI, and PBS operates
> > purely as a passive storage endpoint. For non-PVE / standalone Linux
> > hosts, backups must be scheduled locally per-host (e.g. cron or systemd
> > timers). This creates operational gaps:
> >
> >   - No central control or unified view of backup schedules across hosts.
>
> PBS itself is never aware of backup schedules, not even for PVE (jobs
> managed by each individual PVE cluster/standalone host). I think this
> should remain this way, the PBS server should be agnostic to clients.
>
> >   - No central task log visibility for non-PVE client backups.
>
> PBS itself does track each backup task in its log, but I see that this
> is not what you might intend here, just mentioning for completeness.
>
> >   - No proactive alerting when a host fails to run an expected backup
> >     (a crashed host or a dead timer produces no signal at all).
>
> In my opinion this is again something to be managed by a different
> entity, not PBS itself.
>
> > Proposed shared component: client agent
> > ---------------------------------------
> > Both options below rely on a small, optional agent service running on
> > target Linux hosts. The agent authenticates an incoming request from a
> > central manager and invokes proxmox-backup-client locally. The existing
> > CLI remains the execution engine; the agent only dispatches jobs and
> > reports status and logs back to the manager.
>
> Rather than a dedicated agent, such agent functionality could maybe be
> integrated in the client itself, providing a command to connect to a
> scheduler instance and setup a systemd unit and timer to periodically
> connect to said scheduler instance. Most of your clients might not be
> reachable from your scheduling instance (e.g. blocked firewall ports,
> NATs, ...) and configuration might not always be possible.
>

I can see 2 usecases for PBS:

- For Internet connected clients your assumption makes sense. You expose
the service and clients reach it.
- For Enterprise setups where you have a fleet to manage and full control
of network and platforms, most of the backup solutions I've seen contact
the clients on a dedicated backup VRF with very involved configuration:
jumbo frames, static routes, microsegmentation in the switches (clients can
reach the platform but clients cannot reach other clients on the same
subnet/VLAN, platform can reach the clients too)


> Since the client has to reach out to the PBS instance anyways to perform
> backups, it might make sense to have such a control instance living
> there, but this could be a completely different management host as well,
> e.g. a PDM instance. I would however strictly keep such a scheduling and
> notification/monitoring separate from PBS, for improved flexibility.
>

Not against it, and probably my preferred option for the time being. At
scale (which is what I see as a target, it shouldn't be a problem).


>
> CC'ing @Thomas and @Fabian for opinions.
>
> >
> > Option A: manager integrated into PBS / PDM
> > -------------------------------------------
> > Extend PBS (or PDM) to act as the central manager.
> >
> >   - PBS/PDM stores job objects for standalone clients (target host,
> >     source paths, schedule, retention, namespace, notifications).
> >   - PBS/PDM triggers jobs on remote agents and ingests results.
> >   - Task results and logs surface via the existing PBS/PDM API and UI.
> >
> > Option B: separate standalone manager
> > -------------------------------------
> > Keep the PBS/PDM codebase unchanged and build a separate manager.
> >
> >   - Uses the identical client agent from Option A on target hosts.
> >   - Handles scheduling, central log aggregation (troubleshooting), and
> >     compliance (were the last tasks successful / did every host meet its
> >     SLA window).
> >   - Interacts with PBS strictly via existing public APIs.
> >
> > The architectural decision is strictly where the manager lives (PBS/PDM
> > integration vs. a separate product); the client agent is identical in
> > both.
>
> IMHO it makes most sense to have this independent from PBS (not sure
> about PDM?, CC'ing @Lukas) in the sense that the scheduler might live as
> independent service on a PBS instance as well as being hosted on a
> totally different host.
>

Would love to receive feedback on this point.


>
> >
> > Questions for the list
> > ----------------------
> >   1. Is central scheduling for standalone clients desirable inside
> >      PBS/PDM (Option A), or is a separate manager preferred (Option B)?
>
>  From my perspective it makes mostly sense to be independent from PBS,
> not sure if it would make sense to integrate this into PDM or completely
> standalone. Opinions from other devs welcome.
>

Other backup platforms follow that pattern at scale. A manager as brains
for the compliance and a "data mover" or muscle, which could be done by
PBS. Open question is if PDM should be that compliance brain.


> >   2. Does an optional lightweight client agent align with Proxmox's
> >      architectural vision for standalone host backups?
>
> As stated above i think this could live within the client itself, the
> client fetching and persisting schedules as provided by the centralized
> scheduler instance.
>

Biggest difference is if patches would be accepted upstream to modify the
official client code and package or if I should deliver this as
auxiliary/complementary packages to extend the client functionality
externally.


> >   3. Are there existing design initiatives or preferred patterns for
> >      remote execution in the ecosystem this should align with?
>
> This remains still open for further discussion, but IMO the client
> should act based on it's own, only asking for updated/changes schedules
> and doing status reporting to the scheduler instance.
>

It's important to separate:

- backup policy definition
- backup policy scheduling
- backup job triggering/execution

In my opinion, the first two should not be owned by the client. The last
one depends on correct implementation (can we detect it was not triggered
when it should have been?), what you describe could be acceptable.

  > Happy to prototype whichever direction the maintainers consider viable.
> >
> > Thanks,
> > Ciro Iriarte
>

I'm the end, this is what I see and would try to solve for baremetal or VMs
with special needs like native application backups (not protected via PVE):

- current scenario might work for a user with few machines and a service
provider receiving the backups. Separation of responsability and interests.
Not the same as owning 20-500 baremetal servers to protect.

- in my experience, I've seen too many security incidents where the client
is either neglected and the backup hasn't run properly (wrong directory or
jobs never scheduled) or the attackers just disabled the jobs once
controlling the server; and admins realizing after the attack that backups
were not in place.

- NIST recommends as a good practice the client not owning its protection
(definition, scheduling or triggering).

And I'm clear that probably the initial PVE usecase is easier to govern:

- PVE is detached from the workload, different attack surface. Final client
is the VM, not PVE.
- few clusters can cover lots of clients/workloads. Achievable backup jobs
monitoring at scale.

Regards,
CI.-

>

[-- Attachment #2: Type: text/html, Size: 11232 bytes --]

Re: [RFC] Centralized management for standalone PBS file clients

[Christian Ebner]

Hi,

I've discussed this internally and we do have some in-house development 
effort going on with respect to client job scheduling (not public yet).

So it probably is best to hold of for a bit longer with your development 
efforts, and have a look at the code once it is being send for review on 
the mailing list. I however cannot give you an ETA for that.

Best regards,
Chris