[RFC] Centralized management for standalone PBS file clients

[RFC] Centralized management for standalone PBS file clients

[Ciro Iriarte]

Hi all,

This is a proposal for discussion (not a patch). Currently
proxmox-backup-client is a stateless, outbound-only CLI, and PBS operates
purely as a passive storage endpoint. For non-PVE / standalone Linux
hosts, backups must be scheduled locally per-host (e.g. cron or systemd
timers). This creates operational gaps:

 - No central control or unified view of backup schedules across hosts.
 - No central task log visibility for non-PVE client backups.
 - No proactive alerting when a host fails to run an expected backup
   (a crashed host or a dead timer produces no signal at all).

Proposed shared component: client agent
---------------------------------------
Both options below rely on a small, optional agent service running on
target Linux hosts. The agent authenticates an incoming request from a
central manager and invokes proxmox-backup-client locally. The existing
CLI remains the execution engine; the agent only dispatches jobs and
reports status and logs back to the manager.

Option A: manager integrated into PBS / PDM
-------------------------------------------
Extend PBS (or PDM) to act as the central manager.

 - PBS/PDM stores job objects for standalone clients (target host,
   source paths, schedule, retention, namespace, notifications).
 - PBS/PDM triggers jobs on remote agents and ingests results.
 - Task results and logs surface via the existing PBS/PDM API and UI.

Option B: separate standalone manager
-------------------------------------
Keep the PBS/PDM codebase unchanged and build a separate manager.

 - Uses the identical client agent from Option A on target hosts.
 - Handles scheduling, central log aggregation (troubleshooting), and
   compliance (were the last tasks successful / did every host meet its
   SLA window).
 - Interacts with PBS strictly via existing public APIs.

The architectural decision is strictly where the manager lives (PBS/PDM
integration vs. a separate product); the client agent is identical in
both.

Questions for the list
----------------------
 1. Is central scheduling for standalone clients desirable inside
    PBS/PDM (Option A), or is a separate manager preferred (Option B)?
 2. Does an optional lightweight client agent align with Proxmox's
    architectural vision for standalone host backups?
 3. Are there existing design initiatives or preferred patterns for
    remote execution in the ecosystem this should align with?

Happy to prototype whichever direction the maintainers consider viable.

Thanks,
Ciro Iriarte

Re: [RFC] Centralized management for standalone PBS file clients

[Christian Ebner]

Hi,

thanks for reaching out for discussion.

On 8/9/26 9:41 PM, Ciro Iriarte wrote:
> Hi all,
> 
> This is a proposal for discussion (not a patch). Currently
> proxmox-backup-client is a stateless, outbound-only CLI, and PBS operates
> purely as a passive storage endpoint. For non-PVE / standalone Linux
> hosts, backups must be scheduled locally per-host (e.g. cron or systemd
> timers). This creates operational gaps:
> 
>   - No central control or unified view of backup schedules across hosts.

PBS itself is never aware of backup schedules, not even for PVE (jobs 
managed by each individual PVE cluster/standalone host). I think this 
should remain this way, the PBS server should be agnostic to clients.

>   - No central task log visibility for non-PVE client backups.

PBS itself does track each backup task in its log, but I see that this 
is not what you might intend here, just mentioning for completeness.

>   - No proactive alerting when a host fails to run an expected backup
>     (a crashed host or a dead timer produces no signal at all).

In my opinion this is again something to be managed by a different 
entity, not PBS itself.

> Proposed shared component: client agent
> ---------------------------------------
> Both options below rely on a small, optional agent service running on
> target Linux hosts. The agent authenticates an incoming request from a
> central manager and invokes proxmox-backup-client locally. The existing
> CLI remains the execution engine; the agent only dispatches jobs and
> reports status and logs back to the manager.

Rather than a dedicated agent, such agent functionality could maybe be 
integrated in the client itself, providing a command to connect to a 
scheduler instance and setup a systemd unit and timer to periodically 
connect to said scheduler instance. Most of your clients might not be 
reachable from your scheduling instance (e.g. blocked firewall ports, 
NATs, ...) and configuration might not always be possible.

Since the client has to reach out to the PBS instance anyways to perform 
backups, it might make sense to have such a control instance living 
there, but this could be a completely different management host as well, 
e.g. a PDM instance. I would however strictly keep such a scheduling and 
notification/monitoring separate from PBS, for improved flexibility.

CC'ing @Thomas and @Fabian for opinions.

> 
> Option A: manager integrated into PBS / PDM
> -------------------------------------------
> Extend PBS (or PDM) to act as the central manager.
> 
>   - PBS/PDM stores job objects for standalone clients (target host,
>     source paths, schedule, retention, namespace, notifications).
>   - PBS/PDM triggers jobs on remote agents and ingests results.
>   - Task results and logs surface via the existing PBS/PDM API and UI.
> 
> Option B: separate standalone manager
> -------------------------------------
> Keep the PBS/PDM codebase unchanged and build a separate manager.
> 
>   - Uses the identical client agent from Option A on target hosts.
>   - Handles scheduling, central log aggregation (troubleshooting), and
>     compliance (were the last tasks successful / did every host meet its
>     SLA window).
>   - Interacts with PBS strictly via existing public APIs.
> 
> The architectural decision is strictly where the manager lives (PBS/PDM
> integration vs. a separate product); the client agent is identical in
> both.

IMHO it makes most sense to have this independent from PBS (not sure 
about PDM?, CC'ing @Lukas) in the sense that the scheduler might live as 
independent service on a PBS instance as well as being hosted on a 
totally different host.

> 
> Questions for the list
> ----------------------
>   1. Is central scheduling for standalone clients desirable inside
>      PBS/PDM (Option A), or is a separate manager preferred (Option B)?

 From my perspective it makes mostly sense to be independent from PBS, 
not sure if it would make sense to integrate this into PDM or completely 
standalone. Opinions from other devs welcome.

>   2. Does an optional lightweight client agent align with Proxmox's
>      architectural vision for standalone host backups?

As stated above i think this could live within the client itself, the 
client fetching and persisting schedules as provided by the centralized 
scheduler instance.

>   3. Are there existing design initiatives or preferred patterns for
>      remote execution in the ecosystem this should align with?

This remains still open for further discussion, but IMO the client 
should act based on it's own, only asking for updated/changes schedules 
and doing status reporting to the scheduler instance.
  > Happy to prototype whichever direction the maintainers consider viable.
> 
> Thanks,
> Ciro Iriarte
> 
> 
>