From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 99FB31FF09A for ; Sun, 16 Aug 2026 17:22:41 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 6013B231E0; Sun, 16 Aug 2026 17:22:41 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; t=1786893747; cv=none; d=google.com; s=arc-20260327; b=CGxJlOZtAgyADS+nBFyDPiiqWzFdOMNVgmE22FH19/qiSOeXqVtmoz5jDv1Yoh07yU URS31Djb/zIqaSejqUUAjoUe6CTGKlkzAiyaZLqT3wVcg9+rc5Tuv5ohPjTRAaRgkE/J vuus7jKZUUIlTB1MZqXDtaTFCoEhPonx4O4OxftOehbwZlA2ADihfucsCBKqzxZAVzR2 SlhhDruDy7CtvAZ1lOIajanowzTFZDq/ftctmXJaZjYc3+3EZl0E7ec6SPJhlnNJoP32 Q6PQm80bb4bXSVEIA/Yzb9FQktZIAjAU9HSiVamLuwgEDBDmyn6sAPYl39dO6bWRqWv4 7F9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=EqvkeJmM+6X9pK2NTRwlTLLSDyOmaIHu/JZqnOawSTY=; fh=m85CJQ+lCAmjQau4PtmcU3nAsL92LanRc0gcUNysNYg=; b=kDdrolX5vDtaH3o87iOzwzuDDfezyMlZpgy98Km2CeqPS5pj8jNrZH2Iv26qFbpZAI qiCGn0MY3LjiMyr4WZllQ7VoXOT4TLopWAXyGDFgsKWgHpzOnQ44jLcQrjCwT6gkPAsJ ZSbtxG27TMM58PibsDKCQvIV8QHlxu3729lVkSwwfvyWMwEZhR1a7teXni7mgFhCc/Iu RzzNQ6kFiydfqcNVpWiEj8zCeJgtnyr8Svm0VLMA3JE0SYn395NbBqRpeTyhen/Kk/oi 14tlLcteLXO2TAjfNbov2OdKj6SZK/7wGEv+YJ6TkKrYTrdRQidTTmgKWvPQOxZBP8p5 /Elw==; darn=lists.proxmox.com ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1786893747; x=1787498547; darn=lists.proxmox.com; h=content-type:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:from:to:cc:subject:date:message-id:reply-to :content-type; bh=EqvkeJmM+6X9pK2NTRwlTLLSDyOmaIHu/JZqnOawSTY=; b=MLARGlHLTAodeHlAlxZ433QJWnh69mcdo/IcHE0tBFdiKuonCuz0u3spRxi6+B6i9H ZLD5nSIgObqWnattDq1LynPgEO/nYvcX3FtlXa8FOCRvOSaJlOaTv/GPqSPqK02CjoAx lbnq2KWomxIyCYe3RiGoC5bcTfnQEfx2mF2CRpuN/rZ8S227zUudnpdNC0r+MREPycBe tDDh5W+uT03uWXAEMiByQ13ufALfiBLyV/Rts3Y+527eBNLjHqxJdavd3BwK2lgiuTIe iyKzuyAOZf2w877hoFff0qRouQAVa2PGJ1NQSddDg0KvdD7I0d/upH+dmz/HPT44oLSs 0fBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786893747; x=1787498547; h=content-type:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to:content-type; bh=EqvkeJmM+6X9pK2NTRwlTLLSDyOmaIHu/JZqnOawSTY=; b=ScCu28xssLJVUAUphd31G7OwfPYd5CiDQaj8e//XC+lJAWR9CiMsHtJz12uYXtU7p7 Z3yAXuyLtoV2nVHHlTELWFFwoEfO+Ckt+CgI69+nLaOenU0Z367FOtxfhZdecDKohJRJ Cy3gWT253brpV9ePF8Eghvl+1Z7qbiyh2yjlD/HPQ1rmpTob1RgLaxDFuB8F7PlEaT9m TTdmZ2J+R1hQ61WgTUWtYqahkXfIO5AGfvorG6zp5Uo+ql6ekrJF/3WYv2txz88pPv+Z MlfR0XAoJk75rKDOlqzITbhj6JGKFNywYAtVK4uDbn5bRP/H4TfT/sdKaLxmWQGa58Da jkHg== X-Gm-Message-State: AOJu0YxuDh+m64sxd3oG3zqoSEJ2WZLzYbrbNtwRcGKTNxawjdQvdcUk T+SdoUGVMkJaBJqUXpx1YBzqJxIQj+c1e7NH2ae0kXYgL8Fyb8BMd8AzWnKBJeOBhGQt6qRyRmf DLtQTiq4cvRIyDjYwtp+YpcePOIpVWRc= X-Gm-Gg: AR+sD112c6AUX2dWrIT/mlJJXmTCkBquQ6JrDD/D5AMZ8nBcJVq0horvlz/i/NpwSWb jXdU6zEyFRcuYJUzDQt3u1VBzut0BoWzIzSQvFGBnTUtLmBk/Tj99cFgRgd1+epOh2uRK7ZxL20 OkGr7rnotYXAkiGYmU9uEQ9/kqfyp5hdSRQZdvdaGFDO7HYxFg8dvvLMLqC71JkyxMZxcI5pbp5 Qj14aV7SZxppgq/oUU8hNiAmZLiams5pjQUwIjDGdGjLRkncSCpCU970bbRIVQ6rcrLFERoevFl 9m+4IxgkL21N/xl9J1lNhorBt9sv+VuzIW7rvSNkuzS8iw== X-Received: by 2002:a05:6808:13d2:b0:495:e16d:5d03 with SMTP id 5614622812f47-4b241e8565emr17534505b6e.21.1786893746873; Sun, 16 Aug 2026 08:22:26 -0700 (PDT) MIME-Version: 1.0 References: <6a78d7fc.4987c784.356268.046f@mx.google.com> <27a894c0-ce4b-4bfd-855c-b0aef571ab21@proxmox.com> In-Reply-To: <27a894c0-ce4b-4bfd-855c-b0aef571ab21@proxmox.com> From: Ciro Iriarte Date: Sun, 16 Aug 2026 09:44:35 -0300 X-Gm-Features: AcwNN1WKX2B0yH9XzFnZ1XblMURENPLcAw4PFaAlK0vETDQmBjd58a8603Wnnog Message-ID: Subject: Re: [RFC] Centralized management for standalone PBS file clients To: Christian Ebner Content-Type: multipart/alternative; boundary="00000000000008aba906592b9f4b" X-SPAM-LEVEL: Spam detection results: 0 AWL -0.000 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider HTML_MESSAGE 0.001 HTML included in message RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: TST22YWQMSIZK7Z4VW6H6T7CZYDB27RX X-Message-ID-Hash: TST22YWQMSIZK7Z4VW6H6T7CZYDB27RX X-MailFrom: cyruspy@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: pbs-devel@lists.proxmox.com, Thomas Lamprecht , Lukas Wagner X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --00000000000008aba906592b9f4b Content-Type: text/plain; charset="UTF-8" On Mon, Aug 10, 2026, 04:45 Christian Ebner wrote: > Hi, > > thanks for reaching out for discussion. > Happy to collaborate. > On 8/9/26 9:41 PM, Ciro Iriarte wrote: > > Hi all, > > > > This is a proposal for discussion (not a patch). Currently > > proxmox-backup-client is a stateless, outbound-only CLI, and PBS operates > > purely as a passive storage endpoint. For non-PVE / standalone Linux > > hosts, backups must be scheduled locally per-host (e.g. cron or systemd > > timers). This creates operational gaps: > > > > - No central control or unified view of backup schedules across hosts. > > PBS itself is never aware of backup schedules, not even for PVE (jobs > managed by each individual PVE cluster/standalone host). I think this > should remain this way, the PBS server should be agnostic to clients. > > > - No central task log visibility for non-PVE client backups. > > PBS itself does track each backup task in its log, but I see that this > is not what you might intend here, just mentioning for completeness. > > > - No proactive alerting when a host fails to run an expected backup > > (a crashed host or a dead timer produces no signal at all). > > In my opinion this is again something to be managed by a different > entity, not PBS itself. > > > Proposed shared component: client agent > > --------------------------------------- > > Both options below rely on a small, optional agent service running on > > target Linux hosts. The agent authenticates an incoming request from a > > central manager and invokes proxmox-backup-client locally. The existing > > CLI remains the execution engine; the agent only dispatches jobs and > > reports status and logs back to the manager. > > Rather than a dedicated agent, such agent functionality could maybe be > integrated in the client itself, providing a command to connect to a > scheduler instance and setup a systemd unit and timer to periodically > connect to said scheduler instance. Most of your clients might not be > reachable from your scheduling instance (e.g. blocked firewall ports, > NATs, ...) and configuration might not always be possible. > I can see 2 usecases for PBS: - For Internet connected clients your assumption makes sense. You expose the service and clients reach it. - For Enterprise setups where you have a fleet to manage and full control of network and platforms, most of the backup solutions I've seen contact the clients on a dedicated backup VRF with very involved configuration: jumbo frames, static routes, microsegmentation in the switches (clients can reach the platform but clients cannot reach other clients on the same subnet/VLAN, platform can reach the clients too) > Since the client has to reach out to the PBS instance anyways to perform > backups, it might make sense to have such a control instance living > there, but this could be a completely different management host as well, > e.g. a PDM instance. I would however strictly keep such a scheduling and > notification/monitoring separate from PBS, for improved flexibility. > Not against it, and probably my preferred option for the time being. At scale (which is what I see as a target, it shouldn't be a problem). > > CC'ing @Thomas and @Fabian for opinions. > > > > > Option A: manager integrated into PBS / PDM > > ------------------------------------------- > > Extend PBS (or PDM) to act as the central manager. > > > > - PBS/PDM stores job objects for standalone clients (target host, > > source paths, schedule, retention, namespace, notifications). > > - PBS/PDM triggers jobs on remote agents and ingests results. > > - Task results and logs surface via the existing PBS/PDM API and UI. > > > > Option B: separate standalone manager > > ------------------------------------- > > Keep the PBS/PDM codebase unchanged and build a separate manager. > > > > - Uses the identical client agent from Option A on target hosts. > > - Handles scheduling, central log aggregation (troubleshooting), and > > compliance (were the last tasks successful / did every host meet its > > SLA window). > > - Interacts with PBS strictly via existing public APIs. > > > > The architectural decision is strictly where the manager lives (PBS/PDM > > integration vs. a separate product); the client agent is identical in > > both. > > IMHO it makes most sense to have this independent from PBS (not sure > about PDM?, CC'ing @Lukas) in the sense that the scheduler might live as > independent service on a PBS instance as well as being hosted on a > totally different host. > Would love to receive feedback on this point. > > > > > Questions for the list > > ---------------------- > > 1. Is central scheduling for standalone clients desirable inside > > PBS/PDM (Option A), or is a separate manager preferred (Option B)? > > From my perspective it makes mostly sense to be independent from PBS, > not sure if it would make sense to integrate this into PDM or completely > standalone. Opinions from other devs welcome. > Other backup platforms follow that pattern at scale. A manager as brains for the compliance and a "data mover" or muscle, which could be done by PBS. Open question is if PDM should be that compliance brain. > > 2. Does an optional lightweight client agent align with Proxmox's > > architectural vision for standalone host backups? > > As stated above i think this could live within the client itself, the > client fetching and persisting schedules as provided by the centralized > scheduler instance. > Biggest difference is if patches would be accepted upstream to modify the official client code and package or if I should deliver this as auxiliary/complementary packages to extend the client functionality externally. > > 3. Are there existing design initiatives or preferred patterns for > > remote execution in the ecosystem this should align with? > > This remains still open for further discussion, but IMO the client > should act based on it's own, only asking for updated/changes schedules > and doing status reporting to the scheduler instance. > It's important to separate: - backup policy definition - backup policy scheduling - backup job triggering/execution In my opinion, the first two should not be owned by the client. The last one depends on correct implementation (can we detect it was not triggered when it should have been?), what you describe could be acceptable. > Happy to prototype whichever direction the maintainers consider viable. > > > > Thanks, > > Ciro Iriarte > I'm the end, this is what I see and would try to solve for baremetal or VMs with special needs like native application backups (not protected via PVE): - current scenario might work for a user with few machines and a service provider receiving the backups. Separation of responsability and interests. Not the same as owning 20-500 baremetal servers to protect. - in my experience, I've seen too many security incidents where the client is either neglected and the backup hasn't run properly (wrong directory or jobs never scheduled) or the attackers just disabled the jobs once controlling the server; and admins realizing after the attack that backups were not in place. - NIST recommends as a good practice the client not owning its protection (definition, scheduling or triggering). And I'm clear that probably the initial PVE usecase is easier to govern: - PVE is detached from the workload, different attack surface. Final client is the VM, not PVE. - few clusters can cover lots of clients/workloads. Achievable backup jobs monitoring at scale. Regards, CI.- > --00000000000008aba906592b9f4b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Mon, Aug 10, 2026, 04:45 Christian Ebner <c.ebner@proxmox.com> wrote:
Hi,

thanks for reaching out for discussion.

Happy to collaborate.


On 8/9/26 9:41 PM, Ciro Iriarte wrote:
> Hi all,
>
> This is a proposal for discussion (not a patch). Currently
> proxmox-backup-client is a stateless, outbound-only CLI, and PBS opera= tes
> purely as a passive storage endpoint. For non-PVE / standalone Linux > hosts, backups must be scheduled locally per-host (e.g. cron or system= d
> timers). This creates operational gaps:
>
>=C2=A0 =C2=A0- No central control or unified view of backup schedules a= cross hosts.

PBS itself is never aware of backup schedules, not even for PVE (jobs
managed by each individual PVE cluster/standalone host). I think this
should remain this way, the PBS server should be agnostic to clients.

>=C2=A0 =C2=A0- No central task log visibility for non-PVE client backup= s.

PBS itself does track each backup task in its log, but I see that this
is not what you might intend here, just mentioning for completeness.

>=C2=A0 =C2=A0- No proactive alerting when a host fails to run an expect= ed backup
>=C2=A0 =C2=A0 =C2=A0(a crashed host or a dead timer produces no signal = at all).

In my opinion this is again something to be managed by a different
entity, not PBS itself.

> Proposed shared component: client agent
> ---------------------------------------
> Both options below rely on a small, optional agent service running on<= br> > target Linux hosts. The agent authenticates an incoming request from a=
> central manager and invokes proxmox-backup-client locally. The existin= g
> CLI remains the execution engine; the agent only dispatches jobs and > reports status and logs back to the manager.

Rather than a dedicated agent, such agent functionality could maybe be
integrated in the client itself, providing a command to connect to a
scheduler instance and setup a systemd unit and timer to periodically
connect to said scheduler instance. Most of your clients might not be
reachable from your scheduling instance (e.g. blocked firewall ports,
NATs, ...) and configuration might not always be possible.
=

I can see 2 useca= ses for PBS:

- For Inter= net connected clients your assumption makes sense. You expose the service a= nd clients reach it.
- For Enterprise setups where y= ou have a fleet to manage and full control of network and platforms, most o= f the backup solutions I've seen contact the clients on a dedicated bac= kup VRF with very involved configuration: jumbo frames, static routes, micr= osegmentation in the switches (clients can reach the platform but clients c= annot reach other clients on the same subnet/VLAN, platform can reach the c= lients too)


Since the client has to reach out to the PBS instance anyways to perform backups, it might make sense to have such a control instance living
there, but this could be a completely different management host as well, e.g. a PDM instance. I would however strictly keep such a scheduling and notification/monitoring separate from PBS, for improved flexibility.

Not aga= inst it, and probably my preferred option for the time being. At scale (whi= ch is what I see as a target, it shouldn't be a problem).=C2=A0



CC'ing @Thomas and @Fabian for opinions.

>
> Option A: manager integrated into PBS / PDM
> -------------------------------------------
> Extend PBS (or PDM) to act as the central manager.
>
>=C2=A0 =C2=A0- PBS/PDM stores job objects for standalone clients (targe= t host,
>=C2=A0 =C2=A0 =C2=A0source paths, schedule, retention, namespace, notif= ications).
>=C2=A0 =C2=A0- PBS/PDM triggers jobs on remote agents and ingests resul= ts.
>=C2=A0 =C2=A0- Task results and logs surface via the existing PBS/PDM A= PI and UI.
>
> Option B: separate standalone manager
> -------------------------------------
> Keep the PBS/PDM codebase unchanged and build a separate manager.
>
>=C2=A0 =C2=A0- Uses the identical client agent from Option A on target = hosts.
>=C2=A0 =C2=A0- Handles scheduling, central log aggregation (troubleshoo= ting), and
>=C2=A0 =C2=A0 =C2=A0compliance (were the last tasks successful / did ev= ery host meet its
>=C2=A0 =C2=A0 =C2=A0SLA window).
>=C2=A0 =C2=A0- Interacts with PBS strictly via existing public APIs. >
> The architectural decision is strictly where the manager lives (PBS/PD= M
> integration vs. a separate product); the client agent is identical in<= br> > both.

IMHO it makes most sense to have this independent from PBS (not sure
about PDM?, CC'ing @Lukas) in the sense that the scheduler might live a= s
independent service on a PBS instance as well as being hosted on a
totally different host.

<= /div>
Would love to receive feedback on this point.
<= div dir=3D"auto">


>
> Questions for the list
> ----------------------
>=C2=A0 =C2=A01. Is central scheduling for standalone clients desirable = inside
>=C2=A0 =C2=A0 =C2=A0 PBS/PDM (Option A), or is a separate manager prefe= rred (Option B)?

=C2=A0From my perspective it makes mostly sense to be independent from PBS,=
not sure if it would make sense to integrate this into PDM or completely standalone. Opinions from other devs welcome.
<= div dir=3D"auto">
Other backup platforms follow = that pattern at scale. A manager as brains for the compliance and a "d= ata mover" or muscle, which could be done by PBS. Open question is if = PDM should be that compliance brain.


>=C2=A0 =C2=A02. Does an optional lightweight client agent align with Pr= oxmox's
>=C2=A0 =C2=A0 =C2=A0 architectural vision for standalone host backups?<= br>
As stated above i think this could live within the client itself, the
client fetching and persisting schedules as provided by the centralized scheduler instance.

Biggest difference is if patches would be accepted upstr= eam to modify the official client code and package or if I should deliver t= his as auxiliary/complementary packages to extend the client functionality = externally.


>=C2=A0 =C2=A03. Are there existing design initiatives or preferred patt= erns for
>=C2=A0 =C2=A0 =C2=A0 remote execution in the ecosystem this should alig= n with?

This remains still open for further discussion, but IMO the client
should act based on it's own, only asking for updated/changes schedules=
and doing status reporting to the scheduler instance.

It's important to = separate:

- backup polic= y definition=C2=A0
- backup policy scheduling=C2=A0<= /div>
- backup job triggering/execution=C2=A0

In my opinion, the first two should = not be owned by the client. The last one depends on correct implementation = (can we detect it was not triggered when it should have been?), what you de= scribe could be acceptable.

=C2=A0 > Happy to prototype whichever direction the maintainers consider= viable.
>
> Thanks,
> Ciro Iriarte

<= div dir=3D"auto">
I'm the end, this is what I see and = would try to solve for baremetal or VMs with special needs like native appl= ication backups (not protected via PVE):

<= div dir=3D"auto">- current scenario might work for a user with few machines= and a service provider receiving the backups. Separation of responsability= and interests. Not the same as owning 20-500 baremetal servers to protect.=

- in my experience, I&#= 39;ve seen too many security incidents where the client is either neglected= and the backup hasn't run properly (wrong directory or jobs never sche= duled) or the attackers just disabled the jobs once controlling the server;= and admins realizing after the attack that backups were not in place.

- NIST recommends as a good = practice the client not owning its protection (definition, scheduling or tr= iggering).

And I'm c= lear that probably the initial PVE usecase is easier to govern:

- PVE is detached from the workload= , different attack surface. Final client is the VM, not PVE.
- few clusters can cover lots of clients/workloads. Achievable ba= ckup jobs monitoring at scale.

Regards,
CI.-
--00000000000008aba906592b9f4b--