From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 5B4281FF0C0 for ; Wed, 21 Oct 2026 22:56:00 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id C718F21661; Wed, 21 Oct 2026 22:55:54 +0200 (CEST) From: Thomas Lamprecht To: pbs-devel@lists.proxmox.com Subject: [PATCH 7/7] docs: client: document scheduled backup jobs Date: Wed, 14 Oct 2026 12:51:36 +0200 Message-ID: <20261014105208.2747208-8-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261014105208.2747208-1-t.lamprecht@proxmox.com> References: <20261014105208.2747208-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1792616142400 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.939 Adjusted score from AWL reputation of From: address DATE_IN_PAST_96_XX 2.07 Date: is 96 hours or more before Received: date DMARC_MISSING 0.1 Missing DMARC policy KAM_ASCII_DIVIDERS 0.8 Email that uses ascii formatting dividers and possible spam tricks KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: CE4Y7DKQIMMU44Q3COID6ZCMSVVZ36OQ X-Message-ID-Hash: CE4Y7DKQIMMU44Q3COID6ZCMSVVZ36OQ X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Signed-off-by: Thomas Lamprecht --- docs/backup-client.rst | 274 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 274 insertions(+) diff --git a/docs/backup-client.rst b/docs/backup-client.rst index 39b1f9be0..c224cc19c 100644 --- a/docs/backup-client.rst +++ b/docs/backup-client.rst @@ -210,6 +210,280 @@ Additionally, system credentials (e.g. passed down from the hypervisor to a virtual machine via SMBIOS type 11) can be loaded on a service via `LoadCredential=` as described in the manual page ``systemd.exec(5)``. +.. _client_scheduled_backups: + +Scheduled Backups +----------------- + +The backup client has a built-in scheduler that runs backup jobs at regular +intervals, without :manpage:`cron(8)` or a systemd timer for each job. A job is +a named configuration holding everything a backup run needs: the target +repository, the backup sources, the encryption key, and optionally a script +that decides whether a run should proceed. + +Jobs are configured per user and stored in that user's XDG directories. Manage +the jobs as the user that should run them, for example as ``root`` for a backup +that has to read every file on the system. + +``$XDG_CONFIG_HOME/proxmox-backup/jobs.cfg`` + The job definitions, in section configuration format. The format has no + comment lines and cannot store values with leading or trailing spaces, so the + ``job`` subcommands reject such values. If you edit the file by hand, do not + change jobs with the ``job`` subcommands at the same time, as they rewrite + the whole file. + + If the file does not parse, the ``job`` subcommands report the error and + the scheduler refuses to start, until you fix the file. + +``$XDG_STATE_HOME/proxmox-backup/jobstates/backup-job-.json`` + The outcome of the job's most recent run, see `Job State and Concurrency`_. + +``$XDG_STATE_HOME/proxmox-backup/scheduler.pid`` + The process ID of the running scheduler. + +Without these environment variables, the paths default to +``~/.config/proxmox-backup/`` and ``~/.local/state/proxmox-backup/``. The +configuration and state files are created with mode ``0600``, as they record +where the secrets of a job come from. + +Managing Jobs +~~~~~~~~~~~~~ + +The ``proxmox-backup-client job`` subcommands create, list, show, update, +remove, and run jobs. For example, the following command creates a job: + +.. code-block:: console + + # proxmox-backup-client job create nightly \ + --schedule '02:30' \ + --server pbs.example.com --datastore store1 --auth-id 'backup@pbs!nightly' \ + --backupspec root.pxar:/ --backupspec home.pxar:/home \ + --exclude /var/cache --exclude /var/tmp \ + --keyfile /root/.config/proxmox-backup/key.json \ + --password-file /run/secrets/pbs-token \ + --pre-run-hook /usr/local/bin/pbs-check-battery.sh \ + --comment 'nightly laptop backup' + +As with ``proxmox-backup-client backup``, you can name the target repository +with the separate ``--server``, ``--port``, ``--datastore``, and ``--auth-id`` +options, as above, or as a single URL with ``--repository``. + +All paths in a job must be absolute: the backup sources, the key file, the +files holding secrets, the ``--include-dev`` mount points, and the pre-run +hook. The scheduler does not share the working directory and ``$PATH`` of your +shell, so a relative path could resolve to a different file. The patterns of +``--exclude`` are the exception, as they are matched relative to the backup +source. + +The other subcommands inspect and change existing jobs: + +.. code-block:: console + + # proxmox-backup-client job list + # proxmox-backup-client job show nightly + # proxmox-backup-client job update nightly --schedule 'mon..fri 02:30' + # proxmox-backup-client job update nightly --delete comment --delete pre-run-hook + # proxmox-backup-client job disable nightly + # proxmox-backup-client job enable nightly + # proxmox-backup-client job run nightly + # proxmox-backup-client job run nightly --dry-run + # proxmox-backup-client job remove nightly + +A dry run shows what the job would back up without uploading anything. It +still runs the pre-run hook and connects to the server, but it records no run, +so the last result and the next run of the job stay as they were. + +``update`` accepts the same options as ``create`` and changes only the options +you give. ``--delete `` clears an optional setting. Setting +``--repository`` on a job that uses the separate options clears them, and the +other way around, so you can switch between both forms in one call. + +A disabled job keeps its configuration, but the scheduler does not start it. +You can still run it with ``job run``. + +The ``--rate`` and ``--burst`` options limit the bandwidth of a job, and take +the same values as for ``proxmox-backup-client backup``: + +.. code-block:: console + + # proxmox-backup-client job update nightly --rate 10MB --burst 20MB + +Unattended Authentication +~~~~~~~~~~~~~~~~~~~~~~~~~ + +A scheduled run cannot ask for a password, so a job names the source of each +secret it needs with the following options. A secret configured in the job +takes precedence over the corresponding ``PBS_*`` environment variables. + +``--password-file `` / ``--password-cmd `` + The source of the repository password or API token secret: the first line of + the file, or the first line that the command prints. Use only one of them. + The command runs without a shell, so give it as an absolute path, and use + ``/bin/sh -c '...'`` for shell syntax. If the command fails, the run fails + with its error. An empty password is not caught early, and shows up as an + authentication failure. For example, read a file that systemd decrypted with + ``--password-file /run/secrets/pbs-token``, or ask a password manager with + ``--password-cmd '/usr/bin/pass show pbs/nightly'``. + + The client waits for the command without a time limit, and a hanging command + blocks one of the two backups that the scheduler runs at a time. If the + command can hang, give it a timeout of its own, for example + ``--password-cmd '/usr/bin/timeout 30 /usr/bin/pass show pbs/nightly'``. + +``--encryption-password-file `` / ``--encryption-password-cmd `` + The same for the passphrase of the key file, which the client otherwise reads + from :envvar:`PBS_ENCRYPTION_PASSWORD`. A job needs one of them if its key + file is protected by a passphrase. + +``--fingerprint `` + Pins the fingerprint of the server certificate, in the colon-separated + hexadecimal form that the web interface shows. Use it if the system does not + trust the certificate of the server, for example a self-signed one. A + scheduled run cannot ask you to confirm an unknown fingerprint, so without + this option it fails, unless you already accepted the fingerprint in an + interactive run as the same user, for example with ``job run``. + +Schedules +~~~~~~~~~ + +The ``--schedule`` option accepts the same calendar events as the jobs of the +server, see :ref:`calendar-event-scheduling` for the syntax. Some common +schedules: + +======================= ============================================== +Schedule Meaning +======================= ============================================== +``hourly`` At the top of every hour. +``daily`` At 00:00 every day. +``02:30`` Every day at 02:30. +``mon..fri 02:30`` Weekdays at 02:30. +``*/15`` Every 15 minutes. +``sat *-*-1..7 03:00`` First Saturday of each month at 03:00. +======================= ============================================== + +The next run is computed from the end of the previous one. A run missed while +the machine was switched off therefore starts shortly after the scheduler comes +back up, rather than at the next occurrence. The scheduler checks the schedules +at least once a minute, so a run may start up to a minute late. + +A run that fails or is interrupted, for example because the machine was +switched off, is not retried. The job runs again at its next scheduled time, +or right away with ``job run``. + +``create`` and ``update`` reject a schedule without future events, such as a +date in the past. + +The Scheduler Daemon +~~~~~~~~~~~~~~~~~~~~ + +Jobs only run while a scheduler is running, and creating a job does not start +one. The recommended way to run it is the systemd user service that the client +package ships. The service is not enabled by default, as a backup daemon should +only run for users that configured a job: + +.. code-block:: console + + # systemctl --user enable --now proxmox-backup-client-scheduler + +The scheduler rereads ``jobs.cfg`` at least once a minute. To apply a change at +once, run ``systemctl --user reload proxmox-backup-client-scheduler``. If no +scheduler is running when you create or enable a job, the subcommand prints how +to start one. + +If ``jobs.cfg`` stops parsing while the scheduler runs, the scheduler keeps +running but starts no new runs until you fix the file. Runs already in progress +finish. + +A run started by the scheduler first checks that the job is still due, and does +nothing if the job was disabled, removed, or has already run in the meantime. A +manual ``job run`` always runs the job, regardless of its schedule. + +On ``SIGTERM``, the scheduler starts no new runs and waits for the running ones +to finish. A second ``SIGTERM`` makes it exit without waiting. ``systemctl +--user stop`` therefore lets a running backup finish, up to the +``TimeoutStopSec`` of the service. + +Each run is a separate ``proxmox-backup-client job run `` process with its +own environment, so that the password of one job cannot leak into another. The +runs log to the journal of the user service. At most two backups run at the +same time. Jobs that have to wait start as soon as another run finishes, +longest waiting first. + +Only one scheduler can run per user. A second one refuses to start. + +.. Note:: A systemd user service only runs while its user is logged in, unless + `lingering `_ + is enabled for that user with ``loginctl enable-linger ``. + +Job State and Concurrency +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each run records its outcome in the state file of the job. ``job list`` and +``job show`` display it in the ``last-run`` and ``last-result`` columns, and the +time the job is due next in the ``next-run`` column. In ``job list``, +``next-run`` reads ``disabled`` for a disabled job and ``never`` for a schedule +without future events. ``last-result`` is one of the following: + +``created`` + The job has not run yet. + +``running`` + A run is in progress. + +``ok`` + The last run succeeded. + +``failed`` + The last run failed. ``job show`` prints the error. + +``skipped`` + The pre-run hook skipped the last run, so no backup was made. + +``interrupted`` + The last run never finished, for example because the machine was switched + off or the process was killed. + +Without a readable state file, ``last-run`` and ``last-result`` stay empty, and +the job is due right away. + +A job never runs twice at the same time. Starting ``job run`` for a job that is +already running fails after a few seconds, and ``job remove`` refuses to remove +a running job. + +Pre-Run Hooks +~~~~~~~~~~~~~ + +The ``--pre-run-hook `` option names an executable, by its absolute path, +that decides by its exit status whether a run proceeds: + +- ``0``: The backup proceeds. +- ``1``: This run is skipped and recorded as ``skipped``. The schedule continues + as usual. +- Any other status, or a hook killed by a signal, fails the run, so that a + broken hook is not mistaken for a deliberate skip. + +The hook gets the job ID in the ``PBS_JOB_ID`` environment variable, runs in +``/`` without standard input, and does not see the ``PBS_PASSWORD*``, +``PBS_ENCRYPTION_PASSWORD*``, and ``CREDENTIALS_DIRECTORY`` variables, as it +does not need the secrets of the backup. Its output goes to the log of the run. + +Hooks typically skip a run under conditions that a schedule cannot express, for +example: + +- while the machine runs on battery, or the battery is low +- while the backup server is not reachable +- during working hours + +A hook may take up to one minute. After that, the client stops it along with +any processes it started, and the run fails. + +The client packages ship an example hook that skips a backup when the battery +is low. It is ``examples/pbs-check-battery.sh`` in the documentation directory +of the installed package, ``/usr/share/doc/proxmox-backup-client/`` or +``/usr/share/doc/proxmox-backup-client-static/``. Copy it to a stable path such +as ``/usr/local/bin``, adjust the thresholds, and point ``--pre-run-hook`` at +the copy. + Output Format ------------- -- 2.47.3