From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 19C0E1FF0C0 for ; Wed, 18 Nov 2026 14:08:00 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id DE8342168A; Wed, 18 Nov 2026 14:07:59 +0100 (CET) Message-ID: <9115f14e-13aa-4ef9-9e5d-daadc9eee226@proxmox.com> Date: Wed, 18 Nov 2026 14:07:52 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH many 00/12] snapshot attributes for tags and pools of PVE guests, tape job filter To: Christian Ebner , Jakob-Tobias Winter , pbs-devel@lists.proxmox.com References: <20261113085637.57360-3-tobias@stuttgart.de> Content-Language: en-US From: Samuel Rufinatscha In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.054 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 4DDAWHEFPV4I76F3UFGOF6DMIHPVMAO2 X-Message-ID-Hash: 4DDAWHEFPV4I76F3UFGOF6DMIHPVMAO2 X-MailFrom: s.rufinatscha@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Agree with @Christian's points here. Please see two more comments inline. On 11/16/26 3:57 PM, Christian Ebner wrote: > Thanks for the patches, I took a first look and think the overall > approach does look fine, left a few comments on the individual patches. > > CC'ing Samuel since he started looking into snapshot attributes recently > and might have other/additional comments and suggestions. > > On 11/13/26 10:07 AM, Jakob-Tobias Winter wrote: >> Tape backup jobs on a Proxmox Backup Server select backup groups by >> type and ID only. Splitting guests over media pools by what they are, >> say production and test guests with different retention, means keeping >> lists of IDs up to date by hand in every job. In Proxmox VE the same >> guests are already organized with tags and resource pools, but the >> backup server knows about neither. >> >> This series lets Proxmox VE store the tags and the resource pool of a >> guest with each backup, as attributes of the snapshot, and adds a >> filter on such attributes to tape backup jobs: >> >>    # proxmox-tape backup-job create prod --store store1 --drive lto1 \ >>        --pool prod --attribute-filter pve-tag=prod >>    # proxmox-tape backup-job create rest --store store1 --drive lto1 \ >>        --pool rest --attribute-filter exclude:pve-tag=prod >> >> It is an RFC, as it defines a format and names that others would have >> to live with, and because [1] is assigned already. If there is work in >> progress for it, I am happy to adapt this series to it or to drop >> whatever overlaps. >> >> Proxmox Backup Server >> --------------------- >> >> proxmox 1-2, proxmox-backup 3-7 >> >> Snapshots get attributes, a map from a key to a list of values. They >> are kept in the unprotected part of the manifest, as suggested in [0], >> so they can be set for encrypted backups without touching the >> signature, and they stay with a snapshot that is synced or written to >> tape. A guest with several tags gets one value per tag, nothing is >> joined into one string. >> >> They are read and set through a new endpoint next to the notes, >> GET/PUT /admin/datastore/{store}/attributes, with the privileges of >> the notes, as a list of KEY=VALUE entries. A PUT replaces all >> attributes of the snapshot. An empty set is stored as well, to tell a >> guest without tags from a client which never set anything. >> >> Tape backup jobs get the option 'attribute-filter', a list of >> [exclude:|include:]KEY=VALUE with the include and exclude rules of >> group filters, applied after the group filter. A group has the >> attributes of its newest finished snapshot that has some, so a single >> backup without attributes does not move a group to another media pool. >> Besides the API there is the option for 'proxmox-tape', a field in the >> job window, a column in the job list and documentation. >> >> The server does not interpret keys or values. >> >> Proxmox VE >> ---------- >> >> storage 8, guest-common 9, manager 10-11, docs 12 >> >> After a backup to a PBS storage, vzdump sets 'pve-tag' (one value per >> tag) and 'pve-pool', with the prefix suggested in [0], directly >> through the API like the protection flag. A guest in a nested pool >> gets that pool and each of its parent pools as values, so a filter >> for a pool also matches the guests of the pools nested in it. A >> server without the endpoint answers 404, which is only noted in the >> task log. >> >> As the attributes are not encrypted, the new vzdump option >> 'pbs-snapshot-attributes' (default on) can turn this off for a job, a >> single call, or a node in /etc/vzdump.conf. >> >> Compatibility >> ------------- >> >> * Jobs without the new option select the same groups as before, and >>    nothing changes on tape or in the catalogs. A tape job written by >>    4.2.9 was read and run unchanged. >> * The other way round, a version without the series cannot parse a >>    tape-job.cfg in which the option is used (unknown property), like >>    with other new job properties. >> * A backup server without the series leaves the attributes in the >>    manifest alone (tried with 4.2.9). >> * A PVE node without the series ignores the vzdump option in jobs.cfg >>    with a warning. >> >> Open questions >> -------------- >> >> * The names: 'pve-tag', 'pve-pool', 'attribute-filter' and >>    'pbs-snapshot-attributes'. > > One potential issue I see arise is the additional potential for naming > conflicts. While I would expect the backup groups to be organized within > different namespaces, that might not always be the case. So maybe it > makes sense to add the cluster name as additional tag as well? > > However, each additional tag leaks more information to the PBS, which > might not be what some users expect. So a bit torn on this one.... > More of a workaround but for setups where clusters share a namespace, maybe requiring an additional user defined PVE tag would be fine for now? An user could additionally tag the relevant guests with the cluster name (or any other prefered alias) and filter based on that. > Maybe it would make sense to allow users to control which information is > being set as attribute? E.g. tags only? > >> * A guest in the pool 'a/b' gets the 'pve-pool' values 'a/b' and 'a', >>    so one filter covers a pool with everything nested in it. A backup >>    job for the pool 'a' in Proxmox VE only takes the direct members of >>    'a'. Is that difference fine, or should only 'a/b' be stored? > > IMHO this should reflect the exact pool. While it may be convenient to > only have to specify e.g. a filter on `a` on the PBS side, I think this > would not reflect the actual state? Filters on the PBS side can any-ways > be additive? Further, what if one wants snapshots from pool `a/b` but > not `a`? > >>    Since nested pools are quite new I wasn't sure about that at all. >> * Setting the attributes after the backup, like notes and protection, >>    needs no change in the backup protocol or in the QEMU library. But >>    a user that may only append, as in [2], could then not set them. >>    Passing them when the backup is created would solve that, at the >>    price of changes in proxmox-backup-qemu, pve-qemu, qemu-server and >>    pve-container which I found a bit excessive for some attributes. > > Although more work and dependency churn, adding this for exactly the > append only backup you mentioned makes this the way to go IMHO. > >> * On by default with the option to opt out, or the other way round? >>    Is the switch needed at all? > > I would prefer this to be opt-in, especially since it leaks tags and > pool information to the PBS instance even if snapshots are encrypted, so > any user should explicitly opt-in after being made aware of that. > >> * This is a separate job option and not a new kind of group filter, >>    as group filters are shared with sync jobs and match on the name of >>    a group only. Using attributes in other jobs and showing them in the >>    web interface could follow, if wanted. > > Yes, extending this to other jobs and especially show this kind of > information to users with the required permissions in the PBS UI would > be the natural followups. I think showing the stored attributes under Datastore > Content when a snapshot is inspected would be a good start, considering the permissions hold for that? this may help users check which keys and values to use when configuring a job. > >> * For a group in which no snapshot has attributes, all its manifests >>    are read when a job with a filter runs. Is that acceptable, or >>    should only the newest few be looked at? > > As noted on the individual patch for this, I'm not so sure about the > group attributes as defined at all. Might make sense to do this on a > per-snapshot level like e.g. for sync jobs with the encrypted-only/ > verified-only flags. > >> >> Testing >> ------- >> >> proxmox-backup master (4.2.11) with the series: rustfmt, clippy >> without new warnings, every commit builds, 'cargo test --workspace' >> passes (162 tests, one of them new), the docs build as HTML and man >> pages. The tests of pbs-api-types pass, with five new ones for the >> filter. pbs-api-types 1.0.22 came from the proxmox repository, as it >> is not in the devel repository yet. >> >> The PVE patches are perltidy and biome clean, and the test suites of >> the three Perl repositories give the same results with and without >> them. >> >> End to end in an unprivileged sandbox with the real daemons of both >> products: the packages of PVE 9.2.26 with the PVE patches applied and >> a backup server built from master with the series. There is no KVM in >> it, so the guests were stopped VMs and a container. >> >> * vzdump of VMs and a container to a plain and an encrypted PBS >>    storage, the latter with a namespace, as a user with DatastoreBackup >>    only; several tags, pools nested up to three levels, changed and >>    removed tags; the option per call, per job (also run by pvescheduler >>    and from the web interface) and in /etc/vzdump.conf; restore of an >>    encrypted backup >> * the same against 4.2.9 and plain master: noted in the task log, no >>    warning, the backup is fine >> * the API as owner, as another user, as auditor and as admin; invalid >>    values; verification, notes, protection and a local sync job keep >>    the attributes >> * tape jobs on the virtual tape drive (needs [3]) selecting by tag, by >>    pool and by the parent of a nested pool, combined with a group >>    filter and with namespaces, started by hand and by the scheduler; a >>    restore from tape keeps the attributes >> * both web interfaces in a browser >> >> Not yet tested: real tape hardware, clusters with more than one node, >> datastores with an S3 backend. >> >> I wasn't sure if I was supposed to send to pbs-devel and pve-devel. >> >> [0] https://bugzilla.proxmox.com/show_bug.cgi?id=4186#c32 >> [1] https://bugzilla.proxmox.com/show_bug.cgi?id=7452 >> [2] https://lore.proxmox.com/pbs-devel/20261106172235.890542-1- >> c.ebner@proxmox.com/ >> [3] https://lore.proxmox.com/pbs-devel/20261111204853.531985-2- >> tobias@stuttgart.de/ >> >> >> proxmox: >> >> Jakob-Tobias Winter (2): >>    pbs-api-types: add types and schemas for snapshot attributes >>    pbs-api-types: tape: add attribute filter to backup jobs >> >>   pbs-api-types/src/datastore.rs                |  22 ++- >>   pbs-api-types/src/jobs.rs                     |  98 ++++++++++- >>   pbs-api-types/src/lib.rs                      |   3 + >>   pbs-api-types/tests/attribute_filter_tests.rs | 155 ++++++++++++++++++ >>   4 files changed, 276 insertions(+), 2 deletions(-) >>   create mode 100644 pbs-api-types/tests/attribute_filter_tests.rs >> >> >> proxmox-backup: >> >> Jakob-Tobias Winter (5): >>    datastore: add attributes to backup snapshots >>    fix #7452: api: datastore: add endpoints for snapshot attributes >>    tape: backup job: allow selecting groups by snapshot attributes >>    ui: tape backup job: add attribute filter >>    docs: describe snapshot attributes and the tape job attribute filter >> >>   docs/pve-integration.rst           |  23 ++++++ >>   docs/tape-backup.rst               |  27 ++++++ >>   docs/terminology.rst               |  33 ++++++++ >>   pbs-datastore/src/backup_info.rs   |  18 +++- >>   pbs-datastore/src/datastore.rs     |  15 +++- >>   pbs-datastore/src/manifest.rs      |  87 +++++++++++++++++++- >>   src/api2/admin/datastore.rs        | 128 ++++++++++++++++++++++++++++- >>   src/api2/config/tape_backup_job.rs |   8 ++ >>   src/api2/tape/backup.rs            |  19 ++++- >>   src/bin/proxmox-tape.rs            |  11 ++- >>   www/tape/BackupJobs.js             |   7 ++ >>   www/tape/window/TapeBackupJob.js   |  27 +++++- >>   12 files changed, 391 insertions(+), 12 deletions(-) >> >> >> storage: >> >> Jakob-Tobias Winter (1): >>    pbs: add helper to set the attributes of a backup snapshot >> >>   src/PVE/Storage/PBSPlugin.pm | 23 +++++++++++++++++++++++ >>   1 file changed, 23 insertions(+) >> >> >> guest-common: >> >> Jakob-Tobias Winter (1): >>    vzdump: add pbs-snapshot-attributes option >> >>   src/PVE/VZDump/Common.pm | 8 ++++++++ >>   1 file changed, 8 insertions(+) >> >> >> manager: >> >> Jakob-Tobias Winter (2): >>    vzdump: store guest tags and pool as snapshot attributes on PBS >>    ui: backup job: add selector for PBS snapshot attributes >> >>   PVE/VZDump.pm                               | 42 +++++++++++++++++++++ >>   configs/vzdump.conf                         |  1 + >>   www/manager6/panel/BackupAdvancedOptions.js | 22 +++++++++++ >>   3 files changed, 65 insertions(+) >> >> >> docs: >> >> Jakob-Tobias Winter (1): >>    vzdump: describe snapshot attributes on Proxmox Backup Server >> >>   vzdump.adoc | 36 ++++++++++++++++++++++++++++++++++++ >>   1 file changed, 36 insertions(+) >> >> >> Summary over all repositories: >>    22 files changed, 799 insertions(+), 14 deletions(-) >> >