public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Samuel Rufinatscha <s.rufinatscha@proxmox.com>
To: Jakob-Tobias Winter <tobias@stuttgart.de>, pbs-devel@lists.proxmox.com
Subject: Re: [RFC PATCH proxmox-backup 05/12] tape: backup job: allow selecting groups by snapshot attributes
Date: Wed, 18 Nov 2026 14:06:59 +0100	[thread overview]
Message-ID: <0ab4a114-3946-4855-842f-9a6dfb5ae1bf@proxmox.com> (raw)
In-Reply-To: <20261113085637.57360-8-tobias@stuttgart.de>

On 11/13/26 10:06 AM, Jakob-Tobias Winter wrote:
> Tape backup jobs can only select backup groups by type and ID so
> far. Such filters have to be kept up to date by hand for every new
> guest, and an ID says nothing about how important a guest is.
> 
> Add an attribute filter to tape backup jobs. With a client which
> stores the tags of a guest as snapshot attributes, production and
> test guests can be written to different media pools with different
> retention policies, by using one job for each media pool. The media
> sets of the production guests then need fewer tapes and are faster
> to restore.
> 
> A group is included if it has one of the attributes given as include
> filter, or if there are no include filters, and none of the
> attributes given as exclude filter. The filter is applied in
> addition to the group filter.
> 
> Jobs without an attribute filter select the same groups as before.
> 
> Signed-off-by: Jakob-Tobias Winter <tobias@stuttgart.de>
> ---
> A version without this patch cannot parse a tape-job.cfg in which the
> new option is used (unknown property 'attribute-filter').
> 
>   src/api2/config/tape_backup_job.rs |  8 ++++++++
>   src/api2/tape/backup.rs            | 19 ++++++++++++++++---
>   src/bin/proxmox-tape.rs            | 11 ++++++++---
>   3 files changed, 32 insertions(+), 6 deletions(-)
> 
> diff --git a/src/api2/config/tape_backup_job.rs b/src/api2/config/tape_backup_job.rs
> index 4bfc7f871..cb9e63ea6 100644
> --- a/src/api2/config/tape_backup_job.rs
> +++ b/src/api2/config/tape_backup_job.rs
> @@ -144,6 +144,8 @@ pub enum DeletableProperty {
>       NotificationMode,
>       /// Delete the 'group_filter' property
>       GroupFilter,
> +    /// Delete the 'attribute-filter' property
> +    AttributeFilter,
>       /// Delete the 'max-depth' property
>       MaxDepth,
>       /// Delete the 'ns' property
> @@ -227,6 +229,9 @@ pub fn update_tape_backup_job(
>                   DeletableProperty::GroupFilter => {
>                       data.setup.group_filter = None;
>                   }
> +                DeletableProperty::AttributeFilter => {
> +                    data.setup.attribute_filter = None;
> +                }
>                   DeletableProperty::MaxDepth => {
>                       data.setup.max_depth = None;
>                   }
> @@ -268,6 +273,9 @@ pub fn update_tape_backup_job(
>       if update.setup.group_filter.is_some() {
>           data.setup.group_filter = update.setup.group_filter;
>       }
> +    if update.setup.attribute_filter.is_some() {
> +        data.setup.attribute_filter = update.setup.attribute_filter;
> +    }
>       if update.setup.ns.is_some() {
>           data.setup.ns = update.setup.ns;
>       }
> diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
> index c0a4ca65b..c2797a97a 100644
> --- a/src/api2/tape/backup.rs
> +++ b/src/api2/tape/backup.rs
> @@ -11,9 +11,9 @@ use proxmox_schema::api;
>   use proxmox_worker_task::WorkerTaskContext;
>   
>   use pbs_api_types::{
> -    Authid, JOB_ID_SCHEMA, MediaPoolConfig, Operation, PRIV_TAPE_AUDIT, TapeBackupJobConfig,
> -    TapeBackupJobSetup, TapeBackupJobStatus, UPID_SCHEMA, print_ns_and_snapshot,
> -    print_store_and_ns,
> +    AttributeFilter, Authid, JOB_ID_SCHEMA, MediaPoolConfig, Operation, PRIV_TAPE_AUDIT,
> +    TapeBackupJobConfig, TapeBackupJobSetup, TapeBackupJobStatus, UPID_SCHEMA,
> +    print_ns_and_snapshot, print_store_and_ns,
>   };
>   
>   use pbs_config::CachedUserInfo;
> @@ -391,6 +391,19 @@ fn backup_worker(
>           None => group_list,
>       };
>   
> +    let group_list = match &setup.attribute_filter {
> +        Some(filters) => {
> +            let mut filtered = Vec::new();
> +            for group in group_list {
> +                if AttributeFilter::apply_filters(filters, &group.attributes()?) {

once we filter for individual snapshots as suggested by @Christian,
how should latest-only behave?

For example, if monday and tuesday are tagged with "test" and wednesday
is tagged with "prod", should a test job copy the tuesday snapshot or
skip the group because the latest backup in the group is on "prod"?

I would lean towards copying tuesday’s snapshot in this example since
it’s the newest one and it matches the job’s filter

> +                    filtered.push(group);
> +                }
> +            }
> +            filtered
> +        }
> +        None => group_list,
> +    };
> +
>       info!(
>           "found {} groups (out of {group_count_full} total)",
>           group_list.len()
> diff --git a/src/bin/proxmox-tape.rs b/src/bin/proxmox-tape.rs
> index dcbf2d0fb..6ea46b709 100644
> --- a/src/bin/proxmox-tape.rs
> +++ b/src/bin/proxmox-tape.rs
> @@ -19,9 +19,10 @@ use pbs_config::drive::complete_drive_name;
>   use pbs_config::media_pool::complete_pool_name;
>   
>   use pbs_api_types::{
> -    Authid, BackupNamespace, DATASTORE_MAP_LIST_SCHEMA, DATASTORE_SCHEMA, DRIVE_NAME_SCHEMA,
> -    GROUP_FILTER_LIST_SCHEMA, GroupListItem, MEDIA_LABEL_SCHEMA, MEDIA_POOL_NAME_SCHEMA,
> -    NS_MAX_DEPTH_SCHEMA, TAPE_RESTORE_NAMESPACE_SCHEMA, TAPE_RESTORE_SNAPSHOT_SCHEMA, Userid,
> +    ATTRIBUTE_FILTER_LIST_SCHEMA, Authid, BackupNamespace, DATASTORE_MAP_LIST_SCHEMA,
> +    DATASTORE_SCHEMA, DRIVE_NAME_SCHEMA, GROUP_FILTER_LIST_SCHEMA, GroupListItem,
> +    MEDIA_LABEL_SCHEMA, MEDIA_POOL_NAME_SCHEMA, NS_MAX_DEPTH_SCHEMA, TAPE_RESTORE_NAMESPACE_SCHEMA,
> +    TAPE_RESTORE_SNAPSHOT_SCHEMA, Userid,
>   };
>   use pbs_tape::{BlockReadError, MediaContentHeader, PROXMOX_BACKUP_CONTENT_HEADER_MAGIC_1_0};
>   
> @@ -850,6 +851,10 @@ async fn clean_drive(mut param: Value) -> Result<(), Error> {
>                   schema: GROUP_FILTER_LIST_SCHEMA,
>                   optional: true,
>               },
> +            "attribute-filter": {
> +                schema: ATTRIBUTE_FILTER_LIST_SCHEMA,
> +                optional: true,
> +            },
>               ns: {
>                   type: BackupNamespace,
>                   optional: true,





  reply	other threads:[~2026-11-18 13:07 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13  8:56 [RFC PATCH many 00/12] snapshot attributes for tags and pools of PVE guests, tape job filter Jakob-Tobias Winter
2026-11-13  8:56 ` [RFC PATCH proxmox 01/12] pbs-api-types: add types and schemas for snapshot attributes Jakob-Tobias Winter
2026-11-16 14:57   ` Christian Ebner
2026-11-18 12:54   ` Samuel Rufinatscha
2026-11-13  8:56 ` [RFC PATCH proxmox 02/12] pbs-api-types: tape: add attribute filter to backup jobs Jakob-Tobias Winter
2026-11-16 14:57   ` Christian Ebner
2026-11-13  8:56 ` [RFC PATCH proxmox-backup 03/12] datastore: add attributes to backup snapshots Jakob-Tobias Winter
2026-11-16 14:57   ` Christian Ebner
2026-11-18 12:55   ` Samuel Rufinatscha
2026-11-13  8:56 ` [RFC PATCH proxmox-backup 04/12] fix #7452: api: datastore: add endpoints for snapshot attributes Jakob-Tobias Winter
2026-11-16 14:57   ` Christian Ebner
2026-11-13  8:56 ` [RFC PATCH proxmox-backup 05/12] tape: backup job: allow selecting groups by " Jakob-Tobias Winter
2026-11-18 13:06   ` Samuel Rufinatscha [this message]
2026-11-18 15:19     ` Christian Ebner
2026-11-13  8:56 ` [RFC PATCH proxmox-backup 06/12] ui: tape backup job: add attribute filter Jakob-Tobias Winter
2026-11-13  8:56 ` [RFC PATCH proxmox-backup 07/12] docs: describe snapshot attributes and the tape job " Jakob-Tobias Winter
2026-11-13  8:56 ` [RFC PATCH storage 08/12] pbs: add helper to set the attributes of a backup snapshot Jakob-Tobias Winter
2026-11-13  8:56 ` [RFC PATCH guest-common 09/12] vzdump: add pbs-snapshot-attributes option Jakob-Tobias Winter
2026-11-13  8:56 ` [RFC PATCH manager 10/12] vzdump: store guest tags and pool as snapshot attributes on PBS Jakob-Tobias Winter
2026-11-13  8:56 ` [RFC PATCH manager 11/12] ui: backup job: add selector for PBS snapshot attributes Jakob-Tobias Winter
2026-11-13  8:56 ` [RFC PATCH docs 12/12] vzdump: describe snapshot attributes on Proxmox Backup Server Jakob-Tobias Winter
2026-11-16 14:57 ` [RFC PATCH many 00/12] snapshot attributes for tags and pools of PVE guests, tape job filter Christian Ebner
2026-11-18 13:07   ` Samuel Rufinatscha
2026-11-18 14:22 ` Samuel Rufinatscha
2026-11-19 11:58   ` Jakob-Tobias Winter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0ab4a114-3946-4855-842f-9a6dfb5ae1bf@proxmox.com \
    --to=s.rufinatscha@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    --cc=tobias@stuttgart.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal