From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <t.lamprecht@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id 9F6EC807C3
 for <pbs-devel@lists.proxmox.com>; Thu, 18 Nov 2021 11:11:34 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id 94F7718F41
 for <pbs-devel@lists.proxmox.com>; Thu, 18 Nov 2021 11:11:34 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [94.136.29.106])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS id A5AE818F35
 for <pbs-devel@lists.proxmox.com>; Thu, 18 Nov 2021 11:11:33 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 6FE6043CCC
 for <pbs-devel@lists.proxmox.com>; Thu, 18 Nov 2021 11:11:33 +0100 (CET)
Message-ID: <b47dbeb1-cad8-a275-b5d6-302c42a1f45f@proxmox.com>
Date: Thu, 18 Nov 2021 11:11:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:95.0) Gecko/20100101
 Thunderbird/95.0
Content-Language: en-US
To: Proxmox Backup Server development discussion <pbs-devel@lists.proxmox.com>,
 =?UTF-8?Q?Fabian_Gr=c3=bcnbichler?= <f.gruenbichler@proxmox.com>
References: <20211028130058.1308810-1-f.gruenbichler@proxmox.com>
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
In-Reply-To: <20211028130058.1308810-1-f.gruenbichler@proxmox.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-LEVEL: Spam detection results:  0
 AWL 0.110 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
Subject: [pbs-devel] applied: [PATCH v3 proxmox-backup 0/12] pull/sync group
 filter
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
 <pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Thu, 18 Nov 2021 10:11:34 -0000

On 28.10.21 15:00, Fabian Gr=C3=BCnbichler wrote:
> this has been requested a few times on the forum, e.g. for a special
> sync job for the most important groups, or seeding of a new datastore
> with a partial view of an existing one.
>=20
> while it's possible to achieve similar results with hacky workarounds
> based on group ownership and reduced "visibility", implementing it
> properly is not that complex.
>=20
> possible future additions in a similar fashion:
>     - exclude filters
>     - filtering in other API calls (tape, listing groups/snapshots)
>     - only sync/pull encrypted snapshots (less trusted off-site
>       location)
>     - only sync/pull latest snapshot in each group (fast seeding of new=

>       datastore)
>=20
> not changed: BackupGroup was not pulled into api-types - we can still
> re-evaluate this at some point if we want to switch the split api
> parameters we currently have, but at the moment this would mean lots of=

> churn for a very small gain.
>=20
> changed since v2:
> - implement UpdaterType for Vec<T> in proxmox-schema, simplify proxmox-=
backup patches
> - pull parameters into struct
> - added patch for flipped remove-vanished default
>=20
> changed since v1:
> - reworked filter to support different types, rebased
> - dropped last patch
> - add docs patch
>=20
> proxmox:
>=20
> Fabian Gr=C3=BCnbichler (1):
>   updater: impl UpdaterType for Vec
>=20
>  proxmox-schema/src/schema.rs | 5 +++++
>  1 file changed, 5 insertions(+)
>=20
> proxmox-backup:
>=20
> Fabian Gr=C3=BCnbichler (11):
>   api-types: add schema for backup group
>   api: add GroupFilter(List) type
>   BackupGroup: add filter helper
>   pull: use BackupGroup consistently
>   pull/sync: extract passed along vars into struct
>   pull: allow pulling groups selectively
>   sync: add group filtering
>   remote: add backup group scanning
>   manager: render group filter properly
>   docs: mention group filter in sync docs
>   fix #sync.cfg/pull: don't remove by default
>=20
>  docs/managing-remotes.rst              |   6 +
>  debian/postinst                        |  31 ++++++
>  pbs-api-types/src/datastore.rs         |   5 +
>  pbs-api-types/src/jobs.rs              |  64 ++++++++++-
>  pbs-datastore/src/backup_info.rs       |  13 +++
>  src/api2/config/remote.rs              |  77 ++++++++++++-
>  src/api2/config/sync.rs                |   5 +
>  src/api2/pull.rs                       |  70 ++++++------
>  src/bin/proxmox-backup-manager.rs      | 117 +++++++++++++++++--
>  src/bin/proxmox_backup_manager/sync.rs |  21 ++++
>  src/server/pull.rs                     | 148 ++++++++++++++++++-------=

>  www/config/SyncView.js                 |  13 ++-
>  www/window/SyncJobEdit.js              |  12 ++
>  13 files changed, 487 insertions(+), 95 deletions(-)
>=20



applied series, thanks!

Made two follow ups:
- renamed `groups` to `group-filter` (please verify that nothing bogus wa=
s done)
- restructured the docs slightly and added a `type` and `group` example