From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 9F6EC807C3 for ; Thu, 18 Nov 2021 11:11:34 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 94F7718F41 for ; Thu, 18 Nov 2021 11:11:34 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id A5AE818F35 for ; Thu, 18 Nov 2021 11:11:33 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 6FE6043CCC for ; Thu, 18 Nov 2021 11:11:33 +0100 (CET) Message-ID: Date: Thu, 18 Nov 2021 11:11:32 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:95.0) Gecko/20100101 Thunderbird/95.0 Content-Language: en-US To: Proxmox Backup Server development discussion , =?UTF-8?Q?Fabian_Gr=c3=bcnbichler?= References: <20211028130058.1308810-1-f.gruenbichler@proxmox.com> From: Thomas Lamprecht In-Reply-To: <20211028130058.1308810-1-f.gruenbichler@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.110 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] applied: [PATCH v3 proxmox-backup 0/12] pull/sync group filter X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2021 10:11:34 -0000 On 28.10.21 15:00, Fabian Gr=C3=BCnbichler wrote: > this has been requested a few times on the forum, e.g. for a special > sync job for the most important groups, or seeding of a new datastore > with a partial view of an existing one. >=20 > while it's possible to achieve similar results with hacky workarounds > based on group ownership and reduced "visibility", implementing it > properly is not that complex. >=20 > possible future additions in a similar fashion: > - exclude filters > - filtering in other API calls (tape, listing groups/snapshots) > - only sync/pull encrypted snapshots (less trusted off-site > location) > - only sync/pull latest snapshot in each group (fast seeding of new= > datastore) >=20 > not changed: BackupGroup was not pulled into api-types - we can still > re-evaluate this at some point if we want to switch the split api > parameters we currently have, but at the moment this would mean lots of= > churn for a very small gain. >=20 > changed since v2: > - implement UpdaterType for Vec in proxmox-schema, simplify proxmox-= backup patches > - pull parameters into struct > - added patch for flipped remove-vanished default >=20 > changed since v1: > - reworked filter to support different types, rebased > - dropped last patch > - add docs patch >=20 > proxmox: >=20 > Fabian Gr=C3=BCnbichler (1): > updater: impl UpdaterType for Vec >=20 > proxmox-schema/src/schema.rs | 5 +++++ > 1 file changed, 5 insertions(+) >=20 > proxmox-backup: >=20 > Fabian Gr=C3=BCnbichler (11): > api-types: add schema for backup group > api: add GroupFilter(List) type > BackupGroup: add filter helper > pull: use BackupGroup consistently > pull/sync: extract passed along vars into struct > pull: allow pulling groups selectively > sync: add group filtering > remote: add backup group scanning > manager: render group filter properly > docs: mention group filter in sync docs > fix #sync.cfg/pull: don't remove by default >=20 > docs/managing-remotes.rst | 6 + > debian/postinst | 31 ++++++ > pbs-api-types/src/datastore.rs | 5 + > pbs-api-types/src/jobs.rs | 64 ++++++++++- > pbs-datastore/src/backup_info.rs | 13 +++ > src/api2/config/remote.rs | 77 ++++++++++++- > src/api2/config/sync.rs | 5 + > src/api2/pull.rs | 70 ++++++------ > src/bin/proxmox-backup-manager.rs | 117 +++++++++++++++++-- > src/bin/proxmox_backup_manager/sync.rs | 21 ++++ > src/server/pull.rs | 148 ++++++++++++++++++-------= > www/config/SyncView.js | 13 ++- > www/window/SyncJobEdit.js | 12 ++ > 13 files changed, 487 insertions(+), 95 deletions(-) >=20 applied series, thanks! Made two follow ups: - renamed `groups` to `group-filter` (please verify that nothing bogus wa= s done) - restructured the docs slightly and added a `type` and `group` example