From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id D39897C1AD for ; Thu, 4 Nov 2021 10:57:38 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D197E1DF5C for ; Thu, 4 Nov 2021 10:57:38 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 68ED81DF51 for ; Thu, 4 Nov 2021 10:57:35 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 41852469D9; Thu, 4 Nov 2021 10:57:35 +0100 (CET) Message-ID: Date: Thu, 4 Nov 2021 10:57:33 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Thunderbird/94.0 Content-Language: en-US To: Proxmox Backup Server development discussion , =?UTF-8?Q?Fabian_Gr=c3=bcnbichler?= References: <20211028130058.1308810-1-f.gruenbichler@proxmox.com> From: Dominik Csapak In-Reply-To: <20211028130058.1308810-1-f.gruenbichler@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 2.010 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -3.528 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [datastore.rs, remote.rs, schema.rs, pull.rs, proxmox-backup-manager.rs, sync.rs, jobs.rs] Subject: Re: [pbs-devel] [PATCH v3 proxmox-backup 0/12] pull/sync group filter X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Nov 2021 09:57:38 -0000 series LGTM, i sent the tape filter series on-top of this series Reviewed-by: Dominik Csapak On 10/28/21 15:00, Fabian Grünbichler wrote: > this has been requested a few times on the forum, e.g. for a special > sync job for the most important groups, or seeding of a new datastore > with a partial view of an existing one. > > while it's possible to achieve similar results with hacky workarounds > based on group ownership and reduced "visibility", implementing it > properly is not that complex. > > possible future additions in a similar fashion: > - exclude filters > - filtering in other API calls (tape, listing groups/snapshots) > - only sync/pull encrypted snapshots (less trusted off-site > location) > - only sync/pull latest snapshot in each group (fast seeding of new > datastore) > > not changed: BackupGroup was not pulled into api-types - we can still > re-evaluate this at some point if we want to switch the split api > parameters we currently have, but at the moment this would mean lots of > churn for a very small gain. > > changed since v2: > - implement UpdaterType for Vec in proxmox-schema, simplify proxmox-backup patches > - pull parameters into struct > - added patch for flipped remove-vanished default > > changed since v1: > - reworked filter to support different types, rebased > - dropped last patch > - add docs patch > > proxmox: > > Fabian Grünbichler (1): > updater: impl UpdaterType for Vec > > proxmox-schema/src/schema.rs | 5 +++++ > 1 file changed, 5 insertions(+) > > proxmox-backup: > > Fabian Grünbichler (11): > api-types: add schema for backup group > api: add GroupFilter(List) type > BackupGroup: add filter helper > pull: use BackupGroup consistently > pull/sync: extract passed along vars into struct > pull: allow pulling groups selectively > sync: add group filtering > remote: add backup group scanning > manager: render group filter properly > docs: mention group filter in sync docs > fix #sync.cfg/pull: don't remove by default > > docs/managing-remotes.rst | 6 + > debian/postinst | 31 ++++++ > pbs-api-types/src/datastore.rs | 5 + > pbs-api-types/src/jobs.rs | 64 ++++++++++- > pbs-datastore/src/backup_info.rs | 13 +++ > src/api2/config/remote.rs | 77 ++++++++++++- > src/api2/config/sync.rs | 5 + > src/api2/pull.rs | 70 ++++++------ > src/bin/proxmox-backup-manager.rs | 117 +++++++++++++++++-- > src/bin/proxmox_backup_manager/sync.rs | 21 ++++ > src/server/pull.rs | 148 ++++++++++++++++++------- > www/config/SyncView.js | 13 ++- > www/window/SyncJobEdit.js | 12 ++ > 13 files changed, 487 insertions(+), 95 deletions(-) >