From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 14032AF9F for ; Tue, 8 Aug 2023 14:14:23 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E8B939750 for ; Tue, 8 Aug 2023 14:13:52 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 8 Aug 2023 14:13:52 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id CCE94437B3 for ; Tue, 8 Aug 2023 14:13:51 +0200 (CEST) From: Hannes Laimer To: pbs-devel@lists.proxmox.com Date: Tue, 8 Aug 2023 14:13:38 +0200 Message-Id: <20230808121344.199500-1-h.laimer@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.017 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [catalog.rs, main.rs, download-speed.rs, jobs.rs, sync.rs, proxmox-backup-manager.rs, tasks.rs, diff.rs, remote.rs, mount.rs, pull.rs] Subject: [pbs-devel] [PATCH proxmox-backup v3 0/6] local sync-jobs X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2023 12:14:23 -0000 Add support for local sync. SyncJobs without a remote are considered local, and use a different logic for pulling. In the course of adding the extra pull logic, the pull code was rewritten to basically be source independent. Also cli completion and the UI was updated to allow Remotes in SyncJobs to be optional. v2, thanks @Fabian Grünbichler for the feedback and suggestions on v1. - make pull logic more source independent v3: thanks @Fabian and @Wolfang for the feedback on v2 - remove enums for Local/Remote - use traits, pull logic now expects a `dyn PullSource`(or `dyn PullReader`) - add lock to dir for local sync - split refactoring of pull logic and implementation of local pulling into two commits Hannes Laimer (6): api2: make Remote for SyncJob optional ui: add support for optional Remote in SyncJob manager: add completion for opt. Remote in SyncJob accept a ref to a HttpClient pull: refactor pulling from a datastore pull: add support for pulling from local datastore Cargo.toml | 2 + examples/download-speed.rs | 2 +- pbs-api-types/src/jobs.rs | 5 +- pbs-client/src/backup_reader.rs | 2 +- pbs-datastore/src/read_chunk.rs | 2 +- proxmox-backup-client/src/catalog.rs | 4 +- proxmox-backup-client/src/main.rs | 2 +- proxmox-backup-client/src/mount.rs | 2 +- proxmox-file-restore/src/main.rs | 4 +- src/api2/config/remote.rs | 16 +- src/api2/config/sync.rs | 41 +- src/api2/node/tasks.rs | 4 +- src/api2/pull.rs | 45 +- src/bin/proxmox-backup-manager.rs | 67 +- src/bin/proxmox_backup_debug/diff.rs | 2 +- src/server/email_notifications.rs | 18 +- src/server/pull.rs | 1080 ++++++++++++++++---------- www/form/RemoteTargetSelector.js | 29 +- www/window/SyncJobEdit.js | 8 +- 19 files changed, 846 insertions(+), 489 deletions(-) -- 2.39.2