From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <c.ebner@proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 068E2B8671 for <pbs-devel@lists.proxmox.com>; Fri, 8 Mar 2024 14:02:35 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D26E3DC5E for <pbs-devel@lists.proxmox.com>; Fri, 8 Mar 2024 14:02:04 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for <pbs-devel@lists.proxmox.com>; Fri, 8 Mar 2024 14:02:03 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id C1EF1488D1 for <pbs-devel@lists.proxmox.com>; Fri, 8 Mar 2024 14:02:02 +0100 (CET) From: Christian Ebner <c.ebner@proxmox.com> To: pbs-devel@lists.proxmox.com Date: Fri, 8 Mar 2024 14:01:47 +0100 Message-Id: <20240308130150.310352-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.040 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [datastore.rs, pull.rs] Subject: [pbs-devel] [PATCH proxmox-backup 0/3] add removed vanished stats to sync job log X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion <pbs-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/> List-Post: <mailto:pbs-devel@lists.proxmox.com> List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Fri, 08 Mar 2024 13:02:35 -0000 Adds a global summary of the vanished and therefore removed snapshots, backup groups and namespaces to the sync jobs task log. Patch 1/3 introduces a BackupGroupDeleteStats object, used to count the number of removed snapshots within a backup group as well as the number of not removed snapshots, since protected. Patch 2/3 adds an optional RemovedVanishedStats object to the PullStats, counting also the removed backup groups and namespaces and utilizing the returned counts of removed snapshots as introduced by the previous patch. Patch 3/3 finally adds the removed vanished entities output to the sync jobs task log. The series was tested by creating a local sync job and syncing a datastore, containing nested namespaces with snapshots performing the following actions: The `remove vanished` flag was unset for the sync job and the absence of the line in the sync jobs task log verified for that case; then the flag was set once again. Further, removed snapshots on the source store, set some of the removed snapshots to be protected on the target store and compared the removed entities of the sync job by the logged output. Finally, removed the protected flag on the snapshots in the target datastore and removed all namespaces and snapshots in the source datastore. Checked once again the sync jobs log output as compared to the actual removed entities. Christian Ebner (3): datastore: group: return basic stats on backup group destroy server: sync job: include removed vanished stats api: sync job: log stats for removed vanished entities pbs-datastore/src/backup_info.rs | 44 ++++++++++++++--- pbs-datastore/src/datastore.rs | 11 +++-- src/api2/admin/datastore.rs | 3 +- src/api2/pull.rs | 10 ++++ src/server/pull.rs | 85 ++++++++++++++++++++++++++------ 5 files changed, 127 insertions(+), 26 deletions(-) -- 2.39.2