From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id A6DF669123 for ; Mon, 1 Mar 2021 15:13:17 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 9A70C21122 for ; Mon, 1 Mar 2021 15:12:47 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id E98D121109 for ; Mon, 1 Mar 2021 15:12:46 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id B9E2845830 for ; Mon, 1 Mar 2021 15:12:46 +0100 (CET) From: Stoiko Ivanov To: pmg-devel@lists.proxmox.com Date: Mon, 1 Mar 2021 15:12:18 +0100 Message-Id: <20210301141225.18394-2-s.ivanov@proxmox.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210301141225.18394-1-s.ivanov@proxmox.com> References: <20210301141225.18394-1-s.ivanov@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.063 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pmg-devel] [PATCH pmg-api v2 1/3] backup: pbs: prevent race in concurrent backups X-BeenThere: pmg-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Mail Gateway development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Mar 2021 14:13:17 -0000 If two pbs backup-creation calls happen simultaneously, it is possible that the first removes the backup dir before the other is done creating or sending it to the pbs remote. This patch takes the same route as non-PBS backups - creating a unique tempdir indexed by remote, PID and current time. the tmp-dir now also needs to be removed in case of error while backing up. (before the next invocation would have wiped it). Noticed while having 2 schedules to different PBS instances with the same interval and w/o random delay. Signed-off-by: Stoiko Ivanov --- src/PMG/API2/PBS/Job.pm | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/src/PMG/API2/PBS/Job.pm b/src/PMG/API2/PBS/Job.pm index 2387422..c095d25 100644 --- a/src/PMG/API2/PBS/Job.pm +++ b/src/PMG/API2/PBS/Job.pm @@ -294,20 +294,28 @@ __PACKAGE__->register_method ({ $param->{statistic} //= $remote_config->{'include-statistics'} // 1; my $pbs = PVE::PBSClient->new($remote_config, $remote, $conf->{secret_dir}); - my $backup_dir = "/var/lib/pmg/backup/current"; + + my $time = time; + my $backup_dir = "/tmp/pbsbackup_${remote}_$$.$time"; my $worker = sub { my $upid = shift; print "starting update of current backup state\n"; - -d $backup_dir || mkdir $backup_dir; - PMG::Backup::pmg_backup($backup_dir, $param->{statistic}); + eval { + -d $backup_dir || mkdir $backup_dir; + PMG::Backup::pmg_backup($backup_dir, $param->{statistic}); - $pbs->backup_fs_tree($backup_dir, $node, 'pmgbackup'); + $pbs->backup_fs_tree($backup_dir, $node, 'pmgbackup'); - rmtree $backup_dir; + rmtree $backup_dir; + }; + if (my $err = $@) { + rmtree $backup_dir; + die "backup failed: $err\n"; + } print "backup finished\n"; my $group = "host/$node"; -- 2.20.1