From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id CAB5E70B16 for ; Mon, 9 May 2022 12:41:01 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B7F2A2B9C1 for ; Mon, 9 May 2022 12:40:31 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 19E3B2B9B7 for ; Mon, 9 May 2022 12:40:31 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id E0129431C0 for ; Mon, 9 May 2022 12:40:30 +0200 (CEST) From: Dominik Csapak To: pbs-devel@lists.proxmox.com Date: Mon, 9 May 2022 12:40:30 +0200 Message-Id: <20220509104030.1943794-1-d.csapak@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.120 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pbs-devel] [PATCH proxmox-backup] chunk_store: insert_chunk: write chunk again if sizes don't match X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 May 2022 10:41:01 -0000 if the on-disk size of a chunk is not correct, write it again when inserting and log a warning. This is currently possible if PBS crashes, but the rename of the chunk was flushed to disk, when the actual data was not. Suggested-by: Fabian Grünbichler Signed-off-by: Dominik Csapak --- pbs-datastore/src/chunk_store.rs | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/pbs-datastore/src/chunk_store.rs b/pbs-datastore/src/chunk_store.rs index 8d7df513..93f56e8b 100644 --- a/pbs-datastore/src/chunk_store.rs +++ b/pbs-datastore/src/chunk_store.rs @@ -458,17 +458,29 @@ impl ChunkStore { let lock = self.mutex.lock(); + let raw_data = chunk.raw_data(); + let encoded_size = raw_data.len() as u64; + if let Ok(metadata) = std::fs::metadata(&chunk_path) { - if metadata.is_file() { - self.touch_chunk(digest)?; - return Ok((true, metadata.len())); - } else { + if !metadata.is_file() { bail!( "Got unexpected file type on store '{}' for chunk {}", self.name, digest_str ); } + let new_len = metadata.len(); + if encoded_size == new_len { + self.touch_chunk(digest)?; + return Ok((true, new_len)); + } else { + log::warn!( + "chunk size mismatch on insert for {}: old {} - new {}", + digest_str, + encoded_size, + new_len + ); + } } let mut tmp_path = chunk_path.clone(); @@ -483,9 +495,6 @@ impl ChunkStore { ) })?; - let raw_data = chunk.raw_data(); - let encoded_size = raw_data.len() as u64; - file.write_all(raw_data).map_err(|err| { format_err!( "writing temporary chunk on store '{}' failed for {} - {}", -- 2.30.2