[pbs-devel] [PATCH proxmox-backup v2] chunk_store: insert_chunk: write chunk again if it is empty on disk

[pbs-devel] [PATCH proxmox-backup v2] chunk_store: insert_chunk: write chunk again if it is empty on disk

[Dominik Csapak]

and issue a warning. We can do this, because we know an empty chunk cannot be
valid, and we (assumedly) have a valid chunk in memory.

Having empty chunks on disk is currently possible when PBS crashes, but the
rename of the chunk was flushed to disk, when the actual data was not.

If it's not empty but there is a size mismatch, return an error.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
replaces: https://lists.proxmox.com/pipermail/pbs-devel/2022-May/005042.html

changes from v1:
* warn only on old_size == 0
* fix old_size/new_size
* expand commit message

we could do further checks here, but those should be caught by verify.
the 'real' fix for most of the 0-size chunks will be to either fsync
the chunks after write/rename, or do a syncfs before the backup ends

 pbs-datastore/src/chunk_store.rs | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/pbs-datastore/src/chunk_store.rs b/pbs-datastore/src/chunk_store.rs
index 8d7df513..85e7cc71 100644
--- a/pbs-datastore/src/chunk_store.rs
+++ b/pbs-datastore/src/chunk_store.rs
@@ -458,17 +458,26 @@ impl ChunkStore {
 
         let lock = self.mutex.lock();
 
+        let raw_data = chunk.raw_data();
+        let encoded_size = raw_data.len() as u64;
+
         if let Ok(metadata) = std::fs::metadata(&chunk_path) {
-            if metadata.is_file() {
-                self.touch_chunk(digest)?;
-                return Ok((true, metadata.len()));
-            } else {
+            if !metadata.is_file() {
                 bail!(
                     "Got unexpected file type on store '{}' for chunk {}",
                     self.name,
                     digest_str
                 );
             }
+            let old_size = metadata.len();
+            if encoded_size == old_size {
+                self.touch_chunk(digest)?;
+                return Ok((true, old_size));
+            } else if old_size == 0 {
+                log::warn!("found empty chunk '{digest_str}' on disk, overwriting");
+            } else {
+                bail!("found chunk size mismatch for '{digest_str}': old {old_size} - new {encoded_size}");
+            }
         }
 
         let mut tmp_path = chunk_path.clone();
@@ -483,9 +492,6 @@ impl ChunkStore {
             )
         })?;
 
-        let raw_data = chunk.raw_data();
-        let encoded_size = raw_data.len() as u64;
-
         file.write_all(raw_data).map_err(|err| {
             format_err!(
                 "writing temporary chunk on store '{}' failed for {} - {}",
-- 
2.30.2

[pbs-devel] applied: [PATCH proxmox-backup v2] chunk_store: insert_chunk: write chunk again if it is empty on disk

[Thomas Lamprecht]

Am 5/9/22 um 16:09 schrieb Dominik Csapak:
> and issue a warning. We can do this, because we know an empty chunk cannot be
> valid, and we (assumedly) have a valid chunk in memory.
> 
> Having empty chunks on disk is currently possible when PBS crashes, but the
> rename of the chunk was flushed to disk, when the actual data was not.
> 
> If it's not empty but there is a size mismatch, return an error.
> 
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> replaces: https://lists.proxmox.com/pipermail/pbs-devel/2022-May/005042.html
> 
> changes from v1:
> * warn only on old_size == 0
> * fix old_size/new_size
> * expand commit message
> 
> we could do further checks here, but those should be caught by verify.
> the 'real' fix for most of the 0-size chunks will be to either fsync
> the chunks after write/rename, or do a syncfs before the backup ends
> 
>  pbs-datastore/src/chunk_store.rs | 20 +++++++++++++-------
>  1 file changed, 13 insertions(+), 7 deletions(-)
> 
>

applied, thanks!