[pbs-devel] [RFC proxmox-backup] api: datastore: determine size and blob type if not in manifest

* [pbs-devel] [RFC proxmox-backup] api: datastore: determine size and blob type if not in manifest
@ 2020-10-14 15:04 Stefan Reiter
  2020-10-14 15:30 ` Thomas Lamprecht
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Stefan Reiter @ 2020-10-14 15:04 UTC (permalink / raw)
  To: pbs-devel

Try to read the first 8 byte (DataBlobHeader magic) from any .blob file
not mentioned in the manifest. This allows classifying the
client.log.blob file as encrypted if it is, and showing the correct
symbol in the GUI for it.

While already open, also determine the file size by seeking to End(0).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---

I noticed that the GUI would show the "download" icon enabled for
'client.log.blob' files, even if they were encrypted. The button wouldn't work
of course, since the server can't decode encrypted blobs, so nothing would
happen except an error logged to the console.

The problem is that currently we don't know if a blob file not listed in the
manifest is encrypted - but we can find out, since it's encoded into the header.
Not sure if this is worth the open() and read() for every unknown blob file, but
it would solve the problem...


 src/api2/admin/datastore.rs | 25 +++++++++++++++++++++++--
 src/backup/data_blob.rs     | 12 ++++++++++++
 2 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
index 11223e6a..2a9ac469 100644
--- a/src/api2/admin/datastore.rs
+++ b/src/api2/admin/datastore.rs
@@ -2,6 +2,7 @@ use std::collections::{HashSet, HashMap};
 use std::ffi::OsStr;
 use std::os::unix::ffi::OsStrExt;
 use std::sync::{Arc, Mutex};
+use std::io::{Read, Seek, SeekFrom};
 
 use anyhow::{bail, format_err, Error};
 use futures::*;
@@ -91,10 +92,30 @@ fn get_all_snapshot_files(
 
     for file in &info.files {
         if file_set.contains(file) { continue; }
+
+        // Try to determine size and crypt mode for unknown blobs
+        let mut crypt_mode = None;
+        let mut size = None;
+        if file.ends_with(".blob") {
+            let mut path = store.snapshot_path(&info.backup_dir);
+            path.push(file);
+            if let Ok(mut file) = std::fs::File::open(path) {
+                let mut buffer = [0u8; 8];
+                if let Ok(n) = file.read(&mut buffer[..]) {
+                    if n == buffer.len() {
+                        crypt_mode = DataBlob::is_blob_magic(&buffer);
+                    }
+                }
+                if let Ok(pos) = file.seek(SeekFrom::End(0)) {
+                    size = Some(pos);
+                }
+            }
+        }
+
         files.push(BackupContent {
             filename: file.to_string(),
-            size: None,
-            crypt_mode: None,
+            size,
+            crypt_mode,
         });
     }
 
diff --git a/src/backup/data_blob.rs b/src/backup/data_blob.rs
index 284dc243..626a5fe4 100644
--- a/src/backup/data_blob.rs
+++ b/src/backup/data_blob.rs
@@ -321,6 +321,18 @@ impl DataBlob {
 
         Ok(())
     }
+
+    /// Determine if the given value is a valid blob magic number.
+    /// Returns CryptMode::Encrypt or CryptMode::None depending on type.
+    pub fn is_blob_magic(magic: &[u8; 8]) -> Option<CryptMode> {
+        if magic == &UNCOMPRESSED_BLOB_MAGIC_1_0 || magic == &COMPRESSED_BLOB_MAGIC_1_0 {
+            Some(CryptMode::None)
+        } else if magic == &ENCRYPTED_BLOB_MAGIC_1_0 || magic == &ENCR_COMPR_BLOB_MAGIC_1_0 {
+            Some(CryptMode::Encrypt)
+        } else {
+            None
+        }
+    }
 }
 
 /// Builder for chunk DataBlobs
-- 
2.20.1





^ permalink raw reply	[flat|nested] 4+ messages in thread