From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup 2/4] sync: verify chunk size and digest, if possible
Date: Mon, 3 Aug 2020 14:10:44 +0200 [thread overview]
Message-ID: <20200803121046.3623216-3-f.gruenbichler@proxmox.com> (raw)
In-Reply-To: <20200803121046.3623216-1-f.gruenbichler@proxmox.com>
for encrypted chunks this is currently not possible, as we need the key
to decode the chunk.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
src/client/pull.rs | 10 ++++++----
src/client/remote_chunk_reader.rs | 4 ++--
2 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/src/client/pull.rs b/src/client/pull.rs
index fc193a8d..629e8266 100644
--- a/src/client/pull.rs
+++ b/src/client/pull.rs
@@ -27,16 +27,18 @@ async fn pull_index_chunks<I: IndexFile>(
for pos in 0..index.index_count() {
- let digest = index.index_digest(pos).unwrap();
- let chunk_exists = target.cond_touch_chunk(digest, false)?;
+ let info = index.chunk_info(pos).unwrap();
+ let chunk_exists = target.cond_touch_chunk(&info.digest, false)?;
if chunk_exists {
//worker.log(format!("chunk {} exists {}", pos, proxmox::tools::digest_to_hex(digest)));
continue;
}
//worker.log(format!("sync {} chunk {}", pos, proxmox::tools::digest_to_hex(digest)));
- let chunk = chunk_reader.read_raw_chunk(&digest).await?;
+ let chunk = chunk_reader.read_raw_chunk(&info.digest).await?;
- target.insert_chunk(&chunk, &digest)?;
+ chunk.verify_unencrypted(info.size() as usize, &info.digest)?;
+
+ target.insert_chunk(&chunk, &info.digest)?;
}
Ok(())
diff --git a/src/client/remote_chunk_reader.rs b/src/client/remote_chunk_reader.rs
index 4db11477..b30d0567 100644
--- a/src/client/remote_chunk_reader.rs
+++ b/src/client/remote_chunk_reader.rs
@@ -35,6 +35,8 @@ impl RemoteChunkReader {
}
}
+ /// Downloads raw chunk. This only verifies the (untrusted) CRC32, use
+ /// DataBlob::verify_unencrypted or DataBlob::decode before storing/processing further.
pub async fn read_raw_chunk(&self, digest: &[u8; 32]) -> Result<DataBlob, Error> {
let mut chunk_data = Vec::with_capacity(4 * 1024 * 1024);
@@ -43,8 +45,6 @@ impl RemoteChunkReader {
.await?;
let chunk = DataBlob::load_from_reader(&mut &chunk_data[..])?;
-
- // fixme: verify digest?
Ok(chunk)
}
--
2.20.1
next prev parent reply other threads:[~2020-08-03 12:10 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-03 12:10 [pbs-devel] [PATCH proxmox-backup 0/4] add missing digest verification Fabian Grünbichler
2020-08-03 12:10 ` [pbs-devel] [PATCH proxmox-backup 1/4] blobs: attempt to verify on decode when possible Fabian Grünbichler
2020-08-03 12:10 ` Fabian Grünbichler [this message]
2020-08-03 12:10 ` [pbs-devel] [PATCH proxmox-backup 3/4] sync: verify size and checksum of pulled archives Fabian Grünbichler
2020-08-03 12:10 ` [pbs-devel] [PATCH proxmox-backup 4/4] datastore: allow browsing signed pxar files Fabian Grünbichler
2020-08-04 5:29 ` [pbs-devel] applied: [PATCH proxmox-backup 0/4] add missing digest verification Dietmar Maurer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200803121046.3623216-3-f.gruenbichler@proxmox.com \
--to=f.gruenbichler@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox