public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: c.ebner@proxmox.com
Cc: pbs-devel@lists.proxmox.com
Subject: Re: [PATCH proxmox-backup v3 28/30] sync: pull: decrypt snapshots with matching encryption key fingerprint
Date: Sun, 19 Apr 2026 22:41:59 +0200	[thread overview]
Message-ID: <20260419210610.3915597-10-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20260414125923.892345-29-c.ebner@proxmox.com>

Am 14.04.26 um 14:59 schrieb Christian Ebner:
> diff --git a/src/server/pull.rs b/src/server/pull.rs
> @@ -783,9 +781,15 @@ async fn pull_snapshot<'a>(
>
> +        new_manifest.unprotected = manifest.unprotected.clone();
> +        if let Some(unprotected) = new_manifest.unprotected.as_object_mut() {
> +            unprotected.remove("key-fingerprint");
> +            unprotected.remove("verify_state");
> +        }

Shouldn't this also remove the change-detection-fingerprint? That field was
computed from the source's encrypted file entries and doesn't match the
decrypted local content. On the next sync:

1. raw manifest comparison fails (encrypted vs decrypted blobs)
2. change_detection_fingerprint() gets recomputed from the local decrypted
   entries, which won't match the source's stored one that was computed from
   encrypted entries
3. no fast path fires and everything gets re-downloaded and re-decrypted

For large datastores that's a pretty significant hit on every re-sync even
when nothing actually changed on the source. I think either removing the
stale fingerprint here or storing the source fingerprint as-is (without
recomputing from local entries on the next round) would fix it.

> +        tmp_manifest_file.write_all(&manifest_data).await?;
> +        tmp_manifest_file.flush().await?;

nit: only flush() but no fsync() before the rename below. Patch 26 correctly
uses nix::unistd::fsync() before its rename, might want to be consistent
here for crash safety (albeit it won't help with peformance, but at least
would be consistent - your call).

> +    if local_manifest_file_fp.is_some() && crypt_config.is_none() {
> +        bail!("local unencrypted snapshot detected, refuse to sync without source decryption");
> +    }

nit: the error message is slightly misleading - the local snapshot isn't
"unencrypted" per se, it was previously *decrypted*. Something like "local
snapshot was previously decrypted but no matching decryption key is
configured" would probably better describe the situation for the user's POV.




  reply	other threads:[~2026-04-19 21:08 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-14 12:58 [PATCH proxmox{,-backup} v3 00/30] fix #7251: implement server side encryption support for push sync jobs Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox v3 01/30] pbs-api-types: define en-/decryption key type and schema Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox v3 02/30] pbs-api-types: sync job: add optional cryptographic keys to config Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox-backup v3 03/30] sync: push: use tracing macros instead of log Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox-backup v3 04/30] datastore: blob: implement async reader for data blobs Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox-backup v3 05/30] datastore: manifest: add helper for change detection fingerprint Christian Ebner
2026-04-14 12:58 ` [PATCH proxmox-backup v3 06/30] pbs-key-config: introduce store_with() for KeyConfig Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 07/30] pbs-config: implement encryption key config handling Christian Ebner
2026-04-14 14:32   ` Michael Köppl
2026-04-15  6:48     ` Christian Ebner
2026-04-15  8:03       ` Daniel Kral
2026-04-15  8:21         ` Christian Ebner
2026-04-15  8:06       ` Thomas Lamprecht
2026-04-19 20:41   ` Thomas Lamprecht
2026-04-20  7:22     ` Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 08/30] pbs-config: acls: add 'encryption-keys' as valid 'system' subpath Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 09/30] ui: expose 'encryption-keys' as acl subpath for 'system' Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 10/30] sync: add helper to check encryption key acls and load key Christian Ebner
2026-04-19 20:41   ` Thomas Lamprecht
2026-04-14 12:59 ` [PATCH proxmox-backup v3 11/30] api: config: add endpoints for encryption key manipulation Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 12/30] api: config: check sync owner has access to en-/decryption keys Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 13/30] api: config: allow encryption key manipulation for sync job Christian Ebner
2026-04-19 20:41   ` Thomas Lamprecht
2026-04-20  8:21     ` Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 14/30] sync: push: rewrite manifest instead of pushing pre-existing one Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 15/30] api: push sync: expose optional encryption key for push sync Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 16/30] sync: push: optionally encrypt data blob on upload Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 17/30] sync: push: optionally encrypt client log on upload if key is given Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 18/30] sync: push: add helper for loading known chunks from previous snapshot Christian Ebner
2026-04-19 20:41   ` Thomas Lamprecht
2026-04-20  8:59     ` Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 19/30] fix #7251: api: push: encrypt snapshots using configured encryption key Christian Ebner
2026-04-15 14:49   ` Michael Köppl
2026-04-15 15:25     ` Christian Ebner
2026-04-19 20:41   ` Thomas Lamprecht
2026-04-20  9:46     ` Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 20/30] ui: define and expose encryption key management menu item and windows Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 21/30] ui: expose assigning encryption key to sync jobs Christian Ebner
2026-04-15 14:49   ` Michael Köppl
2026-04-15 15:20     ` Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 22/30] sync: pull: load encryption key if given in job config Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 23/30] sync: expand source chunk reader trait by crypt config Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 24/30] sync: pull: introduce and use decrypt index writer if " Christian Ebner
2026-04-19 20:41   ` Thomas Lamprecht
2026-04-14 12:59 ` [PATCH proxmox-backup v3 25/30] sync: pull: extend encountered chunk by optional decrypted digest Christian Ebner
2026-04-19 20:41   ` Thomas Lamprecht
2026-04-14 12:59 ` [PATCH proxmox-backup v3 26/30] sync: pull: decrypt blob files on pull if encryption key is configured Christian Ebner
2026-04-19 20:41   ` Thomas Lamprecht
2026-04-14 12:59 ` [PATCH proxmox-backup v3 27/30] sync: pull: decrypt chunks and rewrite index file for matching key Christian Ebner
2026-04-14 12:59 ` [PATCH proxmox-backup v3 28/30] sync: pull: decrypt snapshots with matching encryption key fingerprint Christian Ebner
2026-04-19 20:41   ` Thomas Lamprecht [this message]
2026-04-14 12:59 ` [PATCH proxmox-backup v3 29/30] api: encryption keys: allow to toggle the archived state for keys Christian Ebner
2026-04-19 20:42   ` Thomas Lamprecht
2026-04-14 12:59 ` [PATCH proxmox-backup v3 30/30] docs: add section describing server side encryption for sync jobs Christian Ebner
2026-04-19 20:42   ` Thomas Lamprecht
2026-04-19 20:41 ` [PATCH proxmox{,-backup} v3 00/30] fix #7251: implement server side encryption support for push " Thomas Lamprecht
2026-04-20  6:48   ` Christian Ebner
2026-04-20 16:17 ` Christian Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260419210610.3915597-10-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal