public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Robert Obkircher <r.obkircher@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [PATCH v5 proxmox-backup 11/16] datastore: combine public FixedIndexWriter methods into add_chunk.
Date: Fri, 30 Jan 2026 17:45:35 +0100	[thread overview]
Message-ID: <20260130164552.281581-12-r.obkircher@proxmox.com> (raw)
In-Reply-To: <20260130164552.281581-1-r.obkircher@proxmox.com>

These operations were always performed together anyway, so it makes
sense to simplify the public api.

Signed-off-by: Robert Obkircher <r.obkircher@proxmox.com>
---
 pbs-datastore/src/fixed_index.rs | 24 +++++++++++++++++++++---
 src/api2/backup/environment.rs   |  6 +-----
 2 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/pbs-datastore/src/fixed_index.rs b/pbs-datastore/src/fixed_index.rs
index c4fe899d..f50f4517 100644
--- a/pbs-datastore/src/fixed_index.rs
+++ b/pbs-datastore/src/fixed_index.rs
@@ -380,7 +380,7 @@ impl FixedIndexWriter {
     /// The size also becomes fixed as soon as it is no longer divisible
     /// by the block size, to ensure that only the last block can be
     /// smaller.
-    pub fn grow_to_size(&mut self, requested_size: usize) -> Result<(), Error> {
+    fn grow_to_size(&mut self, requested_size: usize) -> Result<(), Error> {
         if self.size < requested_size {
             if !self.growable_size {
                 bail!("refusing to resize from {} to {requested_size}", self.size);
@@ -467,7 +467,7 @@ impl FixedIndexWriter {
         Ok(index_csum)
     }
 
-    pub fn check_chunk_alignment(&self, offset: usize, chunk_len: usize) -> Result<usize, Error> {
+    fn check_chunk_alignment(&self, offset: usize, chunk_len: usize) -> Result<usize, Error> {
         if offset < chunk_len {
             bail!("got chunk with small offset ({} < {}", offset, chunk_len);
         }
@@ -497,7 +497,7 @@ impl FixedIndexWriter {
         Ok(pos / self.chunk_size)
     }
 
-    pub fn add_digest(&mut self, index: usize, digest: &[u8; 32]) -> Result<(), Error> {
+    fn add_digest(&mut self, index: usize, digest: &[u8; 32]) -> Result<(), Error> {
         if index >= self.index_length {
             bail!(
                 "add digest failed - index out of range ({} >= {})",
@@ -519,6 +519,24 @@ impl FixedIndexWriter {
         Ok(())
     }
 
+    /// Write the digest of a chunk into this index file.
+    ///
+    /// The `start` and `size` parameters encode the range of
+    /// content that is backed up. It is verified that `start` is
+    /// aligned and that only the last chunk may be smaller.
+    ///
+    /// If this writer has been created without a fixed size, the
+    /// index capacity and content size are increased automatically
+    /// until an incomplete chunk is encountered.
+    pub fn add_chunk(&mut self, start: u64, size: u32, digest: &[u8; 32]) -> Result<(), Error> {
+        let Some(end) = start.checked_add(size.into()) else {
+            bail!("add_chunk: start and size are too large: {start}+{size}");
+        };
+        self.grow_to_size(end as usize)?;
+        let idx = self.check_chunk_alignment(end as usize, size as usize)?;
+        self.add_digest(idx, digest)
+    }
+
     pub fn clone_data_from(&mut self, reader: &FixedIndexReader) -> Result<(), Error> {
         if self.growable_size {
             bail!("reusing the index is only supported with known input size");
diff --git a/src/api2/backup/environment.rs b/src/api2/backup/environment.rs
index dc4f9bf5..04c5bf84 100644
--- a/src/api2/backup/environment.rs
+++ b/src/api2/backup/environment.rs
@@ -442,14 +442,10 @@ impl BackupEnvironment {
             );
         }
 
-        let end = (offset as usize) + (size as usize);
-        data.index.grow_to_size(end)?;
-        let idx = data.index.check_chunk_alignment(end, size as usize)?;
+        data.index.add_chunk(offset, size, digest)?;
 
         data.chunk_count += 1;
 
-        data.index.add_digest(idx, digest)?;
-
         Ok(())
     }
 
-- 
2.47.3





  parent reply	other threads:[~2026-01-30 16:46 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-30 16:45 [PATCH v5 proxmox-backup 00/16] fix: #3847 pipe from STDIN to proxmox-backup-client Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 01/16] datastore: remove Arc<ChunkStore> from FixedIndexWriter Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 02/16] datastore: remove Arc<ChunkStore> from DynamicIndexWriter Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 03/16] datastore: add TempTestDir that is automatically deleted on drop Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 04/16] datastore: use temporary directory for chunk store test Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 05/16] datastore: support writing fidx files of unknown size Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 06/16] datastore: test FixedIndexWriter Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 07/16] api: backup: make fixed index file size optional Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 08/16] api: verify fixed index writer size on close Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 09/16] fix #3847: client: support fifo pipe inputs for images Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 10/16] client: treat minus sign as stdin Robert Obkircher
2026-01-30 16:45 ` Robert Obkircher [this message]
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 12/16] datastore: use u64 instead of usize for fidx writer content size Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 13/16] datastore: compute fidx file size with overflow checks Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 14/16] datastore: support writing fidx files on systems with larger page size Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 15/16] datastore: FixedIndexWriter: switch public chunk_size to u32 Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 16/16] datastore: FixedIndexWriter: switch internal " Robert Obkircher

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260130164552.281581-12-r.obkircher@proxmox.com \
    --to=r.obkircher@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal