public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup pxar 0/2] fix fuse mount performance for split archives
@ 2024-06-10 15:11 Christian Ebner
  2024-06-10 15:11 ` [pbs-devel] [PATCH pxar 1/2] decoder: move payload header check for split input Christian Ebner
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Christian Ebner @ 2024-06-10 15:11 UTC (permalink / raw)
  To: pbs-devel

Fuse mounts for split pxar archives currently greatly suffer from the
consistency check between metadata and payload data archives, as
these happen already during decoding of the payload reference entry
in the metadata archive. By moving this check to the content reader
instantiation, the performance can be improved significantly, as now
the payload data chunks only need to be fetched and decoded when
actually accessing the file payloads.

pxar:

Christian Ebner (1):
  decoder: move payload header check for split input

 src/decoder/aio.rs  |  4 +--
 src/decoder/mod.rs  | 60 ++++++++++++++++++++++++++-------------------
 src/decoder/sync.rs |  5 ++--
 3 files changed, 40 insertions(+), 29 deletions(-)

proxmox-backup:

Christian Ebner (1):
  client: pxar: fix fuse mount performance for split archives

 pbs-client/src/pxar/extract.rs | 33 ++++++++++++++++++---------------
 src/api2/tape/restore.rs       |  2 +-
 2 files changed, 19 insertions(+), 16 deletions(-)

-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [pbs-devel] [PATCH pxar 1/2] decoder: move payload header check for split input
  2024-06-10 15:11 [pbs-devel] [PATCH proxmox-backup pxar 0/2] fix fuse mount performance for split archives Christian Ebner
@ 2024-06-10 15:11 ` Christian Ebner
  2024-06-11  8:29   ` Fabian Grünbichler
  2024-06-10 15:11 ` [pbs-devel] [PATCH proxmox-backup 2/2] client: pxar: fix fuse mount performance for split archives Christian Ebner
  2024-06-11 13:31 ` [pbs-devel] [PATCH proxmox-backup pxar 0/2] " Christian Ebner
  2 siblings, 1 reply; 6+ messages in thread
From: Christian Ebner @ 2024-06-10 15:11 UTC (permalink / raw)
  To: pbs-devel

The payload entries in the payload output for split pxar archives are
separated by payload headers, which allow to perform consistency
checks for the payload references encoded in the metadata archive.

Currently, this consistency check is performed right after reading the
entry in the metadata archive, which however has the downside that the
payload has to be fetched and decoded just for this consistency check.
This greatly impacts performance when accessing a metadata archive
with attached payload input reader, e.g. in the fuse implementation to
mount pxar archives, being especially severe when accessed over the
network in combination with a remote chunk reader as the Proxmox
Backup Server does.

Therefore, move this check to the contents reader instantiation
instead and add an additional flag to the decoder's `InPayload` state.

Getting the decoder now needs to be async and the method must return
an error when the check fails.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/decoder/aio.rs  |  4 +--
 src/decoder/mod.rs  | 60 ++++++++++++++++++++++++++-------------------
 src/decoder/sync.rs |  5 ++--
 3 files changed, 40 insertions(+), 29 deletions(-)

diff --git a/src/decoder/aio.rs b/src/decoder/aio.rs
index 3f9881d..19e7023 100644
--- a/src/decoder/aio.rs
+++ b/src/decoder/aio.rs
@@ -60,8 +60,8 @@ impl<T: SeqRead> Decoder<T> {
     }
 
     /// Get a reader for the contents of the current entry, if the entry has contents.
-    pub fn contents(&mut self) -> Option<Contents<T>> {
-        self.inner.content_reader()
+    pub async fn contents(&mut self) -> io::Result<Option<Contents<T>>> {
+        self.inner.content_reader().await
     }
 
     /// Get the size of the current contents, if the entry has contents.
diff --git a/src/decoder/mod.rs b/src/decoder/mod.rs
index 46a21b8..8f92964 100644
--- a/src/decoder/mod.rs
+++ b/src/decoder/mod.rs
@@ -182,6 +182,7 @@ enum State {
     InPayload {
         offset: u64,
         size: u64,
+        header_checked: bool,
     },
 
     /// file entries with no data (fifo, socket)
@@ -296,8 +297,12 @@ impl<I: SeqRead> DecoderImpl<I> {
                     // hierarchy and parse the next PXAR_FILENAME or the PXAR_GOODBYE:
                     self.read_next_item().await?;
                 }
-                State::InPayload { offset, .. } => {
+                State::InPayload { offset, header_checked, .. } => {
                     if self.input.payload().is_some() {
+                        if !header_checked {
+                            // header is only checked if payload has been accessed
+                            self.payload_consumed += size_of::<Header>() as u64;
+                        }
                         // Update consumed payload as given by the offset referenced by the content reader
                         self.payload_consumed += offset;
                     } else {
@@ -370,19 +375,39 @@ impl<I: SeqRead> DecoderImpl<I> {
         }
     }
 
-    pub fn content_reader(&mut self) -> Option<Contents<I>> {
-        if let State::InPayload { offset, size } = &mut self.state {
-            if self.input.payload().is_some() {
-                Some(Contents::new(
+    pub async fn content_reader(&mut self) -> Result<Option<Contents<I>>, io::Error> {
+        if let State::InPayload { offset, size, header_checked } = &mut self.state {
+            if let Some(payload_input) = self.input.payload_mut() {
+                if !*header_checked {
+                    let header: Header = seq_read_entry(payload_input).await?;
+                    if header.htype != format::PXAR_PAYLOAD {
+                        io_bail!(
+                            "unexpected header in payload input: expected {} , got {header}",
+                            format::PXAR_PAYLOAD,
+                        );
+                    }
+                    self.payload_consumed += size_of::<Header>() as u64;
+
+                    if header.content_size() != *size {
+                        io_bail!(
+                            "encountered payload size mismatch: got {size}, expected {}",
+                            header.content_size(),
+                        );
+                    }
+
+                    *header_checked = true;
+                }
+
+                Ok(Some(Contents::new(
                     self.input.payload_mut().unwrap(),
                     offset,
                     *size,
-                ))
+                )))
             } else {
-                Some(Contents::new(self.input.archive_mut(), offset, *size))
+                Ok(Some(Contents::new(self.input.archive_mut(), offset, *size)))
             }
         } else {
-            None
+            Ok(None)
         }
     }
 
@@ -621,6 +646,7 @@ impl<I: SeqRead> DecoderImpl<I> {
                 };
                 self.state = State::InPayload {
                     offset: 0,
+                    header_checked: false,
                     size: self.current_header.content_size(),
                 };
                 return Ok(ItemResult::Entry);
@@ -652,23 +678,6 @@ impl<I: SeqRead> DecoderImpl<I> {
                         let end = start + payload_ref.size + size_of::<Header>() as u64;
                         payload_input.update_range(start..end);
                     }
-
-                    let header: Header = seq_read_entry(payload_input).await?;
-                    if header.htype != format::PXAR_PAYLOAD {
-                        io_bail!(
-                            "unexpected header in payload input: expected {} , got {header}",
-                            format::PXAR_PAYLOAD,
-                        );
-                    }
-                    self.payload_consumed += size_of::<Header>() as u64;
-
-                    if header.content_size() != payload_ref.size {
-                        io_bail!(
-                            "encountered payload size mismatch: got {}, expected {}",
-                            payload_ref.size,
-                            header.content_size(),
-                        );
-                    }
                 }
 
                 self.entry.kind = EntryKind::File {
@@ -678,6 +687,7 @@ impl<I: SeqRead> DecoderImpl<I> {
                 };
                 self.state = State::InPayload {
                     offset: 0,
+                    header_checked: false,
                     size: payload_ref.size,
                 };
                 return Ok(ItemResult::Entry);
diff --git a/src/decoder/sync.rs b/src/decoder/sync.rs
index 8779f87..1116fe8 100644
--- a/src/decoder/sync.rs
+++ b/src/decoder/sync.rs
@@ -77,8 +77,9 @@ impl<T: SeqRead> Decoder<T> {
     }
 
     /// Get a reader for the contents of the current entry, if the entry has contents.
-    pub fn contents(&mut self) -> Option<Contents<T>> {
-        self.inner.content_reader().map(|inner| Contents { inner })
+    pub fn contents(&mut self) -> io::Result<Option<Contents<T>>> {
+        let content_reader = poll_result_once(self.inner.content_reader())?;
+        Ok(content_reader.map(|inner| Contents { inner }))
     }
 
     /// Get the size of the current contents, if the entry has contents.
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [pbs-devel] [PATCH proxmox-backup 2/2] client: pxar: fix fuse mount performance for split archives
  2024-06-10 15:11 [pbs-devel] [PATCH proxmox-backup pxar 0/2] fix fuse mount performance for split archives Christian Ebner
  2024-06-10 15:11 ` [pbs-devel] [PATCH pxar 1/2] decoder: move payload header check for split input Christian Ebner
@ 2024-06-10 15:11 ` Christian Ebner
  2024-06-11  8:29   ` Fabian Grünbichler
  2024-06-11 13:31 ` [pbs-devel] [PATCH proxmox-backup pxar 0/2] " Christian Ebner
  2 siblings, 1 reply; 6+ messages in thread
From: Christian Ebner @ 2024-06-10 15:11 UTC (permalink / raw)
  To: pbs-devel

Adapt to the decoder method changes introduced in the pxar library,
which were introduced in order to move the consistency check for
metadata and payload data archives.

The new location of the check allows to access the pxar archive via
a `Split` variant reader instance, without penalization when just
accessing the metadata, not reading any payload data.

This greatly improves performance when accessing fuse mounted
archives.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-client/src/pxar/extract.rs | 33 ++++++++++++++++++---------------
 src/api2/tape/restore.rs       |  2 +-
 2 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/pbs-client/src/pxar/extract.rs b/pbs-client/src/pxar/extract.rs
index 38d4ca89a..b3545650e 100644
--- a/pbs-client/src/pxar/extract.rs
+++ b/pbs-client/src/pxar/extract.rs
@@ -374,7 +374,10 @@ where
                 }
             }
             (true, EntryKind::File { size, .. }) => {
-                let contents = self.decoder.contents();
+                let contents = match self.decoder.contents() {
+                    Ok(contents) => contents,
+                    Err(err) => return Some(Err(err.into())),
+                };
 
                 if let Some(mut contents) = contents {
                     self.extractor.extract_file(
@@ -872,7 +875,8 @@ where
             match entry.kind() {
                 EntryKind::File { .. } => {
                     let size = decoder.content_size().unwrap_or(0);
-                    tar_add_file(&mut tarencoder, decoder.contents(), size, metadata, path).await?
+                    let contents = decoder.contents().await?;
+                    tar_add_file(&mut tarencoder, contents, size, metadata, path).await?
                 }
                 EntryKind::Hardlink(link) => {
                     if !link.data.is_empty() {
@@ -894,14 +898,9 @@ where
                                     path
                                 } else {
                                     let size = decoder.content_size().unwrap_or(0);
-                                    tar_add_file(
-                                        &mut tarencoder,
-                                        decoder.contents(),
-                                        size,
-                                        metadata,
-                                        path,
-                                    )
-                                    .await?;
+                                    let contents = decoder.contents().await?;
+                                    tar_add_file(&mut tarencoder, contents, size, metadata, path)
+                                        .await?;
                                     hardlinks.insert(realpath.to_owned(), path.to_owned());
                                     continue;
                                 }
@@ -1038,7 +1037,8 @@ where
                         metadata.stat.mode as u16,
                         true,
                     );
-                    zip.add_entry(entry, decoder.contents())
+                    let contents = decoder.contents().await?;
+                    zip.add_entry(entry, contents)
                         .await
                         .context("could not send file entry")?;
                 }
@@ -1056,7 +1056,8 @@ where
                         metadata.stat.mode as u16,
                         true,
                     );
-                    zip.add_entry(entry, decoder.contents())
+                    let contents = decoder.contents().await?;
+                    zip.add_entry(entry, contents)
                         .await
                         .context("could not send file entry")?;
                 }
@@ -1279,14 +1280,16 @@ where
                         .with_context(|| format!("error at entry {file_name_os:?}"))?;
                 }
                 EntryKind::File { size, .. } => {
+                    let mut contents = decoder
+                        .contents()
+                        .await?
+                        .context("found regular file entry without contents in archive")?;
                     extractor
                         .async_extract_file(
                             &file_name,
                             metadata,
                             *size,
-                            &mut decoder
-                                .contents()
-                                .context("found regular file entry without contents in archive")?,
+                            &mut contents,
                             extractor.overwrite_flags.contains(OverwriteFlags::FILE),
                         )
                         .await?
diff --git a/src/api2/tape/restore.rs b/src/api2/tape/restore.rs
index 382909647..2b2025f6d 100644
--- a/src/api2/tape/restore.rs
+++ b/src/api2/tape/restore.rs
@@ -1748,7 +1748,7 @@ fn try_restore_snapshot_archive<R: pxar::decoder::SeqRead>(
         }
 
         let filename = entry.file_name();
-        let mut contents = match decoder.contents() {
+        let mut contents = match decoder.contents()? {
             None => bail!("missing file content"),
             Some(contents) => contents,
         };
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [pbs-devel] [PATCH pxar 1/2] decoder: move payload header check for split input
  2024-06-10 15:11 ` [pbs-devel] [PATCH pxar 1/2] decoder: move payload header check for split input Christian Ebner
@ 2024-06-11  8:29   ` Fabian Grünbichler
  0 siblings, 0 replies; 6+ messages in thread
From: Fabian Grünbichler @ 2024-06-11  8:29 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion

On June 10, 2024 5:11 pm, Christian Ebner wrote:
> The payload entries in the payload output for split pxar archives are
> separated by payload headers, which allow to perform consistency
> checks for the payload references encoded in the metadata archive.
> 
> Currently, this consistency check is performed right after reading the
> entry in the metadata archive, which however has the downside that the
> payload has to be fetched and decoded just for this consistency check.
> This greatly impacts performance when accessing a metadata archive
> with attached payload input reader, e.g. in the fuse implementation to
> mount pxar archives, being especially severe when accessed over the
> network in combination with a remote chunk reader as the Proxmox
> Backup Server does.
> 
> Therefore, move this check to the contents reader instantiation
> instead and add an additional flag to the decoder's `InPayload` state.
> 
> Getting the decoder now needs to be async and the method must return
> an error when the check fails.

as discussed off-list - the accessor used by FUSE uses the content-range
based (unsafe) way to access file contents, and that one is now lacking
the payload marker checks. they can probably be added to the entry's
content_range method that returns the ranges to be passed to the unsafe
interface (and if any other caller uses that unsafe interface with other
ranges, any brokenness is on them anyway ;)).

> 
> Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
> ---
>  src/decoder/aio.rs  |  4 +--
>  src/decoder/mod.rs  | 60 ++++++++++++++++++++++++++-------------------
>  src/decoder/sync.rs |  5 ++--
>  3 files changed, 40 insertions(+), 29 deletions(-)
> 
> diff --git a/src/decoder/aio.rs b/src/decoder/aio.rs
> index 3f9881d..19e7023 100644
> --- a/src/decoder/aio.rs
> +++ b/src/decoder/aio.rs
> @@ -60,8 +60,8 @@ impl<T: SeqRead> Decoder<T> {
>      }
>  
>      /// Get a reader for the contents of the current entry, if the entry has contents.
> -    pub fn contents(&mut self) -> Option<Contents<T>> {
> -        self.inner.content_reader()
> +    pub async fn contents(&mut self) -> io::Result<Option<Contents<T>>> {
> +        self.inner.content_reader().await
>      }
>  
>      /// Get the size of the current contents, if the entry has contents.
> diff --git a/src/decoder/mod.rs b/src/decoder/mod.rs
> index 46a21b8..8f92964 100644
> --- a/src/decoder/mod.rs
> +++ b/src/decoder/mod.rs
> @@ -182,6 +182,7 @@ enum State {
>      InPayload {
>          offset: u64,
>          size: u64,
> +        header_checked: bool,
>      },
>  
>      /// file entries with no data (fifo, socket)
> @@ -296,8 +297,12 @@ impl<I: SeqRead> DecoderImpl<I> {
>                      // hierarchy and parse the next PXAR_FILENAME or the PXAR_GOODBYE:
>                      self.read_next_item().await?;
>                  }
> -                State::InPayload { offset, .. } => {
> +                State::InPayload { offset, header_checked, .. } => {
>                      if self.input.payload().is_some() {
> +                        if !header_checked {
> +                            // header is only checked if payload has been accessed
> +                            self.payload_consumed += size_of::<Header>() as u64;
> +                        }
>                          // Update consumed payload as given by the offset referenced by the content reader
>                          self.payload_consumed += offset;
>                      } else {
> @@ -370,19 +375,39 @@ impl<I: SeqRead> DecoderImpl<I> {
>          }
>      }
>  
> -    pub fn content_reader(&mut self) -> Option<Contents<I>> {
> -        if let State::InPayload { offset, size } = &mut self.state {
> -            if self.input.payload().is_some() {
> -                Some(Contents::new(
> +    pub async fn content_reader(&mut self) -> Result<Option<Contents<I>>, io::Error> {
> +        if let State::InPayload { offset, size, header_checked } = &mut self.state {
> +            if let Some(payload_input) = self.input.payload_mut() {
> +                if !*header_checked {
> +                    let header: Header = seq_read_entry(payload_input).await?;
> +                    if header.htype != format::PXAR_PAYLOAD {
> +                        io_bail!(
> +                            "unexpected header in payload input: expected {} , got {header}",
> +                            format::PXAR_PAYLOAD,
> +                        );
> +                    }
> +                    self.payload_consumed += size_of::<Header>() as u64;
> +
> +                    if header.content_size() != *size {
> +                        io_bail!(
> +                            "encountered payload size mismatch: got {size}, expected {}",
> +                            header.content_size(),
> +                        );
> +                    }
> +
> +                    *header_checked = true;
> +                }
> +
> +                Ok(Some(Contents::new(
>                      self.input.payload_mut().unwrap(),
>                      offset,
>                      *size,
> -                ))
> +                )))
>              } else {
> -                Some(Contents::new(self.input.archive_mut(), offset, *size))
> +                Ok(Some(Contents::new(self.input.archive_mut(), offset, *size)))
>              }
>          } else {
> -            None
> +            Ok(None)
>          }
>      }
>  
> @@ -621,6 +646,7 @@ impl<I: SeqRead> DecoderImpl<I> {
>                  };
>                  self.state = State::InPayload {
>                      offset: 0,
> +                    header_checked: false,
>                      size: self.current_header.content_size(),
>                  };
>                  return Ok(ItemResult::Entry);
> @@ -652,23 +678,6 @@ impl<I: SeqRead> DecoderImpl<I> {
>                          let end = start + payload_ref.size + size_of::<Header>() as u64;
>                          payload_input.update_range(start..end);
>                      }
> -
> -                    let header: Header = seq_read_entry(payload_input).await?;
> -                    if header.htype != format::PXAR_PAYLOAD {
> -                        io_bail!(
> -                            "unexpected header in payload input: expected {} , got {header}",
> -                            format::PXAR_PAYLOAD,
> -                        );
> -                    }
> -                    self.payload_consumed += size_of::<Header>() as u64;
> -
> -                    if header.content_size() != payload_ref.size {
> -                        io_bail!(
> -                            "encountered payload size mismatch: got {}, expected {}",
> -                            payload_ref.size,
> -                            header.content_size(),
> -                        );
> -                    }
>                  }
>  
>                  self.entry.kind = EntryKind::File {
> @@ -678,6 +687,7 @@ impl<I: SeqRead> DecoderImpl<I> {
>                  };
>                  self.state = State::InPayload {
>                      offset: 0,
> +                    header_checked: false,
>                      size: payload_ref.size,
>                  };
>                  return Ok(ItemResult::Entry);
> diff --git a/src/decoder/sync.rs b/src/decoder/sync.rs
> index 8779f87..1116fe8 100644
> --- a/src/decoder/sync.rs
> +++ b/src/decoder/sync.rs
> @@ -77,8 +77,9 @@ impl<T: SeqRead> Decoder<T> {
>      }
>  
>      /// Get a reader for the contents of the current entry, if the entry has contents.
> -    pub fn contents(&mut self) -> Option<Contents<T>> {
> -        self.inner.content_reader().map(|inner| Contents { inner })
> +    pub fn contents(&mut self) -> io::Result<Option<Contents<T>>> {
> +        let content_reader = poll_result_once(self.inner.content_reader())?;
> +        Ok(content_reader.map(|inner| Contents { inner }))
>      }
>  
>      /// Get the size of the current contents, if the entry has contents.
> -- 
> 2.39.2
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
> 


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup 2/2] client: pxar: fix fuse mount performance for split archives
  2024-06-10 15:11 ` [pbs-devel] [PATCH proxmox-backup 2/2] client: pxar: fix fuse mount performance for split archives Christian Ebner
@ 2024-06-11  8:29   ` Fabian Grünbichler
  0 siblings, 0 replies; 6+ messages in thread
From: Fabian Grünbichler @ 2024-06-11  8:29 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion

On June 10, 2024 5:11 pm, Christian Ebner wrote:
> Adapt to the decoder method changes introduced in the pxar library,
> which were introduced in order to move the consistency check for
> metadata and payload data archives.
> 
> The new location of the check allows to access the pxar archive via
> a `Split` variant reader instance, without penalization when just
> accessing the metadata, not reading any payload data.
> 
> This greatly improves performance when accessing fuse mounted
> archives.
> 
> Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
> ---
>  pbs-client/src/pxar/extract.rs | 33 ++++++++++++++++++---------------
>  src/api2/tape/restore.rs       |  2 +-
>  2 files changed, 19 insertions(+), 16 deletions(-)
> 
> diff --git a/pbs-client/src/pxar/extract.rs b/pbs-client/src/pxar/extract.rs
> index 38d4ca89a..b3545650e 100644
> --- a/pbs-client/src/pxar/extract.rs
> +++ b/pbs-client/src/pxar/extract.rs
> @@ -374,7 +374,10 @@ where
>                  }
>              }
>              (true, EntryKind::File { size, .. }) => {
> -                let contents = self.decoder.contents();
> +                let contents = match self.decoder.contents() {
> +                    Ok(contents) => contents,
> +                    Err(err) => return Some(Err(err.into())),
> +                };

the error handling below adds context which this part here is lacking.

I think the whole match arm would be easier to understand, while
preserving the context, if written as:

            (true, EntryKind::File { size, .. }) => match self.decoder.contents() {
                Ok(Some(mut contents)) => self.extractor.extract_file(
                    &file_name,
                    metadata,
                    *size,
                    &mut contents,
                    self.extractor
                        .overwrite_flags
                        .contains(OverwriteFlags::FILE),
                ),
                Ok(None) => Err(format_err!(
                    "found regular file entry without contents in archive"
                )),
                Err(err) => Err(err.into()),
            }
            .context(PxarExtractContext::ExtractFile),


>  
>                  if let Some(mut contents) = contents {
>                      self.extractor.extract_file(
> @@ -872,7 +875,8 @@ where
>              match entry.kind() {
>                  EntryKind::File { .. } => {
>                      let size = decoder.content_size().unwrap_or(0);
> -                    tar_add_file(&mut tarencoder, decoder.contents(), size, metadata, path).await?
> +                    let contents = decoder.contents().await?;
> +                    tar_add_file(&mut tarencoder, contents, size, metadata, path).await?
>                  }
>                  EntryKind::Hardlink(link) => {
>                      if !link.data.is_empty() {
> @@ -894,14 +898,9 @@ where
>                                      path
>                                  } else {
>                                      let size = decoder.content_size().unwrap_or(0);
> -                                    tar_add_file(
> -                                        &mut tarencoder,
> -                                        decoder.contents(),
> -                                        size,
> -                                        metadata,
> -                                        path,
> -                                    )
> -                                    .await?;
> +                                    let contents = decoder.contents().await?;
> +                                    tar_add_file(&mut tarencoder, contents, size, metadata, path)
> +                                        .await?;
>                                      hardlinks.insert(realpath.to_owned(), path.to_owned());
>                                      continue;
>                                  }
> @@ -1038,7 +1037,8 @@ where
>                          metadata.stat.mode as u16,
>                          true,
>                      );
> -                    zip.add_entry(entry, decoder.contents())
> +                    let contents = decoder.contents().await?;
> +                    zip.add_entry(entry, contents)
>                          .await
>                          .context("could not send file entry")?;
>                  }
> @@ -1056,7 +1056,8 @@ where
>                          metadata.stat.mode as u16,
>                          true,
>                      );
> -                    zip.add_entry(entry, decoder.contents())
> +                    let contents = decoder.contents().await?;
> +                    zip.add_entry(entry, contents)
>                          .await
>                          .context("could not send file entry")?;
>                  }
> @@ -1279,14 +1280,16 @@ where
>                          .with_context(|| format!("error at entry {file_name_os:?}"))?;
>                  }
>                  EntryKind::File { size, .. } => {
> +                    let mut contents = decoder
> +                        .contents()
> +                        .await?
> +                        .context("found regular file entry without contents in archive")?;
>                      extractor
>                          .async_extract_file(
>                              &file_name,
>                              metadata,
>                              *size,
> -                            &mut decoder
> -                                .contents()
> -                                .context("found regular file entry without contents in archive")?,
> +                            &mut contents,
>                              extractor.overwrite_flags.contains(OverwriteFlags::FILE),
>                          )
>                          .await?
> diff --git a/src/api2/tape/restore.rs b/src/api2/tape/restore.rs
> index 382909647..2b2025f6d 100644
> --- a/src/api2/tape/restore.rs
> +++ b/src/api2/tape/restore.rs
> @@ -1748,7 +1748,7 @@ fn try_restore_snapshot_archive<R: pxar::decoder::SeqRead>(
>          }
>  
>          let filename = entry.file_name();
> -        let mut contents = match decoder.contents() {
> +        let mut contents = match decoder.contents()? {
>              None => bail!("missing file content"),
>              Some(contents) => contents,
>          };
> -- 
> 2.39.2
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
> 


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup pxar 0/2] fix fuse mount performance for split archives
  2024-06-10 15:11 [pbs-devel] [PATCH proxmox-backup pxar 0/2] fix fuse mount performance for split archives Christian Ebner
  2024-06-10 15:11 ` [pbs-devel] [PATCH pxar 1/2] decoder: move payload header check for split input Christian Ebner
  2024-06-10 15:11 ` [pbs-devel] [PATCH proxmox-backup 2/2] client: pxar: fix fuse mount performance for split archives Christian Ebner
@ 2024-06-11 13:31 ` Christian Ebner
  2 siblings, 0 replies; 6+ messages in thread
From: Christian Ebner @ 2024-06-11 13:31 UTC (permalink / raw)
  To: pbs-devel

Superseded by version 2:
https://lists.proxmox.com/pipermail/pbs-devel/2024-June/009809.html


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2024-06-11 13:31 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-06-10 15:11 [pbs-devel] [PATCH proxmox-backup pxar 0/2] fix fuse mount performance for split archives Christian Ebner
2024-06-10 15:11 ` [pbs-devel] [PATCH pxar 1/2] decoder: move payload header check for split input Christian Ebner
2024-06-11  8:29   ` Fabian Grünbichler
2024-06-10 15:11 ` [pbs-devel] [PATCH proxmox-backup 2/2] client: pxar: fix fuse mount performance for split archives Christian Ebner
2024-06-11  8:29   ` Fabian Grünbichler
2024-06-11 13:31 ` [pbs-devel] [PATCH proxmox-backup pxar 0/2] " Christian Ebner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal