[pbs-devel] [PATCH proxmox-backup] backup-client: mount: fix read of larger files

[pbs-devel] [PATCH proxmox-backup] backup-client: mount: fix read of larger files

[Dominik Csapak]

fuse_lowlevel.h says about read:

 Read should send exactly the number of bytes requested except
 on EOF or error, otherwise the rest of the data will be
 substituted with zeroes.

but we simply forwarded the bytes we got from 'read_at'. The result was
that larger files were truncated as soon as read_at returned not the
exact number of bytes requested.

To fix that, loop over 'read_at' until our buffer is full, or we read
0 bytes, indicating EOF.

reported in the forum:
https://forum.proxmox.com/threads/proxmox-backup-client-mounting-a-pxar-archive-gives-truncated-files.114447/

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
tested with some larger files (>100MiB), before the patch, a 'cmp' with the
original would always fail, after it worked like expected

 pbs-client/src/pxar/fuse.rs | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/pbs-client/src/pxar/fuse.rs b/pbs-client/src/pxar/fuse.rs
index e9e1b6f3..f2ce6a43 100644
--- a/pbs-client/src/pxar/fuse.rs
+++ b/pbs-client/src/pxar/fuse.rs
@@ -566,8 +566,17 @@ impl SessionImpl {
         let file = self.get_lookup(inode)?;
         let content = self.open_content(&file)?;
         let mut buf = vec::undefined(len);
-        let got = content.read_at(&mut buf, offset).await?;
-        buf.truncate(got);
+        let mut pos = 0;
+        loop {
+            let got = content
+                .read_at(&mut buf[pos..], offset + pos as u64)
+                .await?;
+            pos += got;
+            if got == 0 || pos >= len {
+                break;
+            }
+        }
+        buf.truncate(pos);
         Ok(buf)
     }
 
-- 
2.30.2

[pbs-devel] applied: [PATCH proxmox-backup] backup-client: mount: fix read of larger files

[Fabian Grünbichler]

with a comment added and a slight rewording of the commit message, as
triggering the bug doesn't really truncate files, it overwrites parts of 
them with all-zero blocks at different offsets.

On September 2, 2022 9:21 am, Dominik Csapak wrote:
> fuse_lowlevel.h says about read:
> 
>  Read should send exactly the number of bytes requested except
>  on EOF or error, otherwise the rest of the data will be
>  substituted with zeroes.
> 
> but we simply forwarded the bytes we got from 'read_at'. The result was
> that larger files were truncated as soon as read_at returned not the
> exact number of bytes requested.
> 
> To fix that, loop over 'read_at' until our buffer is full, or we read
> 0 bytes, indicating EOF.
> 
> reported in the forum:
> https://forum.proxmox.com/threads/proxmox-backup-client-mounting-a-pxar-archive-gives-truncated-files.114447/
> 
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> tested with some larger files (>100MiB), before the patch, a 'cmp' with the
> original would always fail, after it worked like expected
> 
>  pbs-client/src/pxar/fuse.rs | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/pbs-client/src/pxar/fuse.rs b/pbs-client/src/pxar/fuse.rs
> index e9e1b6f3..f2ce6a43 100644
> --- a/pbs-client/src/pxar/fuse.rs
> +++ b/pbs-client/src/pxar/fuse.rs
> @@ -566,8 +566,17 @@ impl SessionImpl {
>          let file = self.get_lookup(inode)?;
>          let content = self.open_content(&file)?;
>          let mut buf = vec::undefined(len);
> -        let got = content.read_at(&mut buf, offset).await?;
> -        buf.truncate(got);
> +        let mut pos = 0;
> +        loop {
> +            let got = content
> +                .read_at(&mut buf[pos..], offset + pos as u64)
> +                .await?;
> +            pos += got;
> +            if got == 0 || pos >= len {
> +                break;
> +            }
> +        }
> +        buf.truncate(pos);
>          Ok(buf)
>      }
>  
> -- 
> 2.30.2
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
>