public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup v5 1/1] pxar/extract: if possible create files sparesly
Date: Wed, 17 Feb 2021 14:13:22 +0100	[thread overview]
Message-ID: <20210217131322.9129-4-d.csapak@proxmox.com> (raw)
In-Reply-To: <20210217131322.9129-1-d.csapak@proxmox.com>

instead of filling them with zeroes

this fixes an issue where we could not restore a container with large
sparse files in the backup (e.g. a 10GiB sparse file in a container
with a 8GiB disk)

if the last operation of the copy was a seek, we need to truncate
the file to the correct size (seek beyond filesize does not change it)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/pxar/extract.rs | 43 ++++++++++++++++++++++++++++++++++++-------
 1 file changed, 36 insertions(+), 7 deletions(-)

diff --git a/src/pxar/extract.rs b/src/pxar/extract.rs
index b673b4b8..952e2d20 100644
--- a/src/pxar/extract.rs
+++ b/src/pxar/extract.rs
@@ -21,7 +21,10 @@ use pxar::Metadata;
 use pxar::accessor::aio::{Accessor, FileContents, FileEntry};
 
 use proxmox::c_result;
-use proxmox::tools::fs::{create_path, CreateOptions};
+use proxmox::tools::{
+    fs::{create_path, CreateOptions},
+    io::{sparse_copy, sparse_copy_async},
+};
 
 use crate::pxar::dir_stack::PxarDirStack;
 use crate::pxar::metadata;
@@ -411,10 +414,23 @@ impl Extractor {
         )
         .map_err(|err| format_err!("failed to apply initial flags: {}", err))?;
 
-        let extracted = io::copy(&mut *contents, &mut file)
+        let result = sparse_copy(&mut *contents, &mut file)
             .map_err(|err| format_err!("failed to copy file contents: {}", err))?;
-        if size != extracted {
-            bail!("extracted {} bytes of a file of {} bytes", extracted, size);
+
+        if size != result.written {
+            bail!(
+                "extracted {} bytes of a file of {} bytes",
+                result.written,
+                size
+            );
+        }
+
+        if result.seeked_last {
+            while match nix::unistd::ftruncate(file.as_raw_fd(), size as i64) {
+                Ok(_) => false,
+                Err(nix::Error::Sys(errno)) if errno == nix::errno::Errno::EINTR => true,
+                Err(err) => bail!("error setting file size: {}", err),
+            } {}
         }
 
         metadata::apply(
@@ -454,11 +470,24 @@ impl Extractor {
         )
         .map_err(|err| format_err!("failed to apply initial flags: {}", err))?;
 
-        let extracted = tokio::io::copy(&mut *contents, &mut file)
+        let result = sparse_copy_async(&mut *contents, &mut file)
             .await
             .map_err(|err| format_err!("failed to copy file contents: {}", err))?;
-        if size != extracted {
-            bail!("extracted {} bytes of a file of {} bytes", extracted, size);
+
+        if size != result.written {
+            bail!(
+                "extracted {} bytes of a file of {} bytes",
+                result.written,
+                size
+            );
+        }
+
+        if result.seeked_last {
+            while match nix::unistd::ftruncate(file.as_raw_fd(), size as i64) {
+                Ok(_) => false,
+                Err(nix::Error::Sys(errno)) if errno == nix::errno::Errno::EINTR => true,
+                Err(err) => bail!("error setting file size: {}", err),
+            } {}
         }
 
         metadata::apply(
-- 
2.20.1





  parent reply	other threads:[~2021-02-17 13:13 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-17 13:13 [pbs-devel] [PATCH proxmox/proxmox-backup v5] restore files from pxar sparsely Dominik Csapak
2021-02-17 13:13 ` [pbs-devel] [PATCH proxmox v5 1/2] proxmox: add test/{io, task} modules Dominik Csapak
2021-02-17 13:13 ` [pbs-devel] [PATCH proxmox v5 2/2] proxmox: add sparse_copy(_async) to tools::io Dominik Csapak
2021-02-17 13:13 ` Dominik Csapak [this message]
2021-02-23 14:08 ` [pbs-devel] applied: [PATCH proxmox/proxmox-backup v5] restore files from pxar sparsely Wolfgang Bumiller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210217131322.9129-4-d.csapak@proxmox.com \
    --to=d.csapak@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal