From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup 1/1] pxar/extrac: if possible create files sparesly
Date: Fri, 11 Dec 2020 13:08:59 +0100 [thread overview]
Message-ID: <20201211120859.17323-4-d.csapak@proxmox.com> (raw)
In-Reply-To: <20201211120859.17323-1-d.csapak@proxmox.com>
instead of filling them with zeroes
this fixes an issue where we could not restore a container with large
sparse files in the backup (e.g. a 10GiB sparse file in a container
with a 8GiB disk)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
| 32 ++++++++++++++++++++++++++------
1 file changed, 26 insertions(+), 6 deletions(-)
--git a/src/pxar/extract.rs b/src/pxar/extract.rs
index ed238a2c..dd084ead 100644
--- a/src/pxar/extract.rs
+++ b/src/pxar/extract.rs
@@ -18,7 +18,10 @@ use pxar::format::Device;
use pxar::Metadata;
use proxmox::c_result;
-use proxmox::tools::fs::{create_path, CreateOptions};
+use proxmox::tools::{
+ fs::{create_path, CreateOptions},
+ io::{sparse_copy, sparse_copy_async}
+};
use crate::pxar::dir_stack::PxarDirStack;
use crate::pxar::metadata;
@@ -392,6 +395,11 @@ impl Extractor {
)
};
+ let copy_sparse = match nix::unistd::ftruncate(file.as_raw_fd(), size as i64) {
+ Ok(_) => true,
+ Err(_) => false,
+ };
+
metadata::apply_initial_flags(
self.feature_flags,
metadata,
@@ -399,8 +407,12 @@ impl Extractor {
&mut self.on_error,
)?;
- let extracted = io::copy(&mut *contents, &mut file)
- .map_err(|err| format_err!("failed to copy file contents: {}", err))?;
+ let extracted = if copy_sparse {
+ sparse_copy(&mut *contents, &mut file)
+ } else {
+ io::copy(&mut *contents, &mut file)
+ }.map_err(|err| format_err!("failed to copy file contents: {}", err))?;
+
if size != extracted {
bail!("extracted {} bytes of a file of {} bytes", extracted, size);
}
@@ -434,6 +446,11 @@ impl Extractor {
)
});
+ let copy_sparse = match nix::unistd::ftruncate(file.as_raw_fd(), size as i64) {
+ Ok(_) => true,
+ Err(_) => false,
+ };
+
metadata::apply_initial_flags(
self.feature_flags,
metadata,
@@ -441,9 +458,12 @@ impl Extractor {
&mut self.on_error,
)?;
- let extracted = tokio::io::copy(&mut *contents, &mut file)
- .await
- .map_err(|err| format_err!("failed to copy file contents: {}", err))?;
+ let extracted = if copy_sparse {
+ sparse_copy_async(&mut *contents, &mut file).await
+ } else {
+ tokio::io::copy(&mut *contents, &mut file).await
+ }.map_err(|err| format_err!("failed to copy file contents: {}", err))?;
+
if size != extracted {
bail!("extracted {} bytes of a file of {} bytes", extracted, size);
}
--
2.20.1
prev parent reply other threads:[~2020-12-11 12:09 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-11 12:08 [pbs-devel] [PATCH RFC proxmox/proxmox-backup] restore files from pxar sparsely Dominik Csapak
2020-12-11 12:08 ` [pbs-devel] [PATCH proxmox 1/2] add tools/zero: add fast zero comparison code Dominik Csapak
2020-12-14 8:38 ` Thomas Lamprecht
2020-12-14 12:52 ` Wolfgang Bumiller
2020-12-11 12:08 ` [pbs-devel] [PATCH proxmox 2/2] proxmox: add sparse_copy(_async) to tools::io Dominik Csapak
2020-12-11 12:08 ` Dominik Csapak [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201211120859.17323-4-d.csapak@proxmox.com \
--to=d.csapak@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal