From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup v4 1/1] pxar/extract: if possible create files sparesly
Date: Fri, 12 Feb 2021 15:44:33 +0100 [thread overview]
Message-ID: <20210212144433.30668-4-d.csapak@proxmox.com> (raw)
In-Reply-To: <20210212144433.30668-1-d.csapak@proxmox.com>
instead of filling them with zeroes
this fixes an issue where we could not restore a container with large
sparse files in the backup (e.g. a 10GiB sparse file in a container
with a 8GiB disk)
if the last operation of the copy was a seek, we need to truncate
the file to the correct size (seek beyond filesize does not change it)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
| 43 ++++++++++++++++++++++++++++++++++++-------
1 file changed, 36 insertions(+), 7 deletions(-)
--git a/src/pxar/extract.rs b/src/pxar/extract.rs
index 0a61c885..80095e3f 100644
--- a/src/pxar/extract.rs
+++ b/src/pxar/extract.rs
@@ -18,7 +18,10 @@ use pxar::format::Device;
use pxar::Metadata;
use proxmox::c_result;
-use proxmox::tools::fs::{create_path, CreateOptions};
+use proxmox::tools::{
+ fs::{create_path, CreateOptions},
+ io::{sparse_copy, sparse_copy_async},
+};
use crate::pxar::dir_stack::PxarDirStack;
use crate::pxar::metadata;
@@ -406,10 +409,23 @@ impl Extractor {
)
.map_err(|err| format_err!("failed to apply initial flags: {}", err))?;
- let extracted = io::copy(&mut *contents, &mut file)
+ let result = sparse_copy(&mut *contents, &mut file)
.map_err(|err| format_err!("failed to copy file contents: {}", err))?;
- if size != extracted {
- bail!("extracted {} bytes of a file of {} bytes", extracted, size);
+
+ if size != result.written {
+ bail!(
+ "extracted {} bytes of a file of {} bytes",
+ result.written,
+ size
+ );
+ }
+
+ if result.seeked_last {
+ while match nix::unistd::ftruncate(file.as_raw_fd(), size as i64) {
+ Ok(_) => false,
+ Err(nix::Error::Sys(errno)) if errno == nix::errno::Errno::EINTR => true,
+ Err(err) => bail!("error setting file size: {}", err),
+ } {}
}
metadata::apply(
@@ -449,11 +465,24 @@ impl Extractor {
)
.map_err(|err| format_err!("failed to apply initial flags: {}", err))?;
- let extracted = tokio::io::copy(&mut *contents, &mut file)
+ let result = sparse_copy_async(&mut *contents, &mut file)
.await
.map_err(|err| format_err!("failed to copy file contents: {}", err))?;
- if size != extracted {
- bail!("extracted {} bytes of a file of {} bytes", extracted, size);
+
+ if size != result.written {
+ bail!(
+ "extracted {} bytes of a file of {} bytes",
+ result.written,
+ size
+ );
+ }
+
+ if result.seeked_last {
+ while match nix::unistd::ftruncate(file.as_raw_fd(), size as i64) {
+ Ok(_) => false,
+ Err(nix::Error::Sys(errno)) if errno == nix::errno::Errno::EINTR => true,
+ Err(err) => bail!("error setting file size: {}", err),
+ } {}
}
metadata::apply(
--
2.20.1
prev parent reply other threads:[~2021-02-12 14:44 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-12 14:44 [pbs-devel] [PATCH proxmox v4 0/2] restore files from pxar sparsely Dominik Csapak
2021-02-12 14:44 ` [pbs-devel] [PATCH proxmox v4 1/2] proxmox/tools: add poll_once module for testing Dominik Csapak
2021-02-12 14:44 ` [pbs-devel] [PATCH proxmox v4 2/2] proxmox: add sparse_copy(_async) to tools::io Dominik Csapak
2021-02-12 14:44 ` Dominik Csapak [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210212144433.30668-4-d.csapak@proxmox.com \
--to=d.csapak@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox