From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <d.csapak@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 301D96AEEE
for <pbs-devel@lists.proxmox.com>; Wed, 17 Feb 2021 14:13:55 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 21E142249A
for <pbs-devel@lists.proxmox.com>; Wed, 17 Feb 2021 14:13:25 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[212.186.127.180])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id D7F6022477
for <pbs-devel@lists.proxmox.com>; Wed, 17 Feb 2021 14:13:23 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 9CF98461E3
for <pbs-devel@lists.proxmox.com>; Wed, 17 Feb 2021 14:13:23 +0100 (CET)
From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Date: Wed, 17 Feb 2021 14:13:22 +0100
Message-Id: <20210217131322.9129-4-d.csapak@proxmox.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210217131322.9129-1-d.csapak@proxmox.com>
References: <20210217131322.9129-1-d.csapak@proxmox.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.220 Adjusted score from AWL reputation of From: address
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/,
medium trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [extract.rs]
Subject: [pbs-devel] [PATCH proxmox-backup v5 1/1] pxar/extract: if possible
create files sparesly
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
<pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 17 Feb 2021 13:13:55 -0000
instead of filling them with zeroes
this fixes an issue where we could not restore a container with large
sparse files in the backup (e.g. a 10GiB sparse file in a container
with a 8GiB disk)
if the last operation of the copy was a seek, we need to truncate
the file to the correct size (seek beyond filesize does not change it)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/pxar/extract.rs | 43 ++++++++++++++++++++++++++++++++++++-------
1 file changed, 36 insertions(+), 7 deletions(-)
diff --git a/src/pxar/extract.rs b/src/pxar/extract.rs
index b673b4b8..952e2d20 100644
--- a/src/pxar/extract.rs
+++ b/src/pxar/extract.rs
@@ -21,7 +21,10 @@ use pxar::Metadata;
use pxar::accessor::aio::{Accessor, FileContents, FileEntry};
use proxmox::c_result;
-use proxmox::tools::fs::{create_path, CreateOptions};
+use proxmox::tools::{
+ fs::{create_path, CreateOptions},
+ io::{sparse_copy, sparse_copy_async},
+};
use crate::pxar::dir_stack::PxarDirStack;
use crate::pxar::metadata;
@@ -411,10 +414,23 @@ impl Extractor {
)
.map_err(|err| format_err!("failed to apply initial flags: {}", err))?;
- let extracted = io::copy(&mut *contents, &mut file)
+ let result = sparse_copy(&mut *contents, &mut file)
.map_err(|err| format_err!("failed to copy file contents: {}", err))?;
- if size != extracted {
- bail!("extracted {} bytes of a file of {} bytes", extracted, size);
+
+ if size != result.written {
+ bail!(
+ "extracted {} bytes of a file of {} bytes",
+ result.written,
+ size
+ );
+ }
+
+ if result.seeked_last {
+ while match nix::unistd::ftruncate(file.as_raw_fd(), size as i64) {
+ Ok(_) => false,
+ Err(nix::Error::Sys(errno)) if errno == nix::errno::Errno::EINTR => true,
+ Err(err) => bail!("error setting file size: {}", err),
+ } {}
}
metadata::apply(
@@ -454,11 +470,24 @@ impl Extractor {
)
.map_err(|err| format_err!("failed to apply initial flags: {}", err))?;
- let extracted = tokio::io::copy(&mut *contents, &mut file)
+ let result = sparse_copy_async(&mut *contents, &mut file)
.await
.map_err(|err| format_err!("failed to copy file contents: {}", err))?;
- if size != extracted {
- bail!("extracted {} bytes of a file of {} bytes", extracted, size);
+
+ if size != result.written {
+ bail!(
+ "extracted {} bytes of a file of {} bytes",
+ result.written,
+ size
+ );
+ }
+
+ if result.seeked_last {
+ while match nix::unistd::ftruncate(file.as_raw_fd(), size as i64) {
+ Ok(_) => false,
+ Err(nix::Error::Sys(errno)) if errno == nix::errno::Errno::EINTR => true,
+ Err(err) => bail!("error setting file size: {}", err),
+ } {}
}
metadata::apply(
--
2.20.1