From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <d.csapak@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id E6DD26B721
 for <pbs-devel@lists.proxmox.com>; Fri, 11 Dec 2020 13:09:31 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id E2C391FBE1
 for <pbs-devel@lists.proxmox.com>; Fri, 11 Dec 2020 13:09:01 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [212.186.127.180])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS id AF56D1FBBE
 for <pbs-devel@lists.proxmox.com>; Fri, 11 Dec 2020 13:09:00 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 7A7F544F6A
 for <pbs-devel@lists.proxmox.com>; Fri, 11 Dec 2020 13:09:00 +0100 (CET)
From: Dominik Csapak <d.csapak@proxmox.com>
To: pbs-devel@lists.proxmox.com
Date: Fri, 11 Dec 2020 13:08:58 +0100
Message-Id: <20201211120859.17323-3-d.csapak@proxmox.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20201211120859.17323-1-d.csapak@proxmox.com>
References: <20201211120859.17323-1-d.csapak@proxmox.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results:  0
 AWL 0.288 Adjusted score from AWL reputation of From: address
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 RCVD_IN_DNSWL_MED        -2.3 Sender listed at https://www.dnswl.org/,
 medium trust
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
 information. [mod.rs]
Subject: [pbs-devel] [PATCH proxmox 2/2] proxmox: add sparse_copy(_async) to
 tools::io
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
 <pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Fri, 11 Dec 2020 12:09:31 -0000

this is able to seek the target instead of writing zeroes, which
generates sparse files where supported

it does not guarantee that all zero bytes are skipped, only when the
buffer after a read solely consists of zeroes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 proxmox/src/tools/io/mod.rs | 62 +++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/proxmox/src/tools/io/mod.rs b/proxmox/src/tools/io/mod.rs
index 2e92ebb..53f767c 100644
--- a/proxmox/src/tools/io/mod.rs
+++ b/proxmox/src/tools/io/mod.rs
@@ -3,8 +3,70 @@
 //! The [`ReadExt`] trait provides additional operations for handling byte buffers for types
 //! implementing [`Read`](std::io::Read).
 
+use std::io::{self, Read, Write, Seek, SeekFrom, ErrorKind};
+
 mod read;
 pub use read::*;
 
 mod write;
 pub use write::*;
+
+/// copy similar to io::copy, but seeks the target when encountering
+/// zero bytes instead of writing them
+pub fn sparse_copy<R: Read + ?Sized, W: Write + Seek + ?Sized>(
+    reader: &mut R,
+    writer: &mut W,
+) -> Result<u64, io::Error> {
+    let mut buf = crate::tools::byte_buffer::ByteBuffer::new();
+    let mut written = 0;
+    loop {
+        let len = match buf.read_from(reader) {
+            Ok(0) => return Ok(written),
+            Ok(len) => len,
+            Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
+            Err(e) => return Err(e),
+        };
+
+        if crate::tools::zero::buffer_is_zero(&buf[..]) {
+            writer.seek(SeekFrom::Current(len as i64))?;
+        } else {
+            writer.write_all(&buf[..])?;
+        }
+        buf.clear();
+        written += len as u64;
+    }
+}
+
+#[cfg(feature = "tokio")]
+use tokio::io::{AsyncReadExt, AsyncWriteExt, AsyncSeekExt};
+
+#[cfg(feature = "tokio")]
+/// copy similar to tokio::io::copy, but seeks the target when encountering
+/// zero bytes instead of writing them
+pub async fn sparse_copy_async<R, W>(
+    reader: &mut R,
+    writer: &mut W,
+) -> Result<u64, io::Error>
+where
+    R: AsyncReadExt + Unpin,
+    W: AsyncWriteExt + AsyncSeekExt + Unpin,
+{
+    let mut buf = crate::tools::byte_buffer::ByteBuffer::new();
+    let mut written = 0;
+    loop {
+        let len = match buf.read_from_async(reader).await {
+            Ok(0) => return Ok(written),
+            Ok(len) => len,
+            Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
+            Err(e) => return Err(e),
+        };
+
+        if crate::tools::zero::buffer_is_zero(&buf[..]) {
+            writer.seek(SeekFrom::Current(len as i64)).await?;
+        } else {
+            writer.write_all(&buf[..]).await?;
+        }
+        buf.clear();
+        written += len as u64;
+    }
+}
-- 
2.20.1