From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E729863174 for ; Tue, 9 Feb 2021 13:04:02 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DCBA72CAA4 for ; Tue, 9 Feb 2021 13:04:02 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 0E6C22CA88 for ; Tue, 9 Feb 2021 13:03:59 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id CF837454E9 for ; Tue, 9 Feb 2021 13:03:58 +0100 (CET) From: Stefan Reiter To: pbs-devel@lists.proxmox.com Date: Tue, 9 Feb 2021 13:03:46 +0100 Message-Id: <20210209120348.8359-1-s.reiter@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.036 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [create.rs, aio.rs, sync.rs, catar.rs, mod.rs, pxar.rs] Subject: [pbs-devel] [PATCH 0/2] use async pxar encoder X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Feb 2021 12:04:02 -0000 No change in functionality intended for these two patches, but they will be needed for my upcoming file-restore series. (or, well, they avoid doing the same sync-but-in-thread style we follow in PxarBackupStream right now) Changes the pxar archive creation code to fully embracy async/await and the aio::Encoder from the pxar crate. Requires some changes in both crates, to ensure 'SeqWrite + Send' is properly supported. Testing and benchmarking showed no real difference in behaviour, except for a very slight slowdown when backing up a folder with many smaller files (I tested on my local QEMU repo, that's 11.2 GiB of git indices and data/code files atm). This may be attributable to the introduced Mutex locking, or async/await overhead - or just a fluke on my machine. 17s without, 19s with the patches. Folders with fewer, but bigger files are unaffected. Certainly learned a lot of Rust from this :) Lifetimes are still hard to follow IMHO, especially with async... Feedback is appreciated of course, if there's anything that can be done better. pxar: Stefan Reiter (1): make aio::Encoder actually behave with async src/encoder/aio.rs | 48 ++++++++--------- src/encoder/mod.rs | 128 +++++++++++++++++++++++--------------------- src/encoder/sync.rs | 28 ++++------ 3 files changed, 101 insertions(+), 103 deletions(-) proxmox-backup: Stefan Reiter (1): asyncify pxar create_archive src/bin/pxar.rs | 6 +- src/client/pxar_backup_stream.rs | 65 +++++----- src/pxar/create.rs | 207 ++++++++++++++++--------------- tests/catar.rs | 5 +- 4 files changed, 143 insertions(+), 140 deletions(-) -- 2.20.1