From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id CAD1777B35 for ; Wed, 28 Apr 2021 18:07:35 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C191B119B3 for ; Wed, 28 Apr 2021 18:07:05 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id EF33D11996 for ; Wed, 28 Apr 2021 18:07:04 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id C9B3E46495 for ; Wed, 28 Apr 2021 18:07:04 +0200 (CEST) From: Stefan Reiter To: pbs-devel@lists.proxmox.com Date: Wed, 28 Apr 2021 18:06:52 +0200 Message-Id: <20210428160655.29941-1-s.reiter@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.017 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH 0/3] Add cache for live-restore X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Apr 2021 16:07:35 -0000 Adds a mmap-backed buffer (as suggested by Dietmar to avoid memory starvation) of 64+256 MiB for accessing chunks from PBS. Helps to accelerate live-restore. Quantitative benchmarking appears really hard, variance between live-restore runs is really high for some reason (both time-to-desktop and time-to-finish). On average it seems to be between equal and up to 30% faster - again, with a lot of variance between runs. In testing with a Windows VM I saw cache hit rates of about 25% or higher, so the cache is certainly used, but there seem to be too many other factors in play... proxmox-backup: Stefan Reiter (2): tools: add mmap_buffer module RemoteChunkReader: add LRU cached variant src/bin/proxmox_backup_client/mount.rs | 12 ++- src/client/remote_chunk_reader.rs | 110 ++++++++++++++++++++---- src/tools.rs | 1 + src/tools/mmap_buffer.rs | 113 +++++++++++++++++++++++++ 4 files changed, 218 insertions(+), 18 deletions(-) create mode 100644 src/tools/mmap_buffer.rs proxmox-backup-qemu: Stefan Reiter (1): access: use bigger cache and LRU chunk reader src/restore.rs | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) -- 2.20.1