From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 44F991FF168 for ; Sun, 28 Jul 2024 16:58:45 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8107DA32E; Sun, 28 Jul 2024 16:58:42 +0200 (CEST) Date: Sun, 28 Jul 2024 16:58:38 +0200 (CEST) From: Dietmar Maurer To: Jonathan Nicklin Message-ID: <1007402234.7276.1722178718125@webmail.proxmox.com> In-Reply-To: <1C86CC96-2C9C-466A-A2A9-FC95906C098E@blockbridge.com> References: <1137775171.7131.1722093632719@webmail.proxmox.com> <392733040.7156.1722149167597@webmail.proxmox.com> <1C86CC96-2C9C-466A-A2A9-FC95906C098E@blockbridge.com> MIME-Version: 1.0 X-Priority: 3 Importance: Normal X-Mailer: Open-Xchange Mailer v7.10.6-Rev67 X-Originating-Client: open-xchange-appsuite X-SPAM-LEVEL: Spam detection results: 0 AWL 0.348 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Cc: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" > In hyper-converged deployments, the node performing the backup is sourcing ((nodes-1)/(nodes))*bytes) of backup data (i.e., ingress traffic) and then sending 1*bytes to PBS (i.e., egress traffic). If PBS were to pull the data from the nodes directly, the maximum load on any one host would be (1/nodes)*bytes of egress traffic only... that's a considerable improvement! I guess it would be possible to write a tool like proxmox-backup-client that pull ceph backups directly from PBS. Or extend the backup protokoll allowing direct storage access. But this is a considerable amount of development, and needs much more configuration/setup than the current approach. But patches are always welcome... Also, it is not clear to me how we can implement a "backup provider API" if we add such optimizations? And yes, network traffic would be reduced. But IMHO it is easier to add a dedicated network card for the backup server (if the network is the limiting factor). With this setup, the maximum load on the ceph network is (1/nodes)*bytess of egress traffic only. The backup traffic is on the dedicated backup net. _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel