From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
	by lore.proxmox.com (Postfix) with ESMTPS id 5EB171FF16E
	for <inbox@lore.proxmox.com>; Mon, 29 Jul 2024 22:49:53 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 5AA6D324BC;
	Mon, 29 Jul 2024 22:49:49 +0200 (CEST)
In-Reply-To: <392733040.7156.1722149167597@webmail.proxmox.com>
Date: Sun, 28 Jul 2024 09:54:48 -0400
References: <mailman.0.1722092937.302.pve-devel@lists.proxmox.com>
 <1137775171.7131.1722093632719@webmail.proxmox.com>
 <E6295C3B-9E33-47C2-BC0E-9CEC701A2716@blockbridge.com>
 <392733040.7156.1722149167597@webmail.proxmox.com>
To: Dietmar Maurer <dietmar@proxmox.com>
X-Mailman-Approved-At: Mon, 29 Jul 2024 22:49:46 +0200
MIME-Version: 1.0
Message-ID: <mailman.37.1722286187.302.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Jonathan Nicklin via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Jonathan Nicklin <jnicklin@blockbridge.com>,
 Proxmox VE development discussion <pve-devel@lists.proxmox.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager
 00/23] backup provider API
Content-Type: multipart/mixed; boundary="===============3531864196194424006=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>

--===============3531864196194424006==
Content-Type: message/rfc822
Content-Disposition: inline

Return-Path: <jnicklin@blockbridge.com>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
	(No client certificate requested)
	by lists.proxmox.com (Postfix) with ESMTPS id 261B5C04C9
	for <pve-devel@lists.proxmox.com>; Sun, 28 Jul 2024 15:55:29 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 06E609578
	for <pve-devel@lists.proxmox.com>; Sun, 28 Jul 2024 15:54:59 +0200 (CEST)
Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
	(No client certificate requested)
	by firstgate.proxmox.com (Proxmox) with ESMTPS
	for <pve-devel@lists.proxmox.com>; Sun, 28 Jul 2024 15:54:57 +0200 (CEST)
Received: by mail-qk1-x72f.google.com with SMTP id af79cd13be357-7a1df0a93eeso138978485a.1
        for <pve-devel@lists.proxmox.com>; Sun, 28 Jul 2024 06:54:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=blockbridge.com; s=google; t=1722174890; x=1722779690; darn=lists.proxmox.com;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=hJRcyiQ4umoQXZrVMQiDAkTWwcdKqHnXPsegmkqPMvc=;
        b=pC/y952L2pMXlk/NgXva4t+Ci7ewyRjf9mMCbaFT6nR7ta3LhGZHZR+twfDDqmlgk2
         w6ClSdGhy8k2WSclRSKps9YKbWFinCfKEKwpSWGCOFqgpLqadZQL3Jkxpc7EZRMZhFSq
         I6GqgvUZubbjrTv+WKYzcaq9ARtZSzGMakRo5e/qArJKn1kPkJ6e8S+aWlIR6BdcyDeO
         a3kW3tSklueyFOetJmLR0M/d6HAnjtDBjc+l2jnup/0Q/5d62AOe4+2OIMASXlaz4eNB
         5qibuUWKrSSf5UdYvOqT0vRm9pUVNc5XsOBnojK2GVt/uf/NUj/GwfZBG6NQfjK0H9t5
         +QAA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1722174890; x=1722779690;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=hJRcyiQ4umoQXZrVMQiDAkTWwcdKqHnXPsegmkqPMvc=;
        b=rStfw6Mbe5eeTXkxg8+b1hd6B3pN7/qJhy9+6j+e/rzGq1dF4Dj5rysU4LZw9qug9V
         W4YQbjaiM8G7jnNvaO+kl14z6bUVc1J9HQf+bRpWRKez+5Of5pC1bh9X2I7gg4kawKH3
         7uSqaHfZgjxVPHTZ2JZQQD0m8Psu0Cha3+7nlaRXvC+PoboSZg/SnaW8ZBRf5UYnIuCJ
         wmf7djz8q+/ZyTOMt5w/rkVvW8eP+13TlV2Jtb4FT8gd/favmw+AjKAEwZDIoVRukbDg
         UQOjoE/N609PwB3wmdbKG+1NWbP9nZ+VvhK2NWy6t/OTCDKIGJ4RdWAk0ByxiCCsNlTD
         x/Rw==
X-Gm-Message-State: AOJu0Yz/bDUuZlfLweIXYnzYxNCSeFhz7iWYEwI4WSFsLDObS33TQy4p
	LFG7m0LRWv2voEkhqGeUlnE+2Av2OsDgygjNgt4dEzxzOCNfhzvGqD6CN7iWWLJdckk2TrbZ68s
	d
X-Google-Smtp-Source: AGHT+IHMT/YTBzBcT11Jbv+dKSQkgWWNx2lQMg7Lk66LMXQ1x3fdDVMm/i/6TncWhZ5Ig7BU50FZbQ==
X-Received: by 2002:a05:6214:c6d:b0:6b5:61:53a9 with SMTP id 6a1803df08f44-6bb55a44458mr54944846d6.28.1722174890206;
        Sun, 28 Jul 2024 06:54:50 -0700 (PDT)
Received: from smtpclient.apple (pool-108-7-52-138.bstnma.fios.verizon.net. [108.7.52.138])
        by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6bb3f8d8280sm42445486d6.18.2024.07.28.06.54.49
        (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
        Sun, 28 Jul 2024 06:54:49 -0700 (PDT)
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\))
Subject: Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager
 00/23] backup provider API
From: Jonathan Nicklin <jnicklin@blockbridge.com>
X-Priority: 3
In-Reply-To: <392733040.7156.1722149167597@webmail.proxmox.com>
Date: Sun, 28 Jul 2024 09:54:48 -0400
Cc: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <1C86CC96-2C9C-466A-A2A9-FC95906C098E@blockbridge.com>
References: <mailman.0.1722092937.302.pve-devel@lists.proxmox.com>
 <1137775171.7131.1722093632719@webmail.proxmox.com>
 <E6295C3B-9E33-47C2-BC0E-9CEC701A2716@blockbridge.com>
 <392733040.7156.1722149167597@webmail.proxmox.com>
To: Dietmar Maurer <dietmar@proxmox.com>
X-Mailer: Apple Mail (2.3693.20.0.1.32)
X-SPAM-LEVEL: Spam detection results:  0
	AWL                     0.230 Adjusted score from AWL reputation of From: address
	BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
	DKIM_SIGNED               0.1 Message has a DKIM or DK signature, not necessarily valid
	DKIM_VALID               -0.1 Message has at least one valid DKIM or DK signature
	DKIM_VALID_AU            -0.1 Message has a valid DKIM or DK signature from author's domain
	DKIM_VALID_EF            -0.1 Message has a valid DKIM or DK signature from envelope-from domain
	DMARC_PASS               -0.1 DMARC pass policy
	RCVD_IN_DNSWL_NONE     -0.0001 Sender listed at https://www.dnswl.org/, no trust
	SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
	SPF_PASS               -0.001 SPF: sender matches SPF record
X-Mailman-Approved-At: Mon, 29 Jul 2024 22:49:46 +0200

In hyper-converged deployments, the node performing the backup is =
sourcing ((nodes-1)/(nodes))*bytes) of backup data (i.e., ingress =
traffic) and then sending 1*bytes to PBS (i.e., egress traffic). If PBS =
were to pull the data from the nodes directly, the maximum load on any =
one host would be (1/nodes)*bytes of egress traffic only... that's a =
considerable improvement!

Further, nodes that don't host OSDs would be completely quiet. So, in =
the case of non-converged CEPH, the hypervisor nodes do not need to =
participate in the backup flow at all.

> On Jul 28, 2024, at 2:46 AM, Dietmar Maurer <dietmar@proxmox.com> =
wrote:
>=20
>> Today, I believe the client is reading the data and pushing it to
>> PBS. In the case of CEPH, wouldn't this involve sourcing data from
>> multiple nodes and then sending it to PBS? Wouldn't it be more
>> efficient for PBS to read it directly from storage? In the case of
>> centralized storage, we'd like to eliminate the client load
>> completely, having PBS ingest increment differences directly from
>> storage without passing through the client.
>=20
> But Ceph is not a central storage. Instead, data is distributed among =
the nodes, so you always need to send some data over the network.
> There is no way to "read it directly from storage".
>=20



--===============3531864196194424006==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

--===============3531864196194424006==--