From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 7B01E1FF16E
for <inbox@lore.proxmox.com>; Mon, 29 Jul 2024 22:49:59 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id BF358324F2;
Mon, 29 Jul 2024 22:49:49 +0200 (CEST)
In-Reply-To: <2133612618.7197.1722153309575@webmail.proxmox.com>
Date: Sun, 28 Jul 2024 10:12:21 -0400
References: <mailman.0.1722092937.302.pve-devel@lists.proxmox.com>
<1137775171.7131.1722093632719@webmail.proxmox.com>
<E6295C3B-9E33-47C2-BC0E-9CEC701A2716@blockbridge.com>
<2133612618.7197.1722153309575@webmail.proxmox.com>
To: Dietmar Maurer <dietmar@proxmox.com>
X-Mailman-Approved-At: Mon, 29 Jul 2024 22:49:46 +0200
MIME-Version: 1.0
Message-ID: <mailman.38.1722286187.302.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Jonathan Nicklin via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Jonathan Nicklin <jnicklin@blockbridge.com>,
Proxmox VE development discussion <pve-devel@lists.proxmox.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager
00/23] backup provider API
Content-Type: multipart/mixed; boundary="===============4844907870986440560=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
--===============4844907870986440560==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <jnicklin@blockbridge.com>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id B7AA9C04CF
for <pve-devel@lists.proxmox.com>; Sun, 28 Jul 2024 16:12:31 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 8E1C5978D
for <pve-devel@lists.proxmox.com>; Sun, 28 Jul 2024 16:12:31 +0200 (CEST)
Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732])
(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Sun, 28 Jul 2024 16:12:30 +0200 (CEST)
Received: by mail-qk1-x732.google.com with SMTP id af79cd13be357-7a1d5f6c56fso162075685a.0
for <pve-devel@lists.proxmox.com>; Sun, 28 Jul 2024 07:12:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=blockbridge.com; s=google; t=1722175943; x=1722780743; darn=lists.proxmox.com;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=3TpHhjbXnVQu5nEbe4swcuFDO+sU7w8z0vTG8Fz647Q=;
b=TXCt9JXIs94CP08nhN4sj3BwkEHFxnvCG9uxaDr2u2Vt7+sMnT4NU6QOBYb7EcKrp/
H63qcNrdxstIhgEGy3Fgvb+5Dj5lPNahDN9hahKLmEgirxTUnwdhEPR7y7QsfsT+Fxy1
EHZ5ldC1ZGHmPm7tlYD8j0DSjHdfwanGV0ud6udqdGe6eXn4xU6C381fxSyrR+MWku+O
k4etvuOQPR8bRXS97Wxm6ZfWdY/RVKGj+iqFD8BvjxpH0vBPVHZTnXZ/QA8+SHe1wSio
rZHius2nGMFVannQn+Bf/eiITBi6AUWjQIck7yYyZURhNKzEfZcIvlVQ+E4qy1+aL416
VJfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1722175943; x=1722780743;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=3TpHhjbXnVQu5nEbe4swcuFDO+sU7w8z0vTG8Fz647Q=;
b=vClJzp5FJYnM+1a6Pl1ytya7nA6qsk1ALWaRUEAgdZN0cqyeJmt4YRWZtm0dlpgfAS
RmofLveXwf0KJ6lD5A6/SbbgHcdZ6fXDMWmLIPJ5hfLNOsupRYjn34p66OlNfAB5TOHF
JOcBUBWnlvyOfMiIvQVQ+qKTckzBqPDRjZhiSERDzvJdCCY6l9vKpachZP0LWYdSGVoG
YKomsnZmbl+vKFwtNga/VeDXJ57PTCSgVkD1WhNVQi+bb5ic2cIO45yyVy3HghgqcVcb
5WaGAsPSVJScavhe0cQvRwG/beN7nwNR5F0NCFahiIKVBG2oNedJdW0U8liuW2ZaKOV6
K6kQ==
X-Gm-Message-State: AOJu0YzZJ1uqbum350vUkGM57PL30z4vGWPpfAtc7UQ+j5nUO0wKzNjJ
wV32/c8F5I37FIt7OYUWNgASMp5ZIEFFjyDs+FwZi0OS4HNNAgUCEwyEdERjRYGlcXZhLoneZqE
j
X-Google-Smtp-Source: AGHT+IGaN2vnet6297HE4671f/3sIfPAQQjAcsQesGqYEw0ToRZPOHAXVQ6SCr/JipIwLRR1OwTt3A==
X-Received: by 2002:a05:620a:4105:b0:79d:5af9:9e8a with SMTP id af79cd13be357-7a1e52f8e64mr656284885a.69.1722175943086;
Sun, 28 Jul 2024 07:12:23 -0700 (PDT)
Received: from smtpclient.apple (pool-108-7-52-138.bstnma.fios.verizon.net. [108.7.52.138])
by smtp.gmail.com with ESMTPSA id af79cd13be357-7a1d73ed3bcsm436893485a.66.2024.07.28.07.12.22
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Sun, 28 Jul 2024 07:12:22 -0700 (PDT)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\))
Subject: Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager
00/23] backup provider API
From: Jonathan Nicklin <jnicklin@blockbridge.com>
X-Priority: 3
In-Reply-To: <2133612618.7197.1722153309575@webmail.proxmox.com>
Date: Sun, 28 Jul 2024 10:12:21 -0400
Cc: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <CD15F8AE-29C2-441B-8874-62E3FB92C1F8@blockbridge.com>
References: <mailman.0.1722092937.302.pve-devel@lists.proxmox.com>
<1137775171.7131.1722093632719@webmail.proxmox.com>
<E6295C3B-9E33-47C2-BC0E-9CEC701A2716@blockbridge.com>
<2133612618.7197.1722153309575@webmail.proxmox.com>
To: Dietmar Maurer <dietmar@proxmox.com>
X-Mailer: Apple Mail (2.3693.20.0.1.32)
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.153 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain
DMARC_PASS -0.1 DMARC pass policy
RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
X-Mailman-Approved-At: Mon, 29 Jul 2024 22:49:46 +0200
I am by no means a CEPH expert. However, my understanding is that other =
backup solutions (in the OpenStack world) have used rbd diff to enable =
incremental backups. I was hoping that would be relevant here.=20
Here's the description of `rbd diff`
<rbd-diff>
Dump a list of byte extents in the image that have changed since the =
specified start snapshot, or since the image was created. Each output =
line includes the starting offset (in bytes), the length of the region =
(in bytes), and either =E2=80=98zero=E2=80=99 or =E2=80=98data=E2=80=99 =
to indicate whether the region is known to be zeros or may contain other =
data.
</rbd-diff>
We (Blockbridge) can also enumerate differences between snapshots in the =
form of extent ranges.
We share the same concerns regarding the consistency of QEMU bitmaps wrt =
storage. That is why relying on the storage to track differences feels =
like a more robust solution.=20
> On Jul 28, 2024, at 3:55 AM, Dietmar Maurer <dietmar@proxmox.com> =
wrote:
>=20
>=20
>> The biggest issue we see reported related to QEMU bitmaps is
>> persistence. The lack of durability results in unpredictable backup
>> behavior at scale. If a host, rack, or data center loses power, =
you're
>> in for a full backup cycle. Even if several VMs are powered off for
>> some reason, it can be a nuisance. Several storage solutions can
>> generate the incremental difference bitmaps from durable sources,
>> eliminating the issue.
>=20
> Several storage solutions provides internal snapshots, but none of =
them has an API to access the dirty bitmap (please correct me if I am =
wrong). Or what storage solution do you talk about exactly?
>=20
> Storing the dirty bitmap persistently would be relatively easy, but so =
far we found no way to make sure the bitmap is always up-to-date.=20
> We support shared storages, so multiple nodes can access and modify =
the data without updating the dirty bitmap, which would lead to corrupt =
backup images...
>=20
--===============4844907870986440560==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--===============4844907870986440560==--