From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 7B01E1FF16E for ; Mon, 29 Jul 2024 22:49:59 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id BF358324F2; Mon, 29 Jul 2024 22:49:49 +0200 (CEST) In-Reply-To: <2133612618.7197.1722153309575@webmail.proxmox.com> Date: Sun, 28 Jul 2024 10:12:21 -0400 References: <1137775171.7131.1722093632719@webmail.proxmox.com> <2133612618.7197.1722153309575@webmail.proxmox.com> To: Dietmar Maurer X-Mailman-Approved-At: Mon, 29 Jul 2024 22:49:46 +0200 MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Jonathan Nicklin via pve-devel Precedence: list Cc: Jonathan Nicklin , Proxmox VE development discussion X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API Content-Type: multipart/mixed; boundary="===============4844907870986440560==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============4844907870986440560== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id B7AA9C04CF for ; Sun, 28 Jul 2024 16:12:31 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8E1C5978D for ; Sun, 28 Jul 2024 16:12:31 +0200 (CEST) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Sun, 28 Jul 2024 16:12:30 +0200 (CEST) Received: by mail-qk1-x732.google.com with SMTP id af79cd13be357-7a1d5f6c56fso162075685a.0 for ; Sun, 28 Jul 2024 07:12:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=blockbridge.com; s=google; t=1722175943; x=1722780743; darn=lists.proxmox.com; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=3TpHhjbXnVQu5nEbe4swcuFDO+sU7w8z0vTG8Fz647Q=; b=TXCt9JXIs94CP08nhN4sj3BwkEHFxnvCG9uxaDr2u2Vt7+sMnT4NU6QOBYb7EcKrp/ H63qcNrdxstIhgEGy3Fgvb+5Dj5lPNahDN9hahKLmEgirxTUnwdhEPR7y7QsfsT+Fxy1 EHZ5ldC1ZGHmPm7tlYD8j0DSjHdfwanGV0ud6udqdGe6eXn4xU6C381fxSyrR+MWku+O k4etvuOQPR8bRXS97Wxm6ZfWdY/RVKGj+iqFD8BvjxpH0vBPVHZTnXZ/QA8+SHe1wSio rZHius2nGMFVannQn+Bf/eiITBi6AUWjQIck7yYyZURhNKzEfZcIvlVQ+E4qy1+aL416 VJfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722175943; x=1722780743; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3TpHhjbXnVQu5nEbe4swcuFDO+sU7w8z0vTG8Fz647Q=; b=vClJzp5FJYnM+1a6Pl1ytya7nA6qsk1ALWaRUEAgdZN0cqyeJmt4YRWZtm0dlpgfAS RmofLveXwf0KJ6lD5A6/SbbgHcdZ6fXDMWmLIPJ5hfLNOsupRYjn34p66OlNfAB5TOHF JOcBUBWnlvyOfMiIvQVQ+qKTckzBqPDRjZhiSERDzvJdCCY6l9vKpachZP0LWYdSGVoG YKomsnZmbl+vKFwtNga/VeDXJ57PTCSgVkD1WhNVQi+bb5ic2cIO45yyVy3HghgqcVcb 5WaGAsPSVJScavhe0cQvRwG/beN7nwNR5F0NCFahiIKVBG2oNedJdW0U8liuW2ZaKOV6 K6kQ== X-Gm-Message-State: AOJu0YzZJ1uqbum350vUkGM57PL30z4vGWPpfAtc7UQ+j5nUO0wKzNjJ wV32/c8F5I37FIt7OYUWNgASMp5ZIEFFjyDs+FwZi0OS4HNNAgUCEwyEdERjRYGlcXZhLoneZqE j X-Google-Smtp-Source: AGHT+IGaN2vnet6297HE4671f/3sIfPAQQjAcsQesGqYEw0ToRZPOHAXVQ6SCr/JipIwLRR1OwTt3A== X-Received: by 2002:a05:620a:4105:b0:79d:5af9:9e8a with SMTP id af79cd13be357-7a1e52f8e64mr656284885a.69.1722175943086; Sun, 28 Jul 2024 07:12:23 -0700 (PDT) Received: from smtpclient.apple (pool-108-7-52-138.bstnma.fios.verizon.net. [108.7.52.138]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a1d73ed3bcsm436893485a.66.2024.07.28.07.12.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 28 Jul 2024 07:12:22 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API From: Jonathan Nicklin X-Priority: 3 In-Reply-To: <2133612618.7197.1722153309575@webmail.proxmox.com> Date: Sun, 28 Jul 2024 10:12:21 -0400 Cc: Proxmox VE development discussion Content-Transfer-Encoding: quoted-printable Message-Id: References: <1137775171.7131.1722093632719@webmail.proxmox.com> <2133612618.7197.1722153309575@webmail.proxmox.com> To: Dietmar Maurer X-Mailer: Apple Mail (2.3693.20.0.1.32) X-SPAM-LEVEL: Spam detection results: 0 AWL 0.153 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record X-Mailman-Approved-At: Mon, 29 Jul 2024 22:49:46 +0200 I am by no means a CEPH expert. However, my understanding is that other = backup solutions (in the OpenStack world) have used rbd diff to enable = incremental backups. I was hoping that would be relevant here.=20 Here's the description of `rbd diff` Dump a list of byte extents in the image that have changed since the = specified start snapshot, or since the image was created. Each output = line includes the starting offset (in bytes), the length of the region = (in bytes), and either =E2=80=98zero=E2=80=99 or =E2=80=98data=E2=80=99 = to indicate whether the region is known to be zeros or may contain other = data. We (Blockbridge) can also enumerate differences between snapshots in the = form of extent ranges. We share the same concerns regarding the consistency of QEMU bitmaps wrt = storage. That is why relying on the storage to track differences feels = like a more robust solution.=20 > On Jul 28, 2024, at 3:55 AM, Dietmar Maurer = wrote: >=20 >=20 >> The biggest issue we see reported related to QEMU bitmaps is >> persistence. The lack of durability results in unpredictable backup >> behavior at scale. If a host, rack, or data center loses power, = you're >> in for a full backup cycle. Even if several VMs are powered off for >> some reason, it can be a nuisance. Several storage solutions can >> generate the incremental difference bitmaps from durable sources, >> eliminating the issue. >=20 > Several storage solutions provides internal snapshots, but none of = them has an API to access the dirty bitmap (please correct me if I am = wrong). Or what storage solution do you talk about exactly? >=20 > Storing the dirty bitmap persistently would be relatively easy, but so = far we found no way to make sure the bitmap is always up-to-date.=20 > We support shared storages, so multiple nodes can access and modify = the data without updating the dirty bitmap, which would lead to corrupt = backup images... >=20 --===============4844907870986440560== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============4844907870986440560==--