From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id A83011FF16E
for <inbox@lore.proxmox.com>; Mon, 17 Mar 2025 11:53:31 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 2859C37AF7;
Mon, 17 Mar 2025 11:53:20 +0100 (CET)
Message-ID: <51c423c9-786c-4881-8819-83c075c89b36@proxmox.com>
Date: Mon, 17 Mar 2025 11:53:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
To: Prashant Patil <Prashant.Gamepatil@veritas.com>,
Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Thomas Lamprecht <t.lamprecht@proxmox.com>
References: <PH0PR20MB4520A4201D4560B18A0C830798C82@PH0PR20MB4520.namprd20.prod.outlook.com>
<PH0PR20MB45201A18272FF3B7B386D98B98C82@PH0PR20MB4520.namprd20.prod.outlook.com>
<2402a2e8-63a5-46c4-9b66-d998f6e421c7@proxmox.com>
<mailman.766.1741156631.293.pve-devel@lists.proxmox.com>
<c355aa91-fcdf-46d2-9ed7-89c215ae5d01@proxmox.com>
<PH0PR20MB4520C688E38C97D5DE5FC25B98DF2@PH0PR20MB4520.namprd20.prod.outlook.com>
Content-Language: en-US
From: Fiona Ebner <f.ebner@proxmox.com>
In-Reply-To: <PH0PR20MB4520C688E38C97D5DE5FC25B98DF2@PH0PR20MB4520.namprd20.prod.outlook.com>
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.041 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [mail-archive.com]
Subject: Re: [pve-devel] About PVE Backup Integration Guide
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>,
Sudhir Subbarao <Sudhir.Subbarao@veritas.com>,
Jason Voneberstein <Jason.vonEberstein@veritas.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Am 17.03.25 um 08:02 schrieb Prashant Patil:
>> The block tracking is ideally done via QEMU, then you don't require any special features for the underlying storages.
>
> Yes, we were able to get sector map of disks present on 'Directory' storage type. However, for other storage types such as lvm, lvm-thin, zfs which supports raw disk format, here we could get entire disk as allocated which is not the case in real. I could not find much information on this, hence wanted to know whether this by-design behaviour or we are missing something? Pasting output of qemu-img map below for the 2 GB disk on zfs. Have also tried getting map over ndb but got the same result. Is there anything that we are missing here?
>
> root@be-proxmox1:/dev/pve# qemu-img map -f raw --output=json /dev/zvol/zfs1/vm-105-disk-2
> [{ "start": 0, "length": 2147483648, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 0}]
That sounds like you want to roll your own external solution instead of
using the backup provider API that is currently being developed. With
that API, you get the images to be backed-up as as NBD exports. For
incremental backup, you can read the dirty bitmap. This can also be done
via NBD. Again, then you don't need to worry about the underlying
storage layer at all to support certain features.
>> certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?
>
> As mentioned earlier, we have found few storage devices which does not support snapshot, but have found that we can take individual disk snapshot through 'blockdev-snapshot-sync'. If we have to take backup of the VM, then are we supposed to use this command to snapshot all VM disks?
Backups in QEMU do not use explicit snapshots. A copy-before-write
filter is inserted on top of the source disk in QEMU's block graph. When
new guest writes happen during backup, old data is first copied away to
a fleecing image (or for regular backup directly to the backup target).
The backup provider API then also inserts a snapshot-access node that is
exported via NBD and allows reading the data from the time of the backup
in a consistent fashion (hence "snapshot-access", it's a
virtual/implicit snapshot).
See also this diagram [0].
> [guest] [NBD export]
> | |
> | root | root
> v file v
> [copy-before-write]<------[snapshot-access]
> | |
> | file | target
> v v
> [active-disk] [temp.qcow2]
Best Regards,
Fiona
[0]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg876056.html
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel