[pve-devel] About PVE Backup Integration Guide

[pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 11241 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: "pve-devel@lists.proxmox.com" <pve-devel@lists.proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: About PVE Backup Integration Guide
Date: Tue, 4 Mar 2025 13:21:17 +0000
Message-ID: <PH0PR20MB4520A4201D4560B18A0C830798C82@PH0PR20MB4520.namprd20.prod.outlook.com>

Hello All,
Hope this email finds you well.

At Arctera/Veritas, we are exploring the addition of support for data protection of Proxmox environments within our backup product. We would like to know if there is a detailed backup/restore API guide available that could serve as a good starting point for our engineering team. Could you please assist us with this? We have already reviewed the REST APIs.

Regards,
Prashant


[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Thomas Lamprecht]

Hello Prashant,

Am 04.03.25 um 16:47 schrieb Prashant Patil:
> Hello All,
> Hope this email finds you well.
> 
> At Arctera/Veritas, we are exploring the addition of support for data protection of Proxmox environments within our backup product. We would like to know if there is a detailed backup/restore API guide available that could serve as a good starting point for our engineering team. Could you please assist us with this? We have already reviewed the REST APIs.

Thanks for your interest in supporting Proxmox VE.

We are in progress of adding a way where external backup provider can
provide a plugin to hook safely and correctly into our backup APIs, see:

https://lore.proxmox.com/pve-devel/20241114150754.374376-1-f.ebner@proxmox.com/

This includes two example plugins to showcase how it would be used.

We would be open to fast-track this effort for inclusion with feedback
from backup vendors. While there are other ways to hook one in, these
are not nice and can cause problems – so in the long term we expect that
most/all backup vendors will switch to this plugin method, at least for
supported setups.

best regards
Thomas


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 14076 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Thomas Lamprecht <t.lamprecht@proxmox.com>, "pve-devel@lists.proxmox.com" <pve-devel@lists.proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>
Subject: RE: About PVE Backup Integration Guide
Date: Wed, 5 Mar 2025 06:36:29 +0000
Message-ID: <PH0PR20MB4520446B9B012DE3352A8C1F98CB2@PH0PR20MB4520.namprd20.prod.outlook.com>

Thank you, Thomas, for your prompt response to this.



We would like to know following things:

  1.  How much part of the plugins method is done?
  2.  How soon we can start the integration?
  3.  Is there any plugin method's documentation already available? If yes, could you please share?



Regards,

Prashant



-----Original Message-----
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Sent: 04 March 2025 10:07 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; pve-devel@lists.proxmox.com
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: About PVE Backup Integration Guide





CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.







Hello Prashant,



Am 04.03.25 um 16:47 schrieb Prashant Patil:

> Hello All,

> Hope this email finds you well.

>

> At Arctera/Veritas, we are exploring the addition of support for data protection of Proxmox environments within our backup product. We would like to know if there is a detailed backup/restore API guide available that could serve as a good starting point for our engineering team. Could you please assist us with this? We have already reviewed the REST APIs.



Thanks for your interest in supporting Proxmox VE.



We are in progress of adding a way where external backup provider can provide a plugin to hook safely and correctly into our backup APIs, see:



https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.proxmox.com%2Fpve-devel%2F20241114150754.374376-1-f.ebner%40proxmox.com%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7C1816d6bad0c4419d245a08dd5b3adcd0%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638767030578307107%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=X3vGHTDdU8eqfj77XLGcn1myfqvNhPJPaum5dAJcw2Q%3D&reserved=0<https://lore.proxmox.com/pve-devel/20241114150754.374376-1-f.ebner@proxmox.com/>



This includes two example plugins to showcase how it would be used.



We would be open to fast-track this effort for inclusion with feedback from backup vendors. While there are other ways to hook one in, these are not nice and can cause problems - so in the long term we expect that most/all backup vendors will switch to this plugin method, at least for supported setups.



best regards

Thomas



[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 05.03.25 um 07:36 schrieb Prashant Patil via pve-devel:
>   1.  How much part of the plugins method is done?

It is an initial, functional version of the API.

>   2.  How soon we can start the integration?

The patches were not applied yet, feedback is welcome. As Thomas already
said "We would be open to fast-track this effort for inclusion with
feedback from backup vendors". I'll see if I can get around to post a
rebased version this week.

>   3.  Is there any plugin method's documentation already available? If yes, could you please share?

It's part of the patches, see:
https://lore.proxmox.com/pve-devel/20241114150754.374376-10-f.ebner@proxmox.com/

>   1.  How to get allocated blocks/sectors from the snapshot of the raw format virtual disk attached to a running VM (disk present on the block based storage such as lvm, lvm-thin, etc)?
>   2.  How to get allocated blocks/sectors from the snapshot of the vmdk format virtual disk attached to a running VM?
>   3.  As per below link, certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?

The block tracking is ideally done via QEMU, then you don't require any
special features for the underlying storages. You can also do dirty
tracking for incremental backups. See the following for how this is done
internally with the proposed API:
https://lore.proxmox.com/pve-devel/20241114150754.374376-4-f.ebner@proxmox.com/
https://lore.proxmox.com/pve-devel/20241114150754.374376-5-f.ebner@proxmox.com/

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 14809 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Mon, 17 Mar 2025 07:02:47 +0000
Message-ID: <PH0PR20MB4520C688E38C97D5DE5FC25B98DF2@PH0PR20MB4520.namprd20.prod.outlook.com>

> The block tracking is ideally done via QEMU, then you don't require any special features for the underlying storages.

Yes, we were able to get sector map of disks present on 'Directory' storage type. However, for other storage types such as lvm, lvm-thin, zfs which supports raw disk format, here we could get entire disk as allocated which is not the case in real. I could not find much information on this, hence wanted to know whether this by-design behaviour or we are missing something? Pasting output of qemu-img map below for the 2 GB disk on zfs. Have also tried getting map over ndb but got the same result. Is there anything that we are missing here?

root@be-proxmox1:/dev/pve# qemu-img map -f raw --output=json /dev/zvol/zfs1/vm-105-disk-2
[{ "start": 0, "length": 2147483648, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 0}]


> certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?

As mentioned earlier, we have found few storage devices which does not support snapshot, but have found that we can take individual disk snapshot through 'blockdev-snapshot-sync'. If we have to take backup of the VM, then are we supposed to use this command to snapshot all VM disks?

Thanks
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 10 March 2025 02:44 PM
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Prashant Patil <Prashant.Gamepatil@veritas.com>; Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 05.03.25 um 07:36 schrieb Prashant Patil via pve-devel:
>   1.  How much part of the plugins method is done?

It is an initial, functional version of the API.

>   2.  How soon we can start the integration?

The patches were not applied yet, feedback is welcome. As Thomas already said "We would be open to fast-track this effort for inclusion with feedback from backup vendors". I'll see if I can get around to post a rebased version this week.

>   3.  Is there any plugin method's documentation already available? If yes, could you please share?

It's part of the patches, see:
https://lore.proxmox.com/pve-devel/20241114150754.374376-10-f.ebner@proxmox.com/

>   1.  How to get allocated blocks/sectors from the snapshot of the raw format virtual disk attached to a running VM (disk present on the block based storage such as lvm, lvm-thin, etc)?
>   2.  How to get allocated blocks/sectors from the snapshot of the vmdk format virtual disk attached to a running VM?
>   3.  As per below link, certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?

The block tracking is ideally done via QEMU, then you don't require any special features for the underlying storages. You can also do dirty tracking for incremental backups. See the following for how this is done internally with the proposed API:
https://lore.proxmox.com/pve-devel/20241114150754.374376-4-f.ebner@proxmox.com/
https://lore.proxmox.com/pve-devel/20241114150754.374376-5-f.ebner@proxmox.com/

Best Regards,
Fiona


This message was sent by an employee of Arctera.


[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 17.03.25 um 08:02 schrieb Prashant Patil:
>> The block tracking is ideally done via QEMU, then you don't require any special features for the underlying storages.
> 
> Yes, we were able to get sector map of disks present on 'Directory' storage type. However, for other storage types such as lvm, lvm-thin, zfs which supports raw disk format, here we could get entire disk as allocated which is not the case in real. I could not find much information on this, hence wanted to know whether this by-design behaviour or we are missing something? Pasting output of qemu-img map below for the 2 GB disk on zfs. Have also tried getting map over ndb but got the same result. Is there anything that we are missing here?
> 
> root@be-proxmox1:/dev/pve# qemu-img map -f raw --output=json /dev/zvol/zfs1/vm-105-disk-2
> [{ "start": 0, "length": 2147483648, "depth": 0, "present": true, "zero": false, "data": true, "compressed": false, "offset": 0}]

That sounds like you want to roll your own external solution instead of
using the backup provider API that is currently being developed. With
that API, you get the images to be backed-up as as NBD exports. For
incremental backup, you can read the dirty bitmap. This can also be done
via NBD. Again, then you don't need to worry about the underlying
storage layer at all to support certain features.

>> certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?
> 
> As mentioned earlier, we have found few storage devices which does not support snapshot, but have found that we can take individual disk snapshot through 'blockdev-snapshot-sync'. If we have to take backup of the VM, then are we supposed to use this command to snapshot all VM disks?

Backups in QEMU do not use explicit snapshots. A copy-before-write
filter is inserted on top of the source disk in QEMU's block graph. When
new guest writes happen during backup, old data is first copied away to
a fleecing image (or for regular backup directly to the backup target).
The backup provider API then also inserts a snapshot-access node that is
exported via NBD and allows reading the data from the time of the backup
in a consistent fashion (hence "snapshot-access", it's a
virtual/implicit snapshot).

See also this diagram [0].

> [guest]                   [NBD export]
>    |                            |
>    | root                       | root
>    v                 file       v
> [copy-before-write]<------[snapshot-access]
>    |           |
>    | file      | target
>    v           v
> [active-disk] [temp.qcow2]

Best Regards,
Fiona

[0]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg876056.html


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 15642 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Mon, 17 Mar 2025 13:30:58 +0000
Message-ID: <PH0PR20MB4520276B3A7061528854180598DF2@PH0PR20MB4520.namprd20.prod.outlook.com>

> That sounds like you want to roll your own external solution instead of using the backup provider API that is currently being developed.
We are happy to integrate with backup provider APIs if they are stable and ready for the integration. On this same thread, I had asked few questions about APIs timelines. Could you please help us to get that info?

> Backups in QEMU do not use explicit snapshots. A copy-before-write filter is inserted on top of the source disk in QEMU's block graph.
What is the API to create copy-before-write snapshot of the disk image? And does it support all formats and storage devices?

Thanks
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 17 March 2025 04:23 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 17.03.25 um 08:02 schrieb Prashant Patil:
>> The block tracking is ideally done via QEMU, then you don't require any special features for the underlying storages.
>
> Yes, we were able to get sector map of disks present on 'Directory' storage type. However, for other storage types such as lvm, lvm-thin, zfs which supports raw disk format, here we could get entire disk as allocated which is not the case in real. I could not find much information on this, hence wanted to know whether this by-design behaviour or we are missing something? Pasting output of qemu-img map below for the 2 GB disk on zfs. Have also tried getting map over ndb but got the same result. Is there anything that we are missing here?
>
> root@be-proxmox1:/dev/pve# qemu-img map -f raw --output=json
> /dev/zvol/zfs1/vm-105-disk-2 [{ "start": 0, "length": 2147483648,
> "depth": 0, "present": true, "zero": false, "data": true,
> "compressed": false, "offset": 0}]

That sounds like you want to roll your own external solution instead of using the backup provider API that is currently being developed. With that API, you get the images to be backed-up as as NBD exports. For incremental backup, you can read the dirty bitmap. This can also be done via NBD. Again, then you don't need to worry about the underlying storage layer at all to support certain features.

>> certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?
>
> As mentioned earlier, we have found few storage devices which does not support snapshot, but have found that we can take individual disk snapshot through 'blockdev-snapshot-sync'. If we have to take backup of the VM, then are we supposed to use this command to snapshot all VM disks?

Backups in QEMU do not use explicit snapshots. A copy-before-write filter is inserted on top of the source disk in QEMU's block graph. When new guest writes happen during backup, old data is first copied away to a fleecing image (or for regular backup directly to the backup target).
The backup provider API then also inserts a snapshot-access node that is exported via NBD and allows reading the data from the time of the backup in a consistent fashion (hence "snapshot-access", it's a virtual/implicit snapshot).

See also this diagram [0].

> [guest]                   [NBD export]
>    |                            |
>    | root                       | root
>    v                 file       v
> [copy-before-write]<------[snapshot-access]
>    |           |
>    | file      | target
>    v           v
> [active-disk] [temp.qcow2]

Best Regards,
Fiona

[0]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg876056.html


This message was sent by an employee of Arctera.


[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 17646 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Tue, 18 Mar 2025 08:59:08 +0000
Message-ID: <PH0PR20MB45209D60E2A2A691D643A3C998DE2@PH0PR20MB4520.namprd20.prod.outlook.com>

We have tried the steps mentioned here to create copy-before-write snapshot - https://www.mail-archive.com/qemu-devel@nongnu.org/msg876056.html

However, one of the commands 'blockdev-replace' fails with below error:
{"error": {"class": "CommandNotFound", "desc": "The command blockdev-replace has not been found"}}

Is the command available in any specific qemu version? Is there any other command that we should be using instead if blockdev-replace?

qemu version:
===========
root@be-proxmox1:/dev/pve# /usr/bin/qemu-system-x86_64 --version
QEMU emulator version 9.0.2 (pve-qemu-kvm_9.0.2-4)

pve version:
==========
root@be-proxmox1:/dev/pve# pveversion -v
proxmox-ve: 8.3.0 (running kernel: 6.8.12-4-pve)
pve-manager: 8.3.0 (running version: 8.3.0/c1689ccb1065a83b)

Thanks
Prashant

-----Original Message-----
From: Prashant Patil <Prashant.Gamepatil@veritas.com>
Sent: 17 March 2025 07:01 PM
To: Fiona Ebner <f.ebner@proxmox.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide

> That sounds like you want to roll your own external solution instead of using the backup provider API that is currently being developed.
We are happy to integrate with backup provider APIs if they are stable and ready for the integration. On this same thread, I had asked few questions about APIs timelines. Could you please help us to get that info?

> Backups in QEMU do not use explicit snapshots. A copy-before-write filter is inserted on top of the source disk in QEMU's block graph.
What is the API to create copy-before-write snapshot of the disk image? And does it support all formats and storage devices?

Thanks
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 17 March 2025 04:23 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 17.03.25 um 08:02 schrieb Prashant Patil:
>> The block tracking is ideally done via QEMU, then you don't require any special features for the underlying storages.
>
> Yes, we were able to get sector map of disks present on 'Directory' storage type. However, for other storage types such as lvm, lvm-thin, zfs which supports raw disk format, here we could get entire disk as allocated which is not the case in real. I could not find much information on this, hence wanted to know whether this by-design behaviour or we are missing something? Pasting output of qemu-img map below for the 2 GB disk on zfs. Have also tried getting map over ndb but got the same result. Is there anything that we are missing here?
>
> root@be-proxmox1:/dev/pve# qemu-img map -f raw --output=json
> /dev/zvol/zfs1/vm-105-disk-2 [{ "start": 0, "length": 2147483648,
> "depth": 0, "present": true, "zero": false, "data": true,
> "compressed": false, "offset": 0}]

That sounds like you want to roll your own external solution instead of using the backup provider API that is currently being developed. With that API, you get the images to be backed-up as as NBD exports. For incremental backup, you can read the dirty bitmap. This can also be done via NBD. Again, then you don't need to worry about the underlying storage layer at all to support certain features.

>> certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?
>
> As mentioned earlier, we have found few storage devices which does not support snapshot, but have found that we can take individual disk snapshot through 'blockdev-snapshot-sync'. If we have to take backup of the VM, then are we supposed to use this command to snapshot all VM disks?

Backups in QEMU do not use explicit snapshots. A copy-before-write filter is inserted on top of the source disk in QEMU's block graph. When new guest writes happen during backup, old data is first copied away to a fleecing image (or for regular backup directly to the backup target).
The backup provider API then also inserts a snapshot-access node that is exported via NBD and allows reading the data from the time of the backup in a consistent fashion (hence "snapshot-access", it's a virtual/implicit snapshot).

See also this diagram [0].

> [guest]                   [NBD export]
>    |                            |
>    | root                       | root
>    v                 file       v
> [copy-before-write]<------[snapshot-access]
>    |           |
>    | file      | target
>    v           v
> [active-disk] [temp.qcow2]

Best Regards,
Fiona

[0]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg876056.html


This message was sent by an employee of Arctera.


[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 17615 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Thu, 20 Mar 2025 08:34:27 +0000
Message-ID: <PH0PR20MB452029AD1F1821EEB2285EB198D82@PH0PR20MB4520.namprd20.prod.outlook.com>

Hello Fiona and Thomas,
Could you please assist in resolving the query mentioned below?

Thanks
Prashant

-----Original Message-----
From: Prashant Patil
Sent: 18 March 2025 02:29 PM
To: Fiona Ebner <f.ebner@proxmox.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide

We have tried the steps mentioned here to create copy-before-write snapshot - https://www.mail-archive.com/qemu-devel@nongnu.org/msg876056.html

However, one of the commands 'blockdev-replace' fails with below error:
{"error": {"class": "CommandNotFound", "desc": "The command blockdev-replace has not been found"}}

Is the command available in any specific qemu version? Is there any other command that we should be using instead if blockdev-replace?

qemu version:
===========
root@be-proxmox1:/dev/pve# /usr/bin/qemu-system-x86_64 --version QEMU emulator version 9.0.2 (pve-qemu-kvm_9.0.2-4)

pve version:
==========
root@be-proxmox1:/dev/pve# pveversion -v
proxmox-ve: 8.3.0 (running kernel: 6.8.12-4-pve)
pve-manager: 8.3.0 (running version: 8.3.0/c1689ccb1065a83b)

Thanks
Prashant

-----Original Message-----
From: Prashant Patil <Prashant.Gamepatil@veritas.com>
Sent: 17 March 2025 07:01 PM
To: Fiona Ebner <f.ebner@proxmox.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide

> That sounds like you want to roll your own external solution instead of using the backup provider API that is currently being developed.
We are happy to integrate with backup provider APIs if they are stable and ready for the integration. On this same thread, I had asked few questions about APIs timelines. Could you please help us to get that info?

> Backups in QEMU do not use explicit snapshots. A copy-before-write filter is inserted on top of the source disk in QEMU's block graph.
What is the API to create copy-before-write snapshot of the disk image? And does it support all formats and storage devices?

Thanks
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 17 March 2025 04:23 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 17.03.25 um 08:02 schrieb Prashant Patil:
>> The block tracking is ideally done via QEMU, then you don't require any special features for the underlying storages.
>
> Yes, we were able to get sector map of disks present on 'Directory' storage type. However, for other storage types such as lvm, lvm-thin, zfs which supports raw disk format, here we could get entire disk as allocated which is not the case in real. I could not find much information on this, hence wanted to know whether this by-design behaviour or we are missing something? Pasting output of qemu-img map below for the 2 GB disk on zfs. Have also tried getting map over ndb but got the same result. Is there anything that we are missing here?
>
> root@be-proxmox1:/dev/pve# qemu-img map -f raw --output=json
> /dev/zvol/zfs1/vm-105-disk-2 [{ "start": 0, "length": 2147483648,
> "depth": 0, "present": true, "zero": false, "data": true,
> "compressed": false, "offset": 0}]

That sounds like you want to roll your own external solution instead of using the backup provider API that is currently being developed. With that API, you get the images to be backed-up as as NBD exports. For incremental backup, you can read the dirty bitmap. This can also be done via NBD. Again, then you don't need to worry about the underlying storage layer at all to support certain features.

>> certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?
>
> As mentioned earlier, we have found few storage devices which does not support snapshot, but have found that we can take individual disk snapshot through 'blockdev-snapshot-sync'. If we have to take backup of the VM, then are we supposed to use this command to snapshot all VM disks?

Backups in QEMU do not use explicit snapshots. A copy-before-write filter is inserted on top of the source disk in QEMU's block graph. When new guest writes happen during backup, old data is first copied away to a fleecing image (or for regular backup directly to the backup target).
The backup provider API then also inserts a snapshot-access node that is exported via NBD and allows reading the data from the time of the backup in a consistent fashion (hence "snapshot-access", it's a virtual/implicit snapshot).

See also this diagram [0].

> [guest]                   [NBD export]
>    |                            |
>    | root                       | root
>    v                 file       v
> [copy-before-write]<------[snapshot-access]
>    |           |
>    | file      | target
>    v           v
> [active-disk] [temp.qcow2]

Best Regards,
Fiona

[0]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg876056.html


This message was sent by an employee of Arctera.


[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 18.03.25 um 09:59 schrieb Prashant Patil:
> We have tried the steps mentioned here to create copy-before-write snapshot - https://www.mail-archive.com/qemu-devel@nongnu.org/msg876056.html
> 
> However, one of the commands 'blockdev-replace' fails with below error:
> {"error": {"class": "CommandNotFound", "desc": "The command blockdev-replace has not been found"}}
> 
> Is the command available in any specific qemu version? Is there any other command that we should be using instead if blockdev-replace?

No, it has not yet been implemented neither upstream nor downstream. The
latest proposed iteration is here [0].

> We are happy to integrate with backup provider APIs if they are stable and ready for the integration. On this same thread, I had asked few questions about APIs timelines. Could you please help us to get that info?

Thomas already answered this in the very first response:

> We would be open to fast-track this effort for inclusion with feedback
> from backup vendors. While there are other ways to hook one in, these
> are not nice and can cause problems – so in the long term we expect that
> most/all backup vendors will switch to this plugin method, at least for
> supported setups.

I sent v5 of the patch series today [1]. Again, I would highly recommend
to try and develop your solution with that API for proper integration.
Otherwise, it's 100% up to you not to cause conflicts with the Proxmox
VE system with your backup solution, and naturally, we cannot give you
much help with development (or later with debugging user issues) of a
fully separate backup stack.

It would be appreciated if you could give us feedback and tell us what
parts of it work and what parts might be difficult to work with for your
scenario. Then we might still be able to adapt to your requirements.
Adapting things retroactively means that there is less freedom to change
the interface, so now is the best chance.

[0]:
https://lore.kernel.org/qemu-devel/20240626115350.405778-1-vsementsov@yandex-team.ru/
[1]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-1-f.ebner@proxmox.com/T/

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 15204 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Mon, 24 Mar 2025 05:20:08 +0000
Message-ID: <PH0PR20MB4520584B52872E16277A621198A42@PH0PR20MB4520.namprd20.prod.outlook.com>

Thanks Fiona.
We would like to do plugin integration POC. So, for that to get started, we would like to know various interfaces of these plugins. Is there any detailed API documentation available for these plugins? Also, do you have any test app code which integrates with the plugins to try out?

Thanks
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 21 March 2025 07:43 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 18.03.25 um 09:59 schrieb Prashant Patil:
> We have tried the steps mentioned here to create copy-before-write
> snapshot -
> https://www/.
> mail-archive.com%2Fqemu-devel%40nongnu.org%2Fmsg876056.html&data=05%7C
> 02%7CPrashant.Gamepatil%40veritas.com%7Ce32cfc43ab294539a7ad08dd68828e
> 0b%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638781632166431273%7CU
> nknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiO
> iJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=c8XOfpg
> JBATIhPvDspiA9dK0QqqCg%2Bymes9WpKZJLmQ%3D&reserved=0
>
> However, one of the commands 'blockdev-replace' fails with below error:
> {"error": {"class": "CommandNotFound", "desc": "The command
> blockdev-replace has not been found"}}
>
> Is the command available in any specific qemu version? Is there any other command that we should be using instead if blockdev-replace?

No, it has not yet been implemented neither upstream nor downstream. The latest proposed iteration is here [0].

> We are happy to integrate with backup provider APIs if they are stable and ready for the integration. On this same thread, I had asked few questions about APIs timelines. Could you please help us to get that info?

Thomas already answered this in the very first response:

> We would be open to fast-track this effort for inclusion with feedback
> from backup vendors. While there are other ways to hook one in, these
> are not nice and can cause problems - so in the long term we expect
> that most/all backup vendors will switch to this plugin method, at
> least for supported setups.

I sent v5 of the patch series today [1]. Again, I would highly recommend to try and develop your solution with that API for proper integration.
Otherwise, it's 100% up to you not to cause conflicts with the Proxmox VE system with your backup solution, and naturally, we cannot give you much help with development (or later with debugging user issues) of a fully separate backup stack.

It would be appreciated if you could give us feedback and tell us what parts of it work and what parts might be difficult to work with for your scenario. Then we might still be able to adapt to your requirements.
Adapting things retroactively means that there is less freedom to change the interface, so now is the best chance.

[0]:
https://lore.kernel.org/qemu-devel/20240626115350.405778-1-vsementsov@yandex-team.ru/
[1]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-1-f.ebner@proxmox.com/T/

Best Regards,
Fiona


This message was sent by an employee of Arctera.


[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 24.03.25 um 06:20 schrieb Prashant Patil:
> We would like to do plugin integration POC. So, for that to get started, we would like to know various interfaces of these plugins. Is there any detailed API documentation available for these plugins? Also, do you have any test app code which integrates with the plugins to try out?

The cover letter of the patch series [0] mentions:
> A backup provider needs to implement a storage plugin as well as a
> backup provider plugin. The storage plugin is for integration in
> Proxmox VE's front-end, so users can manage the backups via
> UI/API/CLI. The backup provider plugin is for interfacing with the
> backup provider's backend to integrate backup and restore with that
> backend into Proxmox VE.

We unfortunately do not have a proper documentation for the storage
plugin API yet, some colleagues are currently working on that. But if
the storage only needs to support backups, you don't need to implement
some of the methods (e.g. for volume snapshots) and don't need much
complexity. See the BackupProviderDirExamplePlugin.pm in [1] for a
simple example of which methods are important. A second example, that
also involves handling secrets (but also not fully complete), is the
BorgBackupPlugin.pm in [2]. Of course, the backup mechanism doesn't need
to be configurable in your case, you can choose one for VMs and one for
LXCs.

The backup provider plugin serves as the bridge between Proxmox VE and
your backup server/storage. The API for this is documented in the
Base.pm plugin in [3] with Perl's POD documentation format. You can read
it by running the command:
perldoc src/PVE/BackupProvider/Plugin/Base.pm
Or some alternative way to view POD documentation. Examples can be found
in [1][2] too.

If you have questions about specific parts of the API (both storage
plugin or backup provider), feel free to ask.

There are pre-built packages with the series incorporated available at
[0] with SHA256SUM [1].

[0]: http://download.proxmox.com/temp/backup-provider-api-v5/
[1]:
> b30ef35447310e4f92e6ed434fce902160a76b64b54432a43696b853e2f073ad  libpve-common-perl_8.2.9+backupproviderapiv5_all.deb
> e2f3b9d2217bbb0bb75c94a73be1a9b76cda688028234fb529f4a25d1006a4ad  libpve-storage-perl_8.3.4+backupproviderapiv5_all.deb
> efb4cb6c3928e20a5c297533e44da93f913b474f7dfb27274c822071890b8975  pve-container_5.2.5+backupproviderapiv5_all.deb
> 138324eafd889a5e90e69c9b5aedd9df3ed7db09be7305d979bdd5a306e1858f  pve-manager_8.3.5+backupproviderapiv5_all.deb
> bdb871a0c57347607aec955f771cdb7ad24245cb1c4bf2517fbb78d734228ccd  pve-qemu-kvm_9.2.0-2+backupproviderapiv5_amd64.deb
> 12eb9034819364d0057b80d827515832589ee0848934b27be4039dbbc78cb060  qemu-server_8.3.9+backupproviderapiv5_amd64.deb
> 04ca13927bfd70800c1855022642e6db88b05b711b44877a120a84949d208c2e  qemu-server-dbgsym_8.3.9+backupproviderapiv5_amd64.deb

Best Regards,
Fiona

[0]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-1-f.ebner@proxmox.com/
[1]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-14-f.ebner@proxmox.com/
[2]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-15-f.ebner@proxmox.com/
[3]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-10-f.ebner@proxmox.com/


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 16035 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Tue, 25 Mar 2025 10:23:39 +0000
Message-ID: <PH0PR20MB4520FF39B329114AB45B2ADD98A72@PH0PR20MB4520.namprd20.prod.outlook.com>

> There are pre-built packages with the series incorporated available at [0] with SHA256SUM [1].

Does that mean the backup provider plugin is not yet available as part of any qemu/proxmox release? If not in the released version, then which version would have these changes?

Thanks
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 25 March 2025 02:29 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 24.03.25 um 06:20 schrieb Prashant Patil:
> We would like to do plugin integration POC. So, for that to get started, we would like to know various interfaces of these plugins. Is there any detailed API documentation available for these plugins? Also, do you have any test app code which integrates with the plugins to try out?

The cover letter of the patch series [0] mentions:
> A backup provider needs to implement a storage plugin as well as a
> backup provider plugin. The storage plugin is for integration in
> Proxmox VE's front-end, so users can manage the backups via
> UI/API/CLI. The backup provider plugin is for interfacing with the
> backup provider's backend to integrate backup and restore with that
> backend into Proxmox VE.

We unfortunately do not have a proper documentation for the storage plugin API yet, some colleagues are currently working on that. But if the storage only needs to support backups, you don't need to implement some of the methods (e.g. for volume snapshots) and don't need much complexity. See the BackupProviderDirExamplePlugin.pm in [1] for a simple example of which methods are important. A second example, that also involves handling secrets (but also not fully complete), is the BorgBackupPlugin.pm in [2]. Of course, the backup mechanism doesn't need to be configurable in your case, you can choose one for VMs and one for LXCs.

The backup provider plugin serves as the bridge between Proxmox VE and your backup server/storage. The API for this is documented in the Base.pm plugin in [3] with Perl's POD documentation format. You can read it by running the command:
perldoc src/PVE/BackupProvider/Plugin/Base.pm
Or some alternative way to view POD documentation. Examples can be found in [1][2] too.

If you have questions about specific parts of the API (both storage plugin or backup provider), feel free to ask.

There are pre-built packages with the series incorporated available at [0] with SHA256SUM [1].

[0]: http://download.proxmox.com/temp/backup-provider-api-v5/
[1]:
> b30ef35447310e4f92e6ed434fce902160a76b64b54432a43696b853e2f073ad
> libpve-common-perl_8.2.9+backupproviderapiv5_all.deb
> e2f3b9d2217bbb0bb75c94a73be1a9b76cda688028234fb529f4a25d1006a4ad
> libpve-storage-perl_8.3.4+backupproviderapiv5_all.deb
> efb4cb6c3928e20a5c297533e44da93f913b474f7dfb27274c822071890b8975
> pve-container_5.2.5+backupproviderapiv5_all.deb
> 138324eafd889a5e90e69c9b5aedd9df3ed7db09be7305d979bdd5a306e1858f
> pve-manager_8.3.5+backupproviderapiv5_all.deb
> bdb871a0c57347607aec955f771cdb7ad24245cb1c4bf2517fbb78d734228ccd
> pve-qemu-kvm_9.2.0-2+backupproviderapiv5_amd64.deb
> 12eb9034819364d0057b80d827515832589ee0848934b27be4039dbbc78cb060
> qemu-server_8.3.9+backupproviderapiv5_amd64.deb
> 04ca13927bfd70800c1855022642e6db88b05b711b44877a120a84949d208c2e
> qemu-server-dbgsym_8.3.9+backupproviderapiv5_amd64.deb

Best Regards,
Fiona

[0]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-1-f.ebner@proxmox.com/
[1]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-14-f.ebner@proxmox.com/
[2]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-15-f.ebner@proxmox.com/
[3]:
https://lore.proxmox.com/pve-devel/20250321134852.103871-10-f.ebner@proxmox.com/


This message was sent by an employee of Arctera.


[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 25.03.25 um 11:23 schrieb Prashant Patil:
>> There are pre-built packages with the series incorporated available at [0] with SHA256SUM [1].
> 
> Does that mean the backup provider plugin is not yet available as part of any qemu/proxmox release? If not in the released version, then which version would have these changes?

Yes, that's what it means. There is no ETA for when it will be released
exactly, but it might be in one of the next versions. But again, if you
wait until it is actually released, it will be harder for us to
retroactively incorporate changes/feedback that would benefit your use case.

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 17387 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Wed, 26 Mar 2025 15:06:49 +0000
Message-ID: <PH0PR20MB452084FCE3C6D7DD55BDEBC398A62@PH0PR20MB4520.namprd20.prod.outlook.com>

Hi Fiona,
Which proxmox version those patches shared earlier are based off? And are there any specific steps/sequence to install those patches. I know we can install package through "dpkg -i".

I'm having this PVE version installed currently:

root@rs-proxmox2:/home/ppatil/patch# pveversion -v
proxmox-ve: 8.3.0 (running kernel: 6.8.12-4-pve)
pve-manager: 8.3.0 (running version: 8.3.0/c1689ccb1065a83b)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.12-4
proxmox-kernel-6.8.12-4-pve-signed: 6.8.12-4
ceph-fuse: 17.2.7-pve3
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.4
libpve-access-control: 8.2.0
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.10
libpve-cluster-perl: 8.0.10
libpve-common-perl: 8.2.9+backupproviderapiv5
libpve-guest-common-perl: 5.1.6
libpve-http-server-perl: 5.1.2
libpve-network-perl: 0.10.0
libpve-rs-perl: 0.9.0
libpve-storage-perl: 8.2.9
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.5.0-1
proxmox-backup-client: 3.2.9-1
proxmox-backup-file-restore: 3.2.9-1
proxmox-firewall: 0.6.0
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.3.1
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.1
pve-cluster: 8.0.10
pve-container: 5.2.2
pve-docs: 8.3.1
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.2
pve-firewall: 5.1.0
pve-firmware: 3.14-1
pve-ha-manager: 4.0.6
pve-i18n: 3.3.1
pve-qemu-kvm: 9.0.2-4
pve-xtermjs: 5.3.0-3
qemu-server: 8.3.0
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.6-pve1
root@rs-proxmox2:/home/ppatil/patch#

Thanks
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 25 March 2025 04:10 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 25.03.25 um 11:23 schrieb Prashant Patil:
>> There are pre-built packages with the series incorporated available at [0] with SHA256SUM [1].
>
> Does that mean the backup provider plugin is not yet available as part of any qemu/proxmox release? If not in the released version, then which version would have these changes?

Yes, that's what it means. There is no ETA for when it will be released exactly, but it might be in one of the next versions. But again, if you wait until it is actually released, it will be harder for us to retroactively incorporate changes/feedback that would benefit your use case.

Best Regards,
Fiona


This message was sent by an employee of Arctera.

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 26.03.25 um 16:06 schrieb Prashant Patil:
> Hi Fiona,
> Which proxmox version those patches shared earlier are based off? And are there any specific steps/sequence to install those patches. I know we can install package through "dpkg -i".

The packages are based on the current master branches in git [0] which
can be slightly ahead of packages available in public repositories. You
should upgrade to the latest available versions from the test
repository[1][2] and then there should be no issues installing the
packages on top. If there are, please let us know! Using
apt install ./*.deb
is recommended, because dpkg by itself will not check for dependencies.

[0]: https://git.proxmox.com/?o=age
[1]: https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_test_repo
[2]:
https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#system_software_updates

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 13892 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Thu, 27 Mar 2025 09:05:21 +0000
Message-ID: <PH0PR20MB452056ED55E525FF89127CFA98A12@PH0PR20MB4520.namprd20.prod.outlook.com>

Ok, Thanks. I will give it a try.

We also have a few questions related to the plugin. So, to fast-track the discussion, can we join over a zoom/teams call based on your availability?

Thanks
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 27 March 2025 02:20 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 26.03.25 um 16:06 schrieb Prashant Patil:
> Hi Fiona,
> Which proxmox version those patches shared earlier are based off? And are there any specific steps/sequence to install those patches. I know we can install package through "dpkg -i".

The packages are based on the current master branches in git [0] which can be slightly ahead of packages available in public repositories. You should upgrade to the latest available versions from the test repository[1][2] and then there should be no issues installing the packages on top. If there are, please let us know! Using apt install ./*.deb is recommended, because dpkg by itself will not check for dependencies.

[0]: https://git.proxmox.com/?o=age
[1]: https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_test_repo
[2]:
https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#system_software_updates

Best Regards,
Fiona


This message was sent by an employee of Arctera.


[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 27.03.25 um 10:05 schrieb Prashant Patil:
> We also have a few questions related to the plugin. So, to fast-track the discussion, can we join over a zoom/teams call based on your availability?

You can post the questions here. In written form, it should be faster to
communicate the precise issues and there will be a reference to look
things up later.

P.S. This is my personal opinion and is not the main reason that the
offer is declined, but you should consider using
privacy-respecting/FLOSS conferencing software.

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 28.03.25 um 10:31 schrieb Prashant Patil:
> Made changes as mentioned in earlier email to use pvetest repo. With
> those changes, 'apt-get update' used to fail, had to disable enterprise
> repositories as per the screenshot below. Wit this, I was able to run
> 'apt-get update', but 'apt-get dist-upgrade' failed with below mentioned
> errors.  Am I missing something?

Yes, you cannot use the enterprise repository without a valid
subscription. But you don't need it, because for testing you only need
the newer package versions from the pvetest repository.

> 
> root@rs-proxmox2:/home/ppatil/patch# apt-get update
> 
> Hit:1 http://security.debian.org <http://security.debian.org/> bookworm-
> security InRelease
> 
> Hit:2 http://ftp.debian.org/debian <http://ftp.debian.org/debian>
> bookworm InRelease
> 
> Hit:3 http://ftp.debian.org/debian <http://ftp.debian.org/debian>
> bookworm-updates InRelease
> 
> Hit:4 http://download.proxmox.com/debian/pve <http://
> download.proxmox.com/debian/pve> bookworm InRelease
> 
> Reading package lists... Done
> 
> root@rs-proxmox2:/home/ppatil/patch# apt-get dist-upgrade
> 
> Reading package lists... Done
> 
> Building dependency tree... Done
> 
> Reading state information... Done
> 
> You might want to run 'apt --fix-broken install' to correct these.
> 
> The following packages have unmet dependencies:
> 
> pve-manager : Depends: proxmox-widget-toolkit (>= 4.3.5) but 4.3.1 is
> installed
> 
>                Recommends: pve-nvidia-vgpu-helper but it is not installed
> 
> E: Unmet dependencies. Try 'apt --fix-broken install' with no packages
> (or specify a solution).

What if you run "apt --fix-broken install"? If that doesn't help, please
share the output of "apt policy proxmox-widget-toolkit".

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil]

Finally, we can install packages from pvtest repository.



We have gone through plugins POD material; we have few questions from very little understanding about plugins:

  1.  Storage plugin
     *   What is the main purpose of storage plugin?
     *   Do we need to implement our own storage plugin? Which all functions we need to implement?



  1.  Backup Provider plugin
     *   What is the main purpose of backup provider plugin?
     *   Do we need to implement our own backup provider plugin? Which all functions we need to implement?
     *   Which model this provider refers to - push model or pull model?
     *   It looks like we have two main callback functions - job_hook() and backup_hook(). Who calls these functions and when do they get called?
     *   backup_get_mechanism() - This function needs to be called by external backup product?
     *   backup_vm() - This function needs to be called by external backup product? Does it support all disk formats and backed storages?



Regards

Prashant



-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 28 March 2025 03:39 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide





CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.







Am 28.03.25 um 10:31 schrieb Prashant Patil:

> Made changes as mentioned in earlier email to use pvetest repo. With

> those changes, 'apt-get update' used to fail, had to disable

> enterprise repositories as per the screenshot below. Wit this, I was

> able to run 'apt-get update', but 'apt-get dist-upgrade' failed with

> below mentioned errors.  Am I missing something?



Yes, you cannot use the enterprise repository without a valid subscription. But you don't need it, because for testing you only need the newer package versions from the pvetest repository.



>

> root@rs-proxmox2:/home/ppatil/patch# apt-get update

>

> Hit:1

> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsecur<http://secur/>

> ity.debian.org%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7C22

> eef66b30184f68a35908dd6de07f32%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%

> 7C0%7C638787533180521868%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRyd

> WUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3

> D%7C0%7C%7C%7C&sdata=vb1NBcqDiAbi27Nx3cqzeDKxORPDWcjIbyRPAJD7%2F%2Fo%3

> D&reserved=0

> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsecu

> rity.debian.org%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7C2

> 2eef66b30184f68a35908dd6de07f32%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0

> %7C0%7C638787533180543007%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRy

> dWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%

> 3D%7C0%7C%7C%7C&sdata=sCDQdb8XGvfU9OAOFG1%2Fc5014xIO%2BLiugHGEq4wwxK0%

> 3D&reserved=0> bookworm- security InRelease

>

> Hit:2

> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fftp.d<http://ftp.d/>

> ebian.org%2Fdebian&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7C2

> 2eef66b30184f68a35908dd6de07f32%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0

> %7C0%7C638787533180557220%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRy

> dWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%

> 3D%7C0%7C%7C%7C&sdata=7xnYO0xJvi5G3o94gbL58Vaz6flZHGGtyHhOitEpZ%2BM%3D

> &reserved=0

> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fftp.

> debian.org%2Fdebian&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7C

> 22eef66b30184f68a35908dd6de07f32%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C

> 0%7C0%7C638787533180570399%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnR

> ydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D

> %3D%7C0%7C%7C%7C&sdata=bByGA2Ze6EbZsdLiWYahc9AHpdHyW%2BIDpbstAa787ds%3

> D&reserved=0>

> bookworm InRelease

>

> Hit:3

> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fftp.d<http://ftp.d/>

> ebian.org%2Fdebian&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7C2

> 2eef66b30184f68a35908dd6de07f32%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0

> %7C0%7C638787533180583014%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRy

> dWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%

> 3D%7C0%7C%7C%7C&sdata=jATUPh0YUYbM55T6Sw72furRWfveOEBv3QRBEFY3E3g%3D&r

> eserved=0

> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fftp.

> debian.org%2Fdebian&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7C

> 22eef66b30184f68a35908dd6de07f32%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C

> 0%7C0%7C638787533180596413%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnR

> ydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D

> %3D%7C0%7C%7C%7C&sdata=cXzlasxJlKn7Gyh0hKb30B2h2XoAXwIFMa%2F0ys%2BdQ6Y

> %3D&reserved=0>

> bookworm-updates InRelease

>

> Hit:4

> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdownl<http://downl/>

> oad.proxmox.com%2Fdebian%2Fpve&data=05%7C02%7CPrashant.Gamepatil%40ver

> itas.com%7C22eef66b30184f68a35908dd6de07f32%7Cfc8e13c0422c4c55b3eaca31

> 8e6cac32%7C0%7C0%7C638787533180608575%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0

> eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIl

> dUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Uep04glbhsK5pPQ%2FmHGpZMn5blwu9%2BYM

> ntwPGIxfyW4%3D&reserved=0 <http:// download.proxmox.com/debian/pve>

> bookworm InRelease

>

> Reading package lists... Done

>

> root@rs-proxmox2:/home/ppatil/patch# apt-get dist-upgrade

>

> Reading package lists... Done

>

> Building dependency tree... Done

>

> Reading state information... Done

>

> You might want to run 'apt --fix-broken install' to correct these.

>

> The following packages have unmet dependencies:

>

> pve-manager : Depends: proxmox-widget-toolkit (>= 4.3.5) but 4.3.1 is

> installed

>

>                Recommends: pve-nvidia-vgpu-helper but it is not

> installed

>

> E: Unmet dependencies. Try 'apt --fix-broken install' with no packages

> (or specify a solution).



What if you run "apt --fix-broken install"? If that doesn't help, please share the output of "apt policy proxmox-widget-toolkit".



Best Regards,

Fiona




This message was sent by an employee of Arctera.
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Note there is a v6 of the patch series now [0].

Am 01.04.25 um 09:02 schrieb Prashant Patil:
> We have gone through plugins POD material; we have few questions from
> very little understanding about plugins:
> 
>  1. Storage plugin
>      1. What is the main purpose of storage plugin?

The storage plugin is needed, so that users can configure your backup
server as a backup storage in Proxmox VE, i.e. select it when
configuring a backup job or starting a backup, list the backups and
issue restore operations.

>      2. Do we need to implement our own storage plugin? Which all
>         functions we need to implement?

Yes. Look at the example plugins [1][2] to see which methods are required.

>  2. Backup Provider plugin
>      1. What is the main purpose of backup provider plugin?

This is used by the Proxmox VE backend to interface with your backup
server/solution during backup and restore tasks.

>      2. Do we need to implement our own backup provider plugin? Which
>         all functions we need to implement?

Yes. You need to implement all methods, but you can decide which backup
and restore mechanisms you want to implement. One for VMs and one for
containers.

>      3. Which model this provider refers to - push model or pull model?

Not sure what you mean here. The Proxmox VE backend will call your
plugins methdos during backup. It's up to you how exactly you handle the
data.

>      4. It looks like we have two main callback functions – job_hook()
>         and backup_hook(). Who calls these functions and when do they
>         get called?

They got replaced by job_init(), job_cleanup(), backup_init(),
backup_cleanup() etc. in v6 of the series, but the purpose is still the
very same. I'll just quote from [3]:

> In Proxmox VE, a backup job consists of backup tasks for individual
> guests. There are methods for initialization and cleanup of the job,
> i.e. job_init() and job_cleanup() and for each guest backup, i.e.
> backup_init() and backup_cleanup().
> 
> The backup_get_mechanism() method is used to decide on the backup
> mechanism. Currently, 'file-handle' or 'nbd' for VMs, and 'directory'
> for containers is possible. The method also let's the plugin indicate
> whether to use a bitmap for incremental VM backup or not. It is enough
> to implement one mechanism for VMs and one mechanism for containers.
> 
> Next, there are methods for backing up the guest's configuration and
> data, backup_vm() for VM backup and backup_container() for container
> backup, with the latter running
> 
> Finally, some helpers like getting the provider name or volume ID for
> the backup target, as well as for handling the backup log.
> 
> The backup transaction looks as follows:
> 
> First, job_init() is called that can be used to check backup server
> availability and prepare the connection. Then for each guest
> backup_init() followed by backup_vm() or backup_container() and finally
> backup_cleanup(). Afterwards job_cleanup() is called. For containers,
> there is an additional backup_container_prepare() call while still
> privileged. The actual backup_container() call happens as the
> (unprivileged) container root user, so that the file owner and group IDs
> match the container's perspective.

See also the documentation for each method for more details.

>      5. backup_get_mechanism() – This function needs to be called by
>         external backup product?

No, all the functions are called by the Proxmox VE backup stack during
backup.

>      6. backup_vm() – This function needs to be called by external
>         backup product? Does it support all disk formats and backed
>         storages?

No. The Proxmox VE backend will call your implementation of this method
during backup of a VM guest.

[0]:
https://lore.proxmox.com/pve-devel/20250331132020.105324-1-f.ebner@proxmox.com/
[1]: BackupProviderDirExamplePlugin.pm in
https://lore.proxmox.com/pve-devel/20250331132020.105324-17-f.ebner@proxmox.com/
[2]: BorgBackupPlugin.pm in
https://lore.proxmox.com/pve-devel/20250331132020.105324-18-f.ebner@proxmox.com/
[3]:
https://lore.proxmox.com/pve-devel/20250331132020.105324-13-f.ebner@proxmox.com/

Best Regards,
Fiona



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 01.04.25 um 10:07 schrieb Fiona Ebner:
> Note there is a v6 of the patch series now [0].

Also available as packages now:

http://download.proxmox.com/temp/backup-provider-api-v6/

SHA256SUMS:

> 08064b8f20997090e31854711ff6fa99154c8c7894c39da12333c9d3d71e42c6  libpve-common-perl_8.2.9+backupproviderapiv6_all.deb
> 43a3d280073685c1327d774223fdbc4fbcf465306f859d47351b4a3366e362b6  libpve-storage-perl_8.3.3+backupproviderapiv6_all.deb
> 9ecc30ebf52490552799a7aab4bb64024897446bc7425e8a0242320fd6481eb8  pve-container_5.2.4+backupproviderapiv6_all.deb
> 67fc180e59da5538ef591c6fe7a84fe50673aa3fa16428d488ecafa4f5983686  pve-manager_8.3.5+backupproviderapiv6_all.deb
> 890c2c21d5e1a974c3e3be01cbb428c24371baf794a4dcabdd101daf7d50ff27  pve-qemu-kvm_9.2.0-2+backupproviderapiv6_amd64.deb
> 5349fa5af94d1ed769160a4f7c6acfb180ccf7ac399937b32590c8a0c01a4ae4  qemu-server_8.3.8+backupproviderapiv6_amd64.deb

> Am 01.04.25 um 09:02 schrieb Prashant Patil:
>>      2. Do we need to implement our own backup provider plugin? Which
>>         all functions we need to implement?
> 
> Yes. You need to implement all methods, but you can decide which backup
> and restore mechanisms you want to implement. One for VMs and one for
> containers.

I suggest you take a look at the examples for these too:

https://lore.proxmox.com/pve-devel/20250331132020.105324-17-f.ebner@proxmox.com/
https://lore.proxmox.com/pve-devel/20250331132020.105324-18-f.ebner@proxmox.com/

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 19895 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Tue, 1 Apr 2025 08:52:59 +0000
Message-ID: <PH0PR20MB45208AD2006C9D22F1EBF0A998AC2@PH0PR20MB4520.namprd20.prod.outlook.com>

Thanks for answering all questions.



When does Proxmox VE backend calls functions from the plugin? The backup/restore tasks/jobs are configured in our Backup Solution hence this question.



Thanks

Prashant



-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 01 April 2025 01:37 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide





CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.







Note there is a v6 of the patch series now [0].



Am 01.04.25 um 09:02 schrieb Prashant Patil:

> We have gone through plugins POD material; we have few questions from

> very little understanding about plugins:

>

>  1. Storage plugin

>      1. What is the main purpose of storage plugin?



The storage plugin is needed, so that users can configure your backup server as a backup storage in Proxmox VE, i.e. select it when configuring a backup job or starting a backup, list the backups and issue restore operations.



>      2. Do we need to implement our own storage plugin? Which all

>         functions we need to implement?



Yes. Look at the example plugins [1][2] to see which methods are required.



>  2. Backup Provider plugin

>      1. What is the main purpose of backup provider plugin?



This is used by the Proxmox VE backend to interface with your backup server/solution during backup and restore tasks.



>      2. Do we need to implement our own backup provider plugin? Which

>         all functions we need to implement?



Yes. You need to implement all methods, but you can decide which backup and restore mechanisms you want to implement. One for VMs and one for containers.



>      3. Which model this provider refers to - push model or pull model?



Not sure what you mean here. The Proxmox VE backend will call your plugins methdos during backup. It's up to you how exactly you handle the data.



>      4. It looks like we have two main callback functions - job_hook()

>         and backup_hook(). Who calls these functions and when do they

>         get called?



They got replaced by job_init(), job_cleanup(), backup_init(),

backup_cleanup() etc. in v6 of the series, but the purpose is still the very same. I'll just quote from [3]:



> In Proxmox VE, a backup job consists of backup tasks for individual

> guests. There are methods for initialization and cleanup of the job,

> i.e. job_init() and job_cleanup() and for each guest backup, i.e.

> backup_init() and backup_cleanup().

>

> The backup_get_mechanism() method is used to decide on the backup

> mechanism. Currently, 'file-handle' or 'nbd' for VMs, and 'directory'

> for containers is possible. The method also let's the plugin indicate

> whether to use a bitmap for incremental VM backup or not. It is enough

> to implement one mechanism for VMs and one mechanism for containers.

>

> Next, there are methods for backing up the guest's configuration and

> data, backup_vm() for VM backup and backup_container() for container

> backup, with the latter running

>

> Finally, some helpers like getting the provider name or volume ID for

> the backup target, as well as for handling the backup log.

>

> The backup transaction looks as follows:

>

> First, job_init() is called that can be used to check backup server

> availability and prepare the connection. Then for each guest

> backup_init() followed by backup_vm() or backup_container() and

> finally backup_cleanup(). Afterwards job_cleanup() is called. For

> containers, there is an additional backup_container_prepare() call

> while still privileged. The actual backup_container() call happens as

> the

> (unprivileged) container root user, so that the file owner and group

> IDs match the container's perspective.



See also the documentation for each method for more details.



>      5. backup_get_mechanism() - This function needs to be called by

>         external backup product?



No, all the functions are called by the Proxmox VE backup stack during backup.



>      6. backup_vm() - This function needs to be called by external

>         backup product? Does it support all disk formats and backed

>         storages?



No. The Proxmox VE backend will call your implementation of this method during backup of a VM guest.



[0]:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.proxmox.com%2Fpve-devel%2F20250331132020.105324-1-f.ebner%40proxmox.com%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7Ce8e90928ffbc4175ad0e08dd70f43d58%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638790916502870977%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ZlxofWsel85kigkOw2RmCWnnQwrUyRatI1f6PAoXz%2BE%3D&reserved=0<https://lore.proxmox.com/pve-devel/20250331132020.105324-1-f.ebner@proxmox.com/>

[1]: BackupProviderDirExamplePlugin.pm in

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.proxmox.com%2Fpve-devel%2F20250331132020.105324-17-f.ebner%40proxmox.com%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7Ce8e90928ffbc4175ad0e08dd70f43d58%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638790916502890406%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=hkNs3aKQpbYf7EE%2Fx4wIGrYBl%2B6UR9gaK5uLWPr6uvA%3D&reserved=0<https://lore.proxmox.com/pve-devel/20250331132020.105324-17-f.ebner@proxmox.com/>

[2]: BorgBackupPlugin.pm in

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.proxmox.com%2Fpve-devel%2F20250331132020.105324-18-f.ebner%40proxmox.com%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7Ce8e90928ffbc4175ad0e08dd70f43d58%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638790916502904656%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=V2SpaW1FCtH8FN0i%2BuZxHU2W6T1gTEv%2FA4J%2FMOpwOQk%3D&reserved=0<https://lore.proxmox.com/pve-devel/20250331132020.105324-18-f.ebner@proxmox.com/>

[3]:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.proxmox.com%2Fpve-devel%2F20250331132020.105324-13-f.ebner%40proxmox.com%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7Ce8e90928ffbc4175ad0e08dd70f43d58%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638790916502917419%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Y6V325iQiGAKzra%2BQm1LUzb4ELtn3VaRFaYjRGyKmpM%3D&reserved=0<https://lore.proxmox.com/pve-devel/20250331132020.105324-13-f.ebner@proxmox.com/>



Best Regards,

Fiona






This message was sent by an employee of Arctera.

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 01.04.25 um 10:52 schrieb Prashant Patil:
> When does Proxmox VE backend calls functions from the plugin? The
> backup/restore tasks/jobs are configured in our Backup Solution hence
> this question.

During running a Proxmox VE backup job or backup/restore invocation via
API. If you manage the jobs on the server side, you will need to start
the backup via the corresponding API call:

For backup:
https://pve.proxmox.com/pve-docs/api-viewer/#/nodes/{node}/vzdump

For restore of VM, POST call with 'archive' argument being the volume ID
used by your storage plugin implementation:
https://pve.proxmox.com/pve-docs/api-viewer/#/nodes/{node}/qemu

For restore of VM, POST call with 'restore' argument set to true and
'ostemplate' argument being the volume ID used by your storage plugin
implementation:
https://pve.proxmox.com/pve-docs/api-viewer/#/nodes/{node}/lxc

See also:

https://pve.proxmox.com/pve-docs/chapter-pveum.html#pveum_tokens

However, note that users will still be able to configure jobs/issue
backups/restores for your backup storage on the Proxmox VE-side. That is
where the backups are taken/needed for restore, so the provider API is
designed to be fully integrated on the Proxmox VE-side.

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 16611 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Tue, 1 Apr 2025 10:59:02 +0000
Message-ID: <PH0PR20MB45206E718C1F7B1C4660D00098AC2@PH0PR20MB4520.namprd20.prod.outlook.com>

Just for clarity - The backup/restore jobs will be configured on the Proxmox VE side, the job gets triggered on Proxmox VE side and that is where the provider API comes into picture to take backups to specified storage. Is this understanding correct?



Thanks

Prashant



-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 01 April 2025 02:35 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide





CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.







Am 01.04.25 um 10:52 schrieb Prashant Patil:

> When does Proxmox VE backend calls functions from the plugin? The

> backup/restore tasks/jobs are configured in our Backup Solution hence

> this question.



During running a Proxmox VE backup job or backup/restore invocation via API. If you manage the jobs on the server side, you will need to start the backup via the corresponding API call:



For backup:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpve.proxmox.com%2Fpve-docs%2Fapi-viewer%2F%23%2Fnodes%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7Cf394f36683574eb0cdf508dd70fc561c%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638790951278353942%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=u%2FmQXat93EOwXF2O9nW%2Bmivf7sujiSnacY4LISZm%2FU4%3D&reserved=0{node}/vzdump<https://pve.proxmox.com/pve-docs/api-viewer/#/nodes/>



For restore of VM, POST call with 'archive' argument being the volume ID used by your storage plugin implementation:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpve.proxmox.com%2Fpve-docs%2Fapi-viewer%2F%23%2Fnodes%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7Cf394f36683574eb0cdf508dd70fc561c%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638790951278382101%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=zuUEmWNhwWiygheMU27uC8xmjmuGWkrbI6Q3SbG51AU%3D&reserved=0{node}/qemu<https://pve.proxmox.com/pve-docs/api-viewer/#/nodes/>



For restore of VM, POST call with 'restore' argument set to true and 'ostemplate' argument being the volume ID used by your storage plugin

implementation:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpve.proxmox.com%2Fpve-docs%2Fapi-viewer%2F%23%2Fnodes%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7Cf394f36683574eb0cdf508dd70fc561c%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638790951278398953%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=rs1rXinYnZBKZL6AhpK6NgV0OTlXpZgeJPaMLYfeceo%3D&reserved=0{node}/lxc<https://pve.proxmox.com/pve-docs/api-viewer/#/nodes/>



See also:



https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpve.proxmox.com%2Fpve-docs%2Fchapter-pveum.html%23pveum_tokens&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7Cf394f36683574eb0cdf508dd70fc561c%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638790951278413524%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=OvRxJgULYhqnbrseGE1nvti7cBLnvK6O4q%2FNXkQY2%2BQ%3D&reserved=0<https://pve.proxmox.com/pve-docs/chapter-pveum.html#pveum_tokens>



However, note that users will still be able to configure jobs/issue backups/restores for your backup storage on the Proxmox VE-side. That is where the backups are taken/needed for restore, so the provider API is designed to be fully integrated on the Proxmox VE-side.



Best Regards,

Fiona




This message was sent by an employee of Arctera.

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 01.04.25 um 12:59 schrieb Prashant Patil:
> Just for clarity - The backup/restore jobs will be configured on the
> Proxmox VE side, the job gets triggered on Proxmox VE side and that is
> where the provider API comes into picture to take backups to specified
> storage. Is this understanding correct?

Yes, the API is designed for the integrated backup/restore tasks on the
Proxmox VE side. The implementation of the API methods will serve as a
bridge to your backup server.

If you really want, nothing prevents to manage your own jobs from the
server side. You'd just need to issue the relevant API calls via the
Proxmox VE REST API, e.g.:
1. Your job on the server side decides to run a backup.
2. It sends the 'vzdump' API call to the Proxmox VE node.
3. The Proxmox VE node will call the methods of your backup provider API
implementation.
4. Inside these methods, you get access to the data that can be
transferred to your backup server.

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 17118 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Tue, 1 Apr 2025 11:54:50 +0000
Message-ID: <PH0PR20MB452007780D6055D15DFD565098AC2@PH0PR20MB4520.namprd20.prod.outlook.com>

Instead of making such multiple hops, can we have APIs exposed by Proxmox VE which we can use? E.g. API to get consistent snapshot-access to a virtual disk.

This way we have better control on monitoring jobs, API contracts are decided, it is also easy to troubleshoot issues on each side, etc. Let me know your thoughts on this. With Proxmox VE REST APIs, we already have several APIs for nodes/VMs management.



Regards

Prashant



-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 01 April 2025 04:49 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide





CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.







Am 01.04.25 um 12:59 schrieb Prashant Patil:

> Just for clarity - The backup/restore jobs will be configured on the

> Proxmox VE side, the job gets triggered on Proxmox VE side and that is

> where the provider API comes into picture to take backups to specified

> storage. Is this understanding correct?



Yes, the API is designed for the integrated backup/restore tasks on the Proxmox VE side. The implementation of the API methods will serve as a bridge to your backup server.



If you really want, nothing prevents to manage your own jobs from the server side. You'd just need to issue the relevant API calls via the Proxmox VE REST API, e.g.:

1. Your job on the server side decides to run a backup.

2. It sends the 'vzdump' API call to the Proxmox VE node.

3. The Proxmox VE node will call the methods of your backup provider API implementation.

4. Inside these methods, you get access to the data that can be transferred to your backup server.



Best Regards,

Fiona




This message was sent by an employee of Arctera.

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 01.04.25 um 13:54 schrieb Prashant Patil:
> Instead of making such multiple hops, can we have APIs exposed by
> Proxmox VE which we can use? E.g. API to get consistent snapshot-access
> to a virtual disk.
>
> This way we have better control on monitoring jobs, API contracts are
> decided, it is also easy to troubleshoot issues on each side, etc. Let
> me know your thoughts on this. With Proxmox VE REST APIs, we already
> have several APIs for nodes/VMs management.

There are no endpoints in the Proxmox VE REST API for setting up backup
access to a guest or similar. And the backup provider API is simply not
designed for this, but to be fully integrated in Proxmox VE. What you
can do is leverage Proxmox VE REST API for the backup/restore tasks, but
not just for parts of such tasks.

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 16072 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>, Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Wed, 2 Apr 2025 06:20:19 +0000
Message-ID: <PH0PR20MB4520109F59C59C3976298E8798AF2@PH0PR20MB4520.namprd20.prod.outlook.com>

Basically, the backup/restore tasks will be managed through our backup solution. So, all we needed is VM config which can be fetched through PVE REST API and a API to get access to consistent disk image which can be then transferred to backup storage that we support. But it looks like this is not possible with current design of provider APIs.

Is there any chance of developing new library which can provide access to consistent disk image in near future?

Regards,
Prashant

-----Original Message-----
From: Fiona Ebner <f.ebner@proxmox.com>
Sent: 01 April 2025 09:09 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>; Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide


CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.



Am 01.04.25 um 13:54 schrieb Prashant Patil:
> Instead of making such multiple hops, can we have APIs exposed by
> Proxmox VE which we can use? E.g. API to get consistent
> snapshot-access to a virtual disk.
>
> This way we have better control on monitoring jobs, API contracts are
> decided, it is also easy to troubleshoot issues on each side, etc. Let
> me know your thoughts on this. With Proxmox VE REST APIs, we already
> have several APIs for nodes/VMs management.

There are no endpoints in the Proxmox VE REST API for setting up backup access to a guest or similar. And the backup provider API is simply not designed for this, but to be fully integrated in Proxmox VE. What you can do is leverage Proxmox VE REST API for the backup/restore tasks, but not just for parts of such tasks.

Best Regards,
Fiona


This message was sent by an employee of Arctera.

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Thomas Lamprecht]

Am 02.04.25 um 08:20 schrieb Prashant Patil:
> Basically, the backup/restore tasks will be managed through our backup solution. So, all we needed is VM config which can be fetched through PVE REST API and a API to get access to consistent disk image which can be then transferred to backup storage that we support. But it looks like this is not possible with current design of provider APIs.

It is though, just create PVE backup task for the VMs in question for a Job
you manage and a Plugin following the mechanism here gives you exactly that.

> Is there any chance of developing new library which can provide access to consistent disk image in near future?

Above already provides that, we enforce this running in our standard backup
environment to ensure it's can be as native as integrated solutions and
that all ACL and resource management is followed correctly while being able
to create an efficient implementation for both sides, ours and the running
VMs and your solution pulling of the data, e.g. including more advanced
techniques like changed block tracking and getting the allocation map to
skip over holes, allowing one to dramatically speed up the backup process.
Same holds for containers, which we also support and a backup solution
should also implement to provide a complete experience that actually nicely
integrates into PVE, and not feels tacked on.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 16942 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Thomas Lamprecht <t.lamprecht@proxmox.com>, Fiona Ebner <f.ebner@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: RE: [pve-devel] About PVE Backup Integration Guide
Date: Wed, 2 Apr 2025 09:04:35 +0000
Message-ID: <PH0PR20MB4520725FF56C2F89E6A6EDDE98AF2@PH0PR20MB4520.namprd20.prod.outlook.com>

Ok, will give plugins a try and see if we can achieve backup/restore management from our backup solution.



  1.  Can we just implement backup provider plugin and not storage plugin? We can build storage plugin in subsequent releases.
  2.  What are the steps to configure/deploy plugins?
  3.  Can I use the example plugins provided to test? Which one exactly? What are the involved steps to test this?
  4.  Is this the REST API to be used to create backup task? POST /api2/json/nodes/{node}/vzdump



Regards

Prashant



-----Original Message-----
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Sent: 02 April 2025 12:45 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com>; Fiona Ebner <f.ebner@proxmox.com>; Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>
Subject: Re: [pve-devel] About PVE Backup Integration Guide





CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.







Am 02.04.25 um 08:20 schrieb Prashant Patil:

> Basically, the backup/restore tasks will be managed through our backup solution. So, all we needed is VM config which can be fetched through PVE REST API and a API to get access to consistent disk image which can be then transferred to backup storage that we support. But it looks like this is not possible with current design of provider APIs.



It is though, just create PVE backup task for the VMs in question for a Job you manage and a Plugin following the mechanism here gives you exactly that.



> Is there any chance of developing new library which can provide access to consistent disk image in near future?



Above already provides that, we enforce this running in our standard backup environment to ensure it's can be as native as integrated solutions and that all ACL and resource management is followed correctly while being able to create an efficient implementation for both sides, ours and the running VMs and your solution pulling of the data, e.g. including more advanced techniques like changed block tracking and getting the allocation map to skip over holes, allowing one to dramatically speed up the backup process.

Same holds for containers, which we also support and a backup solution should also implement to provide a complete experience that actually nicely integrates into PVE, and not feels tacked on.




This message was sent by an employee of Arctera.

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Fiona Ebner]

Am 02.04.25 um 11:04 schrieb Prashant Patil:
> Ok, will give plugins a try and see if we can achieve backup/restore
> management from our backup solution.  
> 
>  1. Can we just implement backup provider plugin and not storage plugin?
>     We can build storage plugin in subsequent releases.

In theory, all you need to implement in the storage plugin is the
new_backup_provider() and parse_volname() methods and an empty/dummy
implementation for everything else. But you really don't need much else
to have proper integration as well, just listing backups, storage usage
status, etc. See the example plugins.

>  2. What are the steps to configure/deploy plugins?

Easiest is to build a Debian package and provide that to your users.

>  3. Can I use the example plugins provided to test? Which one exactly?
>     What are the involved steps to test this?

You can use them to get an idea of how the implementation should look
like. Since communicating with your backup sever and your architecture
will look different from those basic examples, the concrete
implementation will most likely not be useful to you however.

Also, like other code in Proxmox VE, it is licensed under AGPLv3, so
copying non-trivial amounts of it means that copyleft needs to be respected.

>  4. Is this the REST API to be used to create backup task? POST /api2/
>     json/nodes/{node}/vzdump

Yes.

Best Regards,
Fiona


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] About PVE Backup Integration Guide

[Prashant Patil via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 15401 bytes --]

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
To: Thomas Lamprecht <t.lamprecht@proxmox.com>, "pve-devel@lists.proxmox.com" <pve-devel@lists.proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>, Jason Voneberstein <Jason.vonEberstein@veritas.com>, Sudhir Subbarao <Sudhir.Subbarao@veritas.com>
Subject: RE: About PVE Backup Integration Guide
Date: Wed, 5 Mar 2025 15:44:26 +0000
Message-ID: <PH0PR20MB452048F028DE3978CD3BB9BA98CB2@PH0PR20MB4520.namprd20.prod.outlook.com>

Hi Thomas,
We have few more questions in addition to those mentioned in the previous email.

  1.  How to get allocated blocks/sectors from the snapshot of the raw format virtual disk attached to a running VM (disk present on the block based storage such as lvm, lvm-thin, etc)?
  2.  How to get allocated blocks/sectors from the snapshot of the vmdk format virtual disk attached to a running VM?
  3.  As per below link, certain storage types do not support snapshots. In such cases, what is the recommended way to take backup of the running VM?

https://pve.proxmox.com/wiki/Storage

Could you kindly assist us in obtaining answers to these questions?

Thanks
Prashant

From: Prashant Patil <Prashant.Gamepatil@veritas.com>
Sent: 05 March 2025 12:06 PM
To: Thomas Lamprecht <t.lamprecht@proxmox.com>; pve-devel@lists.proxmox.com
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com>; Jason Voneberstein <Jason.vonEberstein@veritas.com>; Sudhir Subbarao <Sudhir.Subbarao@veritas.com>
Subject: RE: About PVE Backup Integration Guide


Thank you, Thomas, for your prompt response to this.



We would like to know following things:

  1.  How much part of the plugins method is done?
  2.  How soon we can start the integration?
  3.  Is there any plugin method's documentation already available? If yes, could you please share?



Regards,

Prashant



-----Original Message-----
From: Thomas Lamprecht <t.lamprecht@proxmox.com<mailto:t.lamprecht@proxmox.com>>
Sent: 04 March 2025 10:07 PM
To: Prashant Patil <Prashant.Gamepatil@veritas.com<mailto:Prashant.Gamepatil@veritas.com>>; pve-devel@lists.proxmox.com<mailto:pve-devel@lists.proxmox.com>
Cc: Anuradha Joshi <Anuradha.Joshi@veritas.com<mailto:Anuradha.Joshi@veritas.com>>; Jason Voneberstein <Jason.vonEberstein@veritas.com<mailto:Jason.vonEberstein@veritas.com>>
Subject: Re: About PVE Backup Integration Guide





CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you believe this is a phishing email, use the Report to Cybersecurity icon in Outlook.







Hello Prashant,



Am 04.03.25 um 16:47 schrieb Prashant Patil:

> Hello All,

> Hope this email finds you well.

>

> At Arctera/Veritas, we are exploring the addition of support for data protection of Proxmox environments within our backup product. We would like to know if there is a detailed backup/restore API guide available that could serve as a good starting point for our engineering team. Could you please assist us with this? We have already reviewed the REST APIs.



Thanks for your interest in supporting Proxmox VE.



We are in progress of adding a way where external backup provider can provide a plugin to hook safely and correctly into our backup APIs, see:



https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.proxmox.com%2Fpve-devel%2F20241114150754.374376-1-f.ebner%40proxmox.com%2F&data=05%7C02%7CPrashant.Gamepatil%40veritas.com%7C1816d6bad0c4419d245a08dd5b3adcd0%7Cfc8e13c0422c4c55b3eaca318e6cac32%7C0%7C0%7C638767030578307107%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=X3vGHTDdU8eqfj77XLGcn1myfqvNhPJPaum5dAJcw2Q%3D&reserved=0<https://lore.proxmox.com/pve-devel/20241114150754.374376-1-f.ebner@proxmox.com/>



This includes two example plugins to showcase how it would be used.



We would be open to fast-track this effort for inclusion with feedback from backup vendors. While there are other ways to hook one in, these are not nice and can cause problems - so in the long term we expect that most/all backup vendors will switch to this plugin method, at least for supported setups.



best regards

Thomas



[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel