From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 95D071FF15E
for <inbox@lore.proxmox.com>; Tue, 11 Mar 2025 15:23:49 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 93AEA12220;
Tue, 11 Mar 2025 15:23:39 +0100 (CET)
From: Filip Schauer <f.schauer@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Tue, 11 Mar 2025 15:23:19 +0100
Message-Id: <20250311142328.112538-1-f.schauer@proxmox.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.169 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
POISEN_SPAM_PILL 0.1 Meta: its spam
POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes
POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH storage v7 0/9] support copying volumes between
storages
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Add the ability to copy a backup, ISO, container template, snippet, or
OVA/OVF between storages and nodes via an API method. Copying a VMA
backup to a Proxmox Backup Server requires the proxmox-vma-to-pbs
package to be installed. Currently only VMA backups can be copied to a
Proxmox Backup Server and copying backups from a Proxmox Backup Server
is currently not supported.
The method can be called from the PVE shell with `pvesm copy-volume`:
```
pvesm copy-volume <source volume> <target storage> [--target-node <node>] [--delete]
```
For example to copy a VMA backup to a Proxmox Backup Server:
```
pvesm copy-volume \
local:backup/vzdump-qemu-100-2024_06_25-13_08_56.vma.zst pbs
```
Or copy a container template to another node and delete the source:
```
pvesm copy-volume \
local:vztmpl/devuan-4.0-standard_4.0_amd64.tar.gz local \
--target-node pvenode2 --delete
```
Or use curl to call the API method:
```
curl https://$APINODE:8006/api2/json/nodes/$SOURCENODE/storage/$SOURCESTORAGE/content/$SOURCEVOLUME \
--insecure --cookie "$(<cookie)" -H "$(<csrftoken)" -X POST \
--data-raw "target-storage=$TARGETSTORAGE&target-node=$TARGETNODE"
```
Changes since v6:
* introduce $vtype+meta export formats
* remove remnant from rsync-based migration
* avoid ssh when moving a volume locally
* rename 'move' API method to 'copy'
* factor out delete behavior into a helper in storage content API
* fix permission checks in 'copy' API method
* check that the source plugin is path-based before calling vma_to_pbs
* do not pass the source plugin to PVE::Storage::PBSPlugin::vma_to_pbs
Changes since v5:
* Resolve merge conflicts when applying patches 1/7 & 3/7 to current
master (e5f4af47d083).
Changes since v4:
* Remove the volume_move subroutine, instead use storage_migrate for
moving volumes between storages on the same node
* Avoid ssh when moving a volume between storages on the same node
* Add command completion to move-volume parameters
* Make the success message of move-volume less verbose for moves within
the same node
* utf8 encode/decode backup notes during export/import
* Support the new "import" volume type
* Code cleanup
* Add descriptions to single line commit messages
Changes since v3:
* Split changes into multiple commits
* Remove superfluous parentheses from post-ifs
* Move vma_to_pbs branch from move_volume into its own helper inside
PBSPlugin
* Use $! instead of $@ to retrieve unlink error in move_volume
* Also support content type 'rootdir'
* Rework permission checks on the move API method
* Fix permissions description on move API method
* Add error for unimplemented content types
Changes since v2:
* Specify permissions for move method
* Add success message to move method
* Limit the move method to non-vdisk volumes
* Check that source and target are not the same in the move method
* Remove the unnecessary movevolume method from pvesm and make the
move-volume command call the move API method directly
* Fail when trying to move a protected volume with the delete option
enabled, instead of ignoring the protection
* Change "not yet supported" to "not supported" in messages indicating
unimplemented features
* Process auxiliary files first when moving a volume locally on a node
* Move a volume instead of copying it when trying to move a volume
locally on a node with the delete option enabled.
* Use the more general `path` function instead of `filesystem_path` to
get the path of a volume
* Loosen the required privileges to move an ISO or a container template,
or when the delete option is not set.
* Move the volume_move sub from PVE::Storage to
PVE::API2::Storage::Content since it is only used there.
* Explicitly check that storages are path-based in volume_move,
except when moving a vma to a Proxmox Backup Server
Changes since v1:
* Rename pvesm command to move-volume
* Add a delete option to control whether the source volume should be
kept
* Move the API method to the POST endpoint of
/nodes/{node}/storage/{storage}/content/{volume}, replacing the
experimental copy method that has not been used since its introduction
in October 2011 883eeea6.
* Implement migrating volumes between nodes
Filip Schauer (9):
storage migrate: remove remnant from rsync-based migration
storage migrate: avoid ssh when moving a volume locally
plugin: allow volume import of iso, snippets, vztmpl and import
api: content: implement copying volumes between storages
introduce $vtype+meta export formats
api: content: support copying backups between path based storages
storage: introduce decompress_archive_into_pipe helper
support copying VMA backups to PBS
pvesm: add a copy-volume command
debian/control | 1 +
src/PVE/API2/Storage/Content.pm | 133 +++++++++++++++-------
src/PVE/CLI/pvesm.pm | 2 +
src/PVE/Storage.pm | 101 +++++++++++------
src/PVE/Storage/PBSPlugin.pm | 59 ++++++++++
src/PVE/Storage/Plugin.pm | 190 +++++++++++++++++++++++++-------
6 files changed, 368 insertions(+), 118 deletions(-)
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel