all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH storage v7 0/9] support copying volumes between storages
@ 2025-03-11 14:23 Filip Schauer
  2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 1/9] storage migrate: remove remnant from rsync-based migration Filip Schauer
                   ` (8 more replies)
  0 siblings, 9 replies; 10+ messages in thread
From: Filip Schauer @ 2025-03-11 14:23 UTC (permalink / raw)
  To: pve-devel

Add the ability to copy a backup, ISO, container template, snippet, or
OVA/OVF between storages and nodes via an API method. Copying a VMA
backup to a Proxmox Backup Server requires the proxmox-vma-to-pbs
package to be installed. Currently only VMA backups can be copied to a
Proxmox Backup Server and copying backups from a Proxmox Backup Server
is currently not supported.

The method can be called from the PVE shell with `pvesm copy-volume`:

```
pvesm copy-volume <source volume> <target storage> [--target-node <node>] [--delete]
```

For example to copy a VMA backup to a Proxmox Backup Server:

```
pvesm copy-volume \
    local:backup/vzdump-qemu-100-2024_06_25-13_08_56.vma.zst pbs
```

Or copy a container template to another node and delete the source:

```
pvesm copy-volume \
    local:vztmpl/devuan-4.0-standard_4.0_amd64.tar.gz local \
    --target-node pvenode2 --delete
```

Or use curl to call the API method:

```
curl https://$APINODE:8006/api2/json/nodes/$SOURCENODE/storage/$SOURCESTORAGE/content/$SOURCEVOLUME \
    --insecure --cookie "$(<cookie)" -H "$(<csrftoken)" -X POST \
    --data-raw "target-storage=$TARGETSTORAGE&target-node=$TARGETNODE"
```

Changes since v6:
* introduce $vtype+meta export formats
* remove remnant from rsync-based migration
* avoid ssh when moving a volume locally
* rename 'move' API method to 'copy'
* factor out delete behavior into a helper in storage content API
* fix permission checks in 'copy' API method
* check that the source plugin is path-based before calling vma_to_pbs
* do not pass the source plugin to PVE::Storage::PBSPlugin::vma_to_pbs

Changes since v5:
* Resolve merge conflicts when applying patches 1/7 & 3/7 to current
  master (e5f4af47d083).

Changes since v4:
* Remove the volume_move subroutine, instead use storage_migrate for
  moving volumes between storages on the same node
* Avoid ssh when moving a volume between storages on the same node
* Add command completion to move-volume parameters
* Make the success message of move-volume less verbose for moves within
  the same node
* utf8 encode/decode backup notes during export/import
* Support the new "import" volume type
* Code cleanup
* Add descriptions to single line commit messages

Changes since v3:
* Split changes into multiple commits
* Remove superfluous parentheses from post-ifs
* Move vma_to_pbs branch from move_volume into its own helper inside
  PBSPlugin
* Use $! instead of $@ to retrieve unlink error in move_volume
* Also support content type 'rootdir'
* Rework permission checks on the move API method
* Fix permissions description on move API method
* Add error for unimplemented content types

Changes since v2:
* Specify permissions for move method
* Add success message to move method
* Limit the move method to non-vdisk volumes
* Check that source and target are not the same in the move method
* Remove the unnecessary movevolume method from pvesm and make the
  move-volume command call the move API method directly
* Fail when trying to move a protected volume with the delete option
  enabled, instead of ignoring the protection
* Change "not yet supported" to "not supported" in messages indicating
  unimplemented features
* Process auxiliary files first when moving a volume locally on a node
* Move a volume instead of copying it when trying to move a volume
  locally on a node with the delete option enabled.
* Use the more general `path` function instead of `filesystem_path` to
  get the path of a volume
* Loosen the required privileges to move an ISO or a container template,
  or when the delete option is not set.
* Move the volume_move sub from PVE::Storage to
  PVE::API2::Storage::Content since it is only used there.
* Explicitly check that storages are path-based in volume_move,
  except when moving a vma to a Proxmox Backup Server

Changes since v1:
* Rename pvesm command to move-volume
* Add a delete option to control whether the source volume should be
  kept
* Move the API method to the POST endpoint of
  /nodes/{node}/storage/{storage}/content/{volume}, replacing the
  experimental copy method that has not been used since its introduction
  in October 2011 883eeea6.
* Implement migrating volumes between nodes

Filip Schauer (9):
  storage migrate: remove remnant from rsync-based migration
  storage migrate: avoid ssh when moving a volume locally
  plugin: allow volume import of iso, snippets, vztmpl and import
  api: content: implement copying volumes between storages
  introduce $vtype+meta export formats
  api: content: support copying backups between path based storages
  storage: introduce decompress_archive_into_pipe helper
  support copying VMA backups to PBS
  pvesm: add a copy-volume command

 debian/control                  |   1 +
 src/PVE/API2/Storage/Content.pm | 133 +++++++++++++++-------
 src/PVE/CLI/pvesm.pm            |   2 +
 src/PVE/Storage.pm              | 101 +++++++++++------
 src/PVE/Storage/PBSPlugin.pm    |  59 ++++++++++
 src/PVE/Storage/Plugin.pm       | 190 +++++++++++++++++++++++++-------
 6 files changed, 368 insertions(+), 118 deletions(-)

-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2025-03-11 14:24 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-03-11 14:23 [pve-devel] [PATCH storage v7 0/9] support copying volumes between storages Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 1/9] storage migrate: remove remnant from rsync-based migration Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 2/9] storage migrate: avoid ssh when moving a volume locally Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 3/9] plugin: allow volume import of iso, snippets, vztmpl and import Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 4/9] api: content: implement copying volumes between storages Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 5/9] introduce $vtype+meta export formats Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 6/9] api: content: support copying backups between path based storages Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 7/9] storage: introduce decompress_archive_into_pipe helper Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 8/9] support copying VMA backups to PBS Filip Schauer
2025-03-11 14:23 ` [pve-devel] [PATCH storage v7 9/9] pvesm: add a copy-volume command Filip Schauer

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal