* [pve-devel] [PATCH storage v8 1/9] storage migrate: remove remnant from rsync-based migration
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 2/9] storage migrate: avoid ssh when moving a volume locally Filip Schauer
` (8 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
rsync-based migration was replaced by import/export in commit
da72898cc65b ("migrate: only use import/export")
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/Storage.pm | 2 --
1 file changed, 2 deletions(-)
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 1dde2b7..fca308a 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -943,8 +943,6 @@ sub storage_migrate {
my $target_ip = $target_sshinfo->{ip};
my $ssh = PVE::SSHInfo::ssh_info_to_command($target_sshinfo);
- my $ssh_base = PVE::SSHInfo::ssh_info_to_command_base($target_sshinfo);
- local $ENV{RSYNC_RSH} = PVE::Tools::cmd2string($ssh_base);
if (!defined($opts->{snapshot})) {
$opts->{migration_snapshot} =
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH storage v8 2/9] storage migrate: avoid ssh when moving a volume locally
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 1/9] storage migrate: remove remnant from rsync-based migration Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 3/9] plugin: allow volume import of iso, snippets, vztmpl and import Filip Schauer
` (7 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
Avoid the overhead of SSH when $target_sshinfo is undefined. Instead
move a volume between storages on the same node.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/Storage.pm | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index fca308a..3467a90 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -940,10 +940,6 @@ sub storage_migrate {
my $target_volid = "${target_storeid}:${target_volname}";
- my $target_ip = $target_sshinfo->{ip};
-
- my $ssh = PVE::SSHInfo::ssh_info_to_command($target_sshinfo);
-
if (!defined($opts->{snapshot})) {
$opts->{migration_snapshot} =
storage_migrate_snapshot($cfg, $storeid, $opts->{with_snapshots});
@@ -962,13 +958,19 @@ sub storage_migrate {
my $format = $formats[0];
my $import_fn = '-'; # let pvesm import read from stdin per default
- if ($insecure) {
- my $net = $target_sshinfo->{network} // $target_sshinfo->{ip};
- $import_fn = "tcp://$net";
+ my $recv = [];
+
+ if (defined($target_sshinfo)) {
+ if ($insecure) {
+ my $net = $target_sshinfo->{network} // $target_sshinfo->{ip};
+ $import_fn = "tcp://$net";
+ }
+
+ my $ssh = PVE::SSHInfo::ssh_info_to_command($target_sshinfo);
+ push @$recv, (@$ssh, '--');
}
- my $recv =
- [@$ssh, '--', $volume_import_prepare->($target_volid, $format, $import_fn, $opts)->@*];
+ push @$recv, ($volume_import_prepare->($target_volid, $format, $import_fn, $opts)->@*);
my $new_volid;
my $pattern = volume_imported_message(undef, 1);
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH storage v8 3/9] plugin: allow volume import of iso, snippets, vztmpl and import
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 1/9] storage migrate: remove remnant from rsync-based migration Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 2/9] storage migrate: avoid ssh when moving a volume locally Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 4/9] api: content: implement copying volumes between storages Filip Schauer
` (6 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
Extend volume import functionality to support 'iso', 'snippets',
'vztmpl', and 'import' types, in addition to the existing support for
'images' and 'rootdir'. This is a prerequisite for the ability to move
ISOs, snippets and container templates between nodes.
Existing behavior for importing VM disks and container volumes remains
unchanged.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/Storage/Plugin.pm | 89 ++++++++++++++++++++++++---------------
1 file changed, 56 insertions(+), 33 deletions(-)
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 2291d72..ecf68c8 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -2039,7 +2039,7 @@ sub volume_export {
if ($format eq 'raw+size') {
die $err_msg if $with_snapshots || $file_format eq 'subvol';
write_common_header($fh, $size);
- if ($file_format eq 'raw') {
+ if ($file_format =~ /^(raw|ova|ovf)$/) {
run_command(
['dd', "if=$file", "bs=4k", "status=progress"],
output => '>&' . fileno($fh),
@@ -2085,14 +2085,14 @@ sub volume_export {
sub volume_export_formats {
my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
if ($scfg->{path} && !defined($snapshot) && !defined($base_snapshot)) {
- my $format = ($class->parse_volname($volname))[6];
+ my ($vtype, $format) = ($class->parse_volname($volname))[0, 6];
if ($with_snapshots) {
return ($format . '+size') if ($format eq 'qcow2' || $format eq 'vmdk');
return ();
}
return ('tar+size') if $format eq 'subvol';
- return ('raw+size');
+ return ('raw+size') if $vtype =~ /^(iso|snippets|vztmpl|import)$/;
}
return ();
}
@@ -2126,14 +2126,20 @@ sub volume_import {
# XXX: Should we bother with conversion routines at this level? This won't
# happen without manual CLI usage, so for now we just error out...
- die "cannot import format $format into a file of format $file_format\n"
- if $data_format ne $file_format && !($data_format eq 'tar' && $file_format eq 'subvol');
+ if (
+ ($vtype eq 'images' || $vtype eq 'rootdir')
+ && $data_format ne $file_format
+ && !($data_format eq 'tar' && $file_format eq 'subvol')
+ ) {
+ die "cannot import format $format into a file of format $file_format\n";
+ }
# Check for an existing file first since interrupting alloc_image doesn't
# free it.
my ($file) = $class->path($scfg, $volname, $storeid);
if (-e $file) {
- die "file '$file' already exists\n" if !$allow_rename;
+ die "file '$file' already exists\n"
+ if !$allow_rename || ($vtype ne 'images' && $vtype ne 'rootdir');
warn "file '$file' already exists - importing with a different name\n";
$name = undef;
}
@@ -2141,33 +2147,49 @@ sub volume_import {
my ($size) = read_common_header($fh);
$size = PVE::Storage::Common::align_size_up($size, 1024) / 1024;
- eval {
- my $allocname = $class->alloc_image($storeid, $scfg, $vmid, $file_format, $name, $size);
- my $oldname = $volname;
- $volname = $allocname;
- if (defined($name) && $allocname ne $oldname) {
- die "internal error: unexpected allocated name: '$allocname' != '$oldname'\n";
+ if ($vtype eq 'images' || $vtype eq 'rootdir') {
+ eval {
+ my $allocname =
+ $class->alloc_image($storeid, $scfg, $vmid, $file_format, $name, $size);
+ my $oldname = $volname;
+ $volname = $allocname;
+ if (defined($name) && $allocname ne $oldname) {
+ die "internal error: unexpected allocated name: '$allocname' != '$oldname'\n";
+ }
+ my ($file) = $class->path($scfg, $volname, $storeid)
+ or die "internal error: failed to get path to newly allocated volume $volname\n";
+ if ($data_format eq 'raw' || $data_format eq 'qcow2' || $data_format eq 'vmdk') {
+ run_command(
+ ['dd', "of=$file", 'conv=sparse', 'bs=64k'],
+ input => '<&' . fileno($fh),
+ );
+ } elsif ($data_format eq 'tar') {
+ run_command(
+ ['tar', @COMMON_TAR_FLAGS, '-C', $file, '-xf', '-'],
+ input => '<&' . fileno($fh),
+ );
+ } else {
+ die "volume import format '$format' not available for $class";
+ }
+ };
+ if (my $err = $@) {
+ eval { $class->free_image($storeid, $scfg, $volname, 0, $file_format) };
+ warn $@ if $@;
+ die $err;
}
- my ($file) = $class->path($scfg, $volname, $storeid)
- or die "internal error: failed to get path to newly allocated volume $volname\n";
- if ($data_format eq 'raw' || $data_format eq 'qcow2' || $data_format eq 'vmdk') {
- run_command(
- ['dd', "of=$file", 'conv=sparse', 'bs=64k'],
- input => '<&' . fileno($fh),
- );
- } elsif ($data_format eq 'tar') {
- run_command(
- ['tar', @COMMON_TAR_FLAGS, '-C', $file, '-xf', '-'],
- input => '<&' . fileno($fh),
- );
- } else {
- die "volume import format '$format' not available for $class";
+ } elsif (grep { $vtype eq $_ } qw(import iso snippets vztmpl)) {
+ eval {
+ run_command(['dd', "of=$file", 'conv=excl', 'bs=64k'], input => '<&' . fileno($fh));
+ };
+ if (my $err = $@) {
+ if (-e $file) {
+ eval { unlink($file) };
+ warn $@ if $@;
+ }
+ die $err;
}
- };
- if (my $err = $@) {
- eval { $class->free_image($storeid, $scfg, $volname, 0, $file_format) };
- warn $@ if $@;
- die $err;
+ } else {
+ die "importing volume of type '$vtype' not implemented\n";
}
return "$storeid:$volname";
@@ -2176,13 +2198,14 @@ sub volume_import {
sub volume_import_formats {
my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
if ($scfg->{path} && !defined($base_snapshot)) {
- my $format = ($class->parse_volname($volname))[6];
+ my ($vtype, $format) = ($class->parse_volname($volname))[0, 6];
+
if ($with_snapshots) {
return ($format . '+size') if ($format eq 'qcow2' || $format eq 'vmdk');
return ();
}
return ('tar+size') if $format eq 'subvol';
- return ('raw+size');
+ return ('raw+size') if $vtype =~ /^(iso|snippets|vztmpl|import)$/;
}
return ();
}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH storage v8 4/9] api: content: implement copying volumes between storages
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
` (2 preceding siblings ...)
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 3/9] plugin: allow volume import of iso, snippets, vztmpl and import Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-11-13 23:19 ` Thomas Lamprecht
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 5/9] introduce $vtype+meta export formats Filip Schauer
` (5 subsequent siblings)
9 siblings, 1 reply; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
Add the ability to copy an iso, snippet or vztmpl between storages and
nodes.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/API2/Storage/Content.pm | 131 ++++++++++++++++++++------------
1 file changed, 83 insertions(+), 48 deletions(-)
diff --git a/src/PVE/API2/Storage/Content.pm b/src/PVE/API2/Storage/Content.pm
index 1fe7303..c69b859 100644
--- a/src/PVE/API2/Storage/Content.pm
+++ b/src/PVE/API2/Storage/Content.pm
@@ -435,6 +435,21 @@ __PACKAGE__->register_method({
},
});
+my $volume_remove = sub {
+ my ($cfg, $volid) = @_;
+
+ my ($path, undef, $vtype) = PVE::Storage::path($cfg, $volid);
+
+ PVE::Storage::vdisk_free($cfg, $volid);
+ if (
+ $vtype eq 'backup'
+ && $path =~ /(.*\/vzdump-\w+-\d+-\d{4}_\d{2}_\d{2}-\d{2}_\d{2}_\d{2})[^\/]+$/
+ ) {
+ # Remove log file #318 and notes file #3972 if they still exist
+ PVE::Storage::archive_auxiliaries_remove($path);
+ }
+};
+
__PACKAGE__->register_method({
name => 'delete',
path => '{volume}',
@@ -493,15 +508,8 @@ __PACKAGE__->register_method({
}
my $worker = sub {
- PVE::Storage::vdisk_free($cfg, $volid);
+ &$volume_remove($cfg, $volid);
print "Removed volume '$volid'\n";
- if (
- $vtype eq 'backup'
- && $path =~ /(.*\/vzdump-\w+-\d+-\d{4}_\d{2}_\d{2}-\d{2}_\d{2}_\d{2})[^\/]+$/
- ) {
- # Remove log file #318 and notes file #3972 if they still exist
- PVE::Storage::archive_auxiliaries_remove($path);
- }
};
my $id = (defined $ownervm ? "$ownervm@" : '') . $storeid;
@@ -532,29 +540,52 @@ __PACKAGE__->register_method({
name => 'copy',
path => '{volume}',
method => 'POST',
- description => "Copy a volume. This is experimental code - do not use.",
+ description => "Copy a volume.",
+ permissions => {
+ description => "If the --delete option is used, the 'Datastore.Allocate' privilege is"
+ . " required on the source storage."
+ . " Without --delete, 'Datastore.AllocateSpace' is required on the target storage.",
+ user => 'all',
+ },
protected => 1,
proxyto => 'node',
parameters => {
additionalProperties => 0,
properties => {
node => get_standard_option('pve-node'),
- storage => get_standard_option('pve-storage-id', { optional => 1 }),
+ storage => get_standard_option(
+ 'pve-storage-id',
+ {
+ optional => 1,
+ completion => \&PVE::Storage::complete_storage_enabled,
+ },
+ ),
volume => {
description => "Source volume identifier",
type => 'string',
+ completion => \&PVE::Storage::complete_volume,
},
- target => {
- description => "Target volume identifier",
- type => 'string',
- },
- target_node => get_standard_option(
+ 'target-storage' => get_standard_option(
+ 'pve-storage-id',
+ {
+ description => "Target storage",
+ completion => \&PVE::Storage::complete_storage_enabled,
+ },
+ ),
+ 'target-node' => get_standard_option(
'pve-node',
{
description => "Target node. Default is local node.",
optional => 1,
},
),
+ delete => {
+ type => 'boolean',
+ description => "Delete the original volume after a successful copy."
+ . " By default the original is kept.",
+ optional => 1,
+ default => 0,
+ },
},
},
returns => {
@@ -563,49 +594,53 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- my $rpcenv = PVE::RPCEnvironment::get();
-
- my $user = $rpcenv->get_user();
-
- my $target_node = $param->{target_node} || PVE::INotify::nodename();
- # pvesh examples
- # cd /nodes/localhost/storage/local/content
- # pve:/> create local:103/vm-103-disk-1.raw -target local:103/vm-103-disk-2.raw
- # pve:/> create 103/vm-103-disk-1.raw -target 103/vm-103-disk-3.raw
+ my $src_volid = $real_volume_id->($param->{storage}, $param->{volume});
+ my $dst_storeid = $param->{'target-storage'};
+ my ($src_storeid, $volname) = PVE::Storage::parse_volume_id($src_volid);
+ my $src_node = PVE::INotify::nodename();
+ my $dst_node = $param->{'target-node'} || $src_node;
+ my $delete = $param->{delete};
- my $src_volid = &$real_volume_id($param->{storage}, $param->{volume});
- my $dst_volid = &$real_volume_id($param->{storage}, $param->{target});
-
- print "DEBUG: COPY $src_volid TO $dst_volid\n";
+ die "source and target cannot be the same\n"
+ if $src_node eq $dst_node && $src_storeid eq $dst_storeid;
my $cfg = PVE::Storage::config();
- # do all parameter checks first
-
- # then do all short running task (to raise errors before we go to background)
+ my ($vtype, undef, $ownervm) = PVE::Storage::parse_volname($cfg, $src_volid);
+ die "use pct move-volume or qm disk move\n"
+ if $vtype eq 'images' || $vtype eq 'rootdir';
+ die "moving volume of type '$vtype' not implemented\n"
+ if !grep { $vtype eq $_ } qw(import iso snippets vztmpl);
- # then start the worker task
- my $worker = sub {
- my $upid = shift;
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $user = $rpcenv->get_user();
- print "DEBUG: starting worker $upid\n";
+ PVE::Storage::check_volume_access($rpcenv, $user, $cfg, $ownervm, $src_volid);
- my ($target_sid, $target_volname) = PVE::Storage::parse_volume_id($dst_volid);
- #my $target_ip = PVE::Cluster::remote_node_ip($target_node);
+ if ($delete) {
+ $rpcenv->check($user, "/storage/$src_storeid", ["Datastore.Allocate"]);
+ }
- # you need to get this working (fails currently, because storage_migrate() uses
- # ssh to connect to local host (which is not needed
- my $sshinfo = PVE::SSHInfo::get_ssh_info($target_node);
- PVE::Storage::storage_migrate(
- $cfg,
- $src_volid,
- $sshinfo,
- $target_sid,
- { 'target_volname' => $target_volname },
- );
+ $rpcenv->check($user, "/storage/$dst_storeid", ["Datastore.AllocateSpace"]);
- print "DEBUG: end worker $upid\n";
+ my $worker = sub {
+ PVE::Storage::storage_check_enabled($cfg, $src_storeid, $src_node);
+ PVE::Storage::storage_check_enabled($cfg, $dst_storeid, $dst_node);
+ my $sshinfo;
+ $sshinfo = PVE::SSHInfo::get_ssh_info($dst_node) if $src_node ne $dst_node;
+ my $opts = { 'target_volname' => $volname };
+ PVE::Storage::storage_migrate($cfg, $src_volid, $sshinfo, $dst_storeid, $opts);
+
+ if ($delete) {
+ &$volume_remove($cfg, $src_volid);
+ }
+ if ($src_node eq $dst_node) {
+ print "Copied volume '$src_volid' to '$dst_storeid'\n";
+ } else {
+ print "Copied volume '$src_volid' on node '$src_node'"
+ . " to '$dst_storeid' on node '$dst_node'\n";
+ }
};
return $rpcenv->fork_worker('imgcopy', undef, $user, $worker);
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [pve-devel] [PATCH storage v8 4/9] api: content: implement copying volumes between storages
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 4/9] api: content: implement copying volumes between storages Filip Schauer
@ 2025-11-13 23:19 ` Thomas Lamprecht
0 siblings, 0 replies; 12+ messages in thread
From: Thomas Lamprecht @ 2025-11-13 23:19 UTC (permalink / raw)
To: Proxmox VE development discussion, Filip Schauer
Am 16.09.25 um 14:36 schrieb Filip Schauer:
> @@ -532,29 +540,52 @@ __PACKAGE__->register_method({
> name => 'copy',
> path => '{volume}',
> method => 'POST',
> - description => "Copy a volume. This is experimental code - do not use.",
> + description => "Copy a volume.",
> + permissions => {
> + description => "If the --delete option is used, the 'Datastore.Allocate' privilege is"
> + . " required on the source storage."
> + . " Without --delete, 'Datastore.AllocateSpace' is required on the target storage.",
> + user => 'all',
> + },
> protected => 1,
> proxyto => 'node',
> parameters => {
> additionalProperties => 0,
> properties => {
> node => get_standard_option('pve-node'),
> - storage => get_standard_option('pve-storage-id', { optional => 1 }),
> + storage => get_standard_option(
> + 'pve-storage-id',
> + {
> + optional => 1,
> + completion => \&PVE::Storage::complete_storage_enabled,
> + },
> + ),
> volume => {
> description => "Source volume identifier",
> type => 'string',
> + completion => \&PVE::Storage::complete_volume,
> },
> - target => {
> - description => "Target volume identifier",
> - type => 'string',
> - },
> - target_node => get_standard_option(
> + 'target-storage' => get_standard_option(
> + 'pve-storage-id',
> + {
> + description => "Target storage",
> + completion => \&PVE::Storage::complete_storage_enabled,
> + },
> + ),
> + 'target-node' => get_standard_option(
> 'pve-node',
> {
> description => "Target node. Default is local node.",
> optional => 1,
> },
> ),
> + delete => {
> + type => 'boolean',
> + description => "Delete the original volume after a successful copy."
> + . " By default the original is kept.",
> + optional => 1,
> + default => 0,
> + },
> },
> },
Can be fine to do due to this being marked as experimental in the description,
but would be really good to argue that in the commit message as it still is
exposed in the API since a long time. If something made this completely useless
until now it would be fine as is, otherwise it might make sense to keep the
"target" param as fallback?
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread
* [pve-devel] [PATCH storage v8 5/9] introduce $vtype+meta export formats
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
` (3 preceding siblings ...)
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 4/9] api: content: implement copying volumes between storages Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 6/9] api: content: support copying backups between path based storages Filip Schauer
` (4 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
These new export formats include a JSON metadata header containing the
vtype and the format. This allows for future extensibility without
breaking backward compatibility when adding additional metadata.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/Storage.pm | 16 +++++-
src/PVE/Storage/Plugin.pm | 101 +++++++++++++++++++++++++++++++++-----
2 files changed, 103 insertions(+), 14 deletions(-)
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 3467a90..d380160 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -47,7 +47,21 @@ use constant APIVER => 12;
# see https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html
use constant APIAGE => 3;
-our $KNOWN_EXPORT_FORMATS = ['raw+size', 'tar+size', 'qcow2+size', 'vmdk+size', 'zfs', 'btrfs'];
+our $KNOWN_EXPORT_FORMATS = [
+ 'raw+size',
+ 'tar+size',
+ 'qcow2+size',
+ 'vmdk+size',
+ 'zfs',
+ 'btrfs',
+ 'images+meta',
+ 'rootdir+meta',
+ 'iso+meta',
+ 'vztmpl+meta',
+ 'backup+meta',
+ 'snippets+meta',
+ 'import+meta',
+];
# load standard plugins
PVE::Storage::DirPlugin->register();
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index ecf68c8..29628ed 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -2022,6 +2022,25 @@ sub read_common_header($) {
return $size;
}
+sub write_meta_header($$) {
+ my ($fh, $meta) = @_;
+ my $meta_json = JSON::encode_json($meta);
+ $meta_json = Encode::encode('utf8', $meta_json);
+ syswrite($fh, pack("Q<", length($meta_json)), 8);
+ syswrite($fh, $meta_json, length($meta_json));
+}
+
+sub read_meta_header($) {
+ my ($fh) = @_;
+ sysread($fh, my $meta_size, 8);
+ $meta_size = unpack('Q<', $meta_size);
+ die "import: no meta size found in export header, aborting.\n" if !defined($meta_size);
+ sysread($fh, my $meta_json, $meta_size);
+ $meta_json = Encode::decode('utf8', $meta_json);
+ my $meta = JSON::decode_json($meta_json);
+ return $meta;
+}
+
# Export a volume into a file handle as a stream of desired format.
sub volume_export {
my ($class, $scfg, $storeid, $fh, $volname, $format, $snapshot, $base_snapshot, $with_snapshots)
@@ -2033,7 +2052,7 @@ sub volume_export {
my $err_msg = "volume export format $format not available for $class\n";
if ($scfg->{path} && !defined($snapshot) && !defined($base_snapshot)) {
my ($file) = $class->path($scfg, $volname, $storeid) or die $err_msg;
- my $file_format = ($class->parse_volname($volname))[6];
+ my ($vtype, $file_format) = ($class->parse_volname($volname))[0, 6];
my $size = file_size_info($file, undef, $file_format);
if ($format eq 'raw+size') {
@@ -2077,6 +2096,52 @@ sub volume_export {
output => '>&' . fileno($fh),
);
return;
+ } elsif ($format eq "$vtype+meta") {
+ die "format $file_format cannot be exported without snapshots\n"
+ if !$with_snapshots && $file_format =~ /^qcow2|vmdk$/;
+ die "format $file_format cannot be exported with snapshots\n"
+ if $with_snapshots && $file_format =~ /^raw|subvol$/;
+
+ my $data_format;
+ if ($file_format eq 'subvol') {
+ $data_format = 'tar';
+ } else {
+ $data_format = $file_format;
+ }
+
+ my $meta = {
+ vtype => $vtype,
+ format => $data_format,
+ };
+
+ write_meta_header($fh, $meta);
+ write_common_header($fh, $size);
+ if ($data_format =~ /^(raw|ova|ovf|qcow2|vmdk)$/) {
+ run_command(
+ ['dd', "if=$file", "bs=4k", "status=progress"],
+ output => '>&' . fileno($fh),
+ );
+ } elsif ($data_format eq 'tar') {
+ run_command(
+ ['tar', @COMMON_TAR_FLAGS, '-cf', '-', '-C', $file, '.'],
+ output => '>&' . fileno($fh),
+ );
+ } else {
+ run_command(
+ [
+ 'qemu-img',
+ 'convert',
+ '-f',
+ $file_format,
+ '-O',
+ 'raw',
+ $file,
+ '/dev/stdout',
+ ],
+ output => '>&' . fileno($fh),
+ );
+ }
+ return;
}
}
die $err_msg;
@@ -2088,11 +2153,11 @@ sub volume_export_formats {
my ($vtype, $format) = ($class->parse_volname($volname))[0, 6];
if ($with_snapshots) {
- return ($format . '+size') if ($format eq 'qcow2' || $format eq 'vmdk');
+ return ("$vtype+meta", $format . '+size') if $format =~ /^(qcow2|vmdk)$/;
return ();
}
- return ('tar+size') if $format eq 'subvol';
- return ('raw+size') if $vtype =~ /^(iso|snippets|vztmpl|import)$/;
+ return ("$vtype+meta", 'tar+size') if $format eq 'subvol';
+ return ("$vtype+meta", 'raw+size') if $vtype =~ /^(iso|snippets|vztmpl|import)$/;
}
return ();
}
@@ -2112,18 +2177,28 @@ sub volume_import {
$allow_rename,
) = @_;
- die "volume import format '$format' not available for $class\n"
- if $format !~ /^(raw|tar|qcow2|vmdk)\+size$/;
- my $data_format = $1;
+ my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $file_format) =
+ $class->parse_volname($volname);
+
+ my $meta;
+ my $data_format;
+
+ if ($format =~ /^(images|rootdir|iso|vztmpl|backup|snippets|import)\+meta$/) {
+ die "volume type does not match import format\n" if $1 ne $vtype;
+ $meta = read_meta_header($fh);
+ die "volume type does not match metadata header\n" if $meta->{vtype} ne $vtype;
+ $data_format = $meta->{format};
+ } elsif ($format =~ /^(raw|tar|qcow2|vmdk)\+size$/) {
+ $data_format = $1;
+ } else {
+ die "volume import format '$format' not available for $class\n";
+ }
die "format $format cannot be imported without snapshots\n"
if !$with_snapshots && ($data_format eq 'qcow2' || $data_format eq 'vmdk');
die "format $format cannot be imported with snapshots\n"
if $with_snapshots && ($data_format eq 'raw' || $data_format eq 'tar');
- my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $file_format) =
- $class->parse_volname($volname);
-
# XXX: Should we bother with conversion routines at this level? This won't
# happen without manual CLI usage, so for now we just error out...
if (
@@ -2201,11 +2276,11 @@ sub volume_import_formats {
my ($vtype, $format) = ($class->parse_volname($volname))[0, 6];
if ($with_snapshots) {
- return ($format . '+size') if ($format eq 'qcow2' || $format eq 'vmdk');
+ return ("$vtype+meta", $format . '+size') if $format =~ /^(qcow2|vmdk)$/;
return ();
}
- return ('tar+size') if $format eq 'subvol';
- return ('raw+size') if $vtype =~ /^(iso|snippets|vztmpl|import)$/;
+ return ("$vtype+meta", 'tar+size') if $format eq 'subvol';
+ return ("$vtype+meta", 'raw+size') if $vtype =~ /^(iso|snippets|vztmpl|import)$/;
}
return ();
}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH storage v8 6/9] api: content: support copying backups between path based storages
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
` (4 preceding siblings ...)
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 5/9] introduce $vtype+meta export formats Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 7/9] storage: introduce decompress_archive_into_pipe helper Filip Schauer
` (3 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
This commit adds support for the "backup+meta" export format. When this
format is used, the notes and protection flag of the backup are included
in the metadata header.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/API2/Storage/Content.pm | 11 +++++++++--
src/PVE/Storage/Plugin.pm | 25 ++++++++++++++++++++++++-
2 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/src/PVE/API2/Storage/Content.pm b/src/PVE/API2/Storage/Content.pm
index c69b859..cfe4f2f 100644
--- a/src/PVE/API2/Storage/Content.pm
+++ b/src/PVE/API2/Storage/Content.pm
@@ -544,7 +544,8 @@ __PACKAGE__->register_method({
permissions => {
description => "If the --delete option is used, the 'Datastore.Allocate' privilege is"
. " required on the source storage."
- . " Without --delete, 'Datastore.AllocateSpace' is required on the target storage.",
+ . " Without --delete, 'Datastore.AllocateSpace' is required on the target storage."
+ . " When moving a backup, 'VM.Backup' is required on the VM or container.",
user => 'all',
},
protected => 1,
@@ -610,7 +611,7 @@ __PACKAGE__->register_method({
die "use pct move-volume or qm disk move\n"
if $vtype eq 'images' || $vtype eq 'rootdir';
die "moving volume of type '$vtype' not implemented\n"
- if !grep { $vtype eq $_ } qw(import iso snippets vztmpl);
+ if !grep { $vtype eq $_ } qw(backup import iso snippets vztmpl);
my $rpcenv = PVE::RPCEnvironment::get();
my $user = $rpcenv->get_user();
@@ -619,6 +620,12 @@ __PACKAGE__->register_method({
if ($delete) {
$rpcenv->check($user, "/storage/$src_storeid", ["Datastore.Allocate"]);
+
+ if ($vtype eq 'backup') {
+ my $protected =
+ PVE::Storage::get_volume_attribute($cfg, $src_volid, 'protected');
+ die "cannot delete protected backup\n" if $protected;
+ }
}
$rpcenv->check($user, "/storage/$dst_storeid", ["Datastore.AllocateSpace"]);
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 29628ed..92c4820 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -2114,6 +2114,17 @@ sub volume_export {
format => $data_format,
};
+ if ($vtype eq 'backup') {
+ $meta->{protected} =
+ $class->get_volume_attribute($scfg, $storeid, $volname, 'protected')
+ ? JSON::true
+ : JSON::false;
+ $meta->{notes} = Encode::encode(
+ 'utf8',
+ $class->get_volume_attribute($scfg, $storeid, $volname, 'notes'),
+ ) // "";
+ }
+
write_meta_header($fh, $meta);
write_common_header($fh, $size);
if ($data_format =~ /^(raw|ova|ovf|qcow2|vmdk)$/) {
@@ -2157,6 +2168,7 @@ sub volume_export_formats {
return ();
}
return ("$vtype+meta", 'tar+size') if $format eq 'subvol';
+ return ('backup+meta') if $vtype eq 'backup';
return ("$vtype+meta", 'raw+size') if $vtype =~ /^(iso|snippets|vztmpl|import)$/;
}
return ();
@@ -2252,7 +2264,7 @@ sub volume_import {
warn $@ if $@;
die $err;
}
- } elsif (grep { $vtype eq $_ } qw(import iso snippets vztmpl)) {
+ } elsif (grep { $vtype eq $_ } qw(import iso snippets vztmpl backup)) {
eval {
run_command(['dd', "of=$file", 'conv=excl', 'bs=64k'], input => '<&' . fileno($fh));
};
@@ -2263,6 +2275,16 @@ sub volume_import {
}
die $err;
}
+
+ if ($vtype eq 'backup' && defined($meta)) {
+ if (defined($meta->{notes})) {
+ my $notes = Encode::decode('utf8', $meta->{notes});
+ $class->update_volume_attribute($scfg, $storeid, $volname, 'notes', $notes);
+ }
+
+ $class->update_volume_attribute($scfg, $storeid, $volname, 'protected', 1)
+ if $meta->{protected};
+ }
} else {
die "importing volume of type '$vtype' not implemented\n";
}
@@ -2280,6 +2302,7 @@ sub volume_import_formats {
return ();
}
return ("$vtype+meta", 'tar+size') if $format eq 'subvol';
+ return ('backup+meta') if $vtype eq 'backup';
return ("$vtype+meta", 'raw+size') if $vtype =~ /^(iso|snippets|vztmpl|import)$/;
}
return ();
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH storage v8 7/9] storage: introduce decompress_archive_into_pipe helper
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
` (5 preceding siblings ...)
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 6/9] api: content: support copying backups between path based storages Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 8/9] support copying VMA backups to PBS Filip Schauer
` (2 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
Extract the file decompression code into its own reusable subroutine.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/Storage.pm | 68 +++++++++++++++++++++++++++-------------------
1 file changed, 40 insertions(+), 28 deletions(-)
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index d380160..5071475 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -1887,6 +1887,45 @@ sub extract_vzdump_config_tar {
return wantarray ? ($raw, $file) : $raw;
}
+sub decompress_archive_into_pipe {
+ my ($archive, $cmd, $outfunc) = @_;
+
+ my $info = archive_info($archive);
+ die "archive is not compressed\n" if !$info->{compression};
+ my $decompressor = $info->{decompressor};
+ my $full_cmd = [[@$decompressor, $archive], $cmd];
+
+ # lzop/zcat exits with 1 when the pipe is closed early,
+ # detect this and ignore the exit code later
+ my $broken_pipe;
+ my $errstring;
+ my $err = sub {
+ my $output = shift;
+ if (
+ $output =~ m/lzop: Broken pipe: <stdout>/
+ || $output =~ m/gzip: stdout: Broken pipe/
+ || $output =~ m/zstd: error 70 : Write error.*Broken pipe/
+ ) {
+ $broken_pipe = 1;
+ } elsif (!defined($errstring) && $output !~ m/^\s*$/) {
+ $errstring = "failed to decompress archive: $output\n";
+ }
+ };
+
+ my $rc = eval { run_command($full_cmd, outfunc => $outfunc, errfunc => $err, noerr => 1) };
+ my $rerr = $@;
+
+ $broken_pipe ||= $rc == 141; # broken pipe from cmd POV
+
+ if (!$errstring && !$broken_pipe && $rc != 0) {
+ die "$rerr\n" if $rerr;
+ die "archive decompression failed with exit code $rc\n";
+ }
+ die "$errstring\n" if $errstring;
+
+ return;
+}
+
sub extract_vzdump_config_vma {
my ($archive, $comp) = @_;
@@ -1898,34 +1937,7 @@ sub extract_vzdump_config_vma {
my $decompressor = $info->{decompressor};
if ($comp) {
- my $cmd = [[@$decompressor, $archive], ["vma", "config", "-"]];
-
- # lzop/zcat exits with 1 when the pipe is closed early by vma, detect this and ignore the exit code later
- my $broken_pipe;
- my $errstring;
- my $err = sub {
- my $output = shift;
- if (
- $output =~ m/lzop: Broken pipe: <stdout>/
- || $output =~ m/gzip: stdout: Broken pipe/
- || $output =~ m/zstd: error 70 : Write error.*Broken pipe/
- ) {
- $broken_pipe = 1;
- } elsif (!defined($errstring) && $output !~ m/^\s*$/) {
- $errstring = "Failed to extract config from VMA archive: $output\n";
- }
- };
-
- my $rc = eval { run_command($cmd, outfunc => $out, errfunc => $err, noerr => 1) };
- my $rerr = $@;
-
- $broken_pipe ||= $rc == 141; # broken pipe from vma POV
-
- if (!$errstring && !$broken_pipe && $rc != 0) {
- die "$rerr\n" if $rerr;
- die "config extraction failed with exit code $rc\n";
- }
- die "$errstring\n" if $errstring;
+ decompress_archive_into_pipe($archive, ["vma", "config", "-"], $out);
} else {
run_command(["vma", "config", $archive], outfunc => $out);
}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH storage v8 8/9] support copying VMA backups to PBS
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
` (6 preceding siblings ...)
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 7/9] storage: introduce decompress_archive_into_pipe helper Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 9/9] pvesm: add a copy-volume command Filip Schauer
2025-11-13 23:19 ` [pve-devel] applied: [PATCH storage v8 0/9] support copying volumes between storages Thomas Lamprecht
9 siblings, 0 replies; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
Extend the copy API to support copying VMA backups to a Proxmox Backup
Server.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
debian/control | 1 +
src/PVE/API2/Storage/Content.pm | 22 +++++++++--
src/PVE/Storage/PBSPlugin.pm | 65 +++++++++++++++++++++++++++++++++
3 files changed, 84 insertions(+), 4 deletions(-)
diff --git a/debian/control b/debian/control
index 5341317..b3cba28 100644
--- a/debian/control
+++ b/debian/control
@@ -45,6 +45,7 @@ Depends: bzip2,
nfs-common,
proxmox-backup-client (>= 2.1.10~),
proxmox-backup-file-restore,
+ proxmox-vma-to-pbs (>= 0.0.2),
pve-cluster (>= 5.0-32),
smartmontools,
smbclient,
diff --git a/src/PVE/API2/Storage/Content.pm b/src/PVE/API2/Storage/Content.pm
index cfe4f2f..0c30606 100644
--- a/src/PVE/API2/Storage/Content.pm
+++ b/src/PVE/API2/Storage/Content.pm
@@ -7,6 +7,7 @@ use PVE::SafeSyslog;
use PVE::Cluster;
use PVE::Storage;
use PVE::Storage::Common; # for 'pve-storage-image-format' standard option
+use PVE::Storage::PBSPlugin;
use PVE::INotify;
use PVE::Exception qw(raise_param_exc);
use PVE::RPCEnvironment;
@@ -633,10 +634,23 @@ __PACKAGE__->register_method({
my $worker = sub {
PVE::Storage::storage_check_enabled($cfg, $src_storeid, $src_node);
PVE::Storage::storage_check_enabled($cfg, $dst_storeid, $dst_node);
- my $sshinfo;
- $sshinfo = PVE::SSHInfo::get_ssh_info($dst_node) if $src_node ne $dst_node;
- my $opts = { 'target_volname' => $volname };
- PVE::Storage::storage_migrate($cfg, $src_volid, $sshinfo, $dst_storeid, $opts);
+
+ my $src_cfg = PVE::Storage::storage_config($cfg, $src_storeid);
+ my $dst_cfg = PVE::Storage::storage_config($cfg, $dst_storeid);
+ if ($vtype eq 'backup' && $src_cfg->{path} && $dst_cfg->{type} eq 'pbs') {
+ my $src_plugin = PVE::Storage::Plugin->lookup($src_cfg->{type});
+ my $src_path = $src_plugin->path($src_cfg, $volname, $src_storeid);
+ my $protected =
+ PVE::Storage::get_volume_attribute($cfg, $src_volid, 'protected');
+ PVE::Storage::PBSPlugin::vma_to_pbs(
+ $ownervm, $src_path, $dst_cfg, $dst_storeid, $protected,
+ );
+ } else {
+ my $sshinfo;
+ $sshinfo = PVE::SSHInfo::get_ssh_info($dst_node) if $src_node ne $dst_node;
+ my $opts = { 'target_volname' => $volname };
+ PVE::Storage::storage_migrate($cfg, $src_volid, $sshinfo, $dst_storeid, $opts);
+ }
if ($delete) {
&$volume_remove($cfg, $src_volid);
diff --git a/src/PVE/Storage/PBSPlugin.pm b/src/PVE/Storage/PBSPlugin.pm
index 5842004..da93c6e 100644
--- a/src/PVE/Storage/PBSPlugin.pm
+++ b/src/PVE/Storage/PBSPlugin.pm
@@ -7,6 +7,7 @@ use warnings;
use Encode qw(decode);
use Fcntl qw(F_GETFD F_SETFD FD_CLOEXEC);
+use File::Basename;
use IO::File;
use JSON;
use MIME::Base64 qw(decode_base64);
@@ -988,4 +989,68 @@ sub volume_has_feature {
return undef;
}
+sub vma_to_pbs {
+ my ($vmid, $source_path, $target_scfg, $target_storeid, $protected) = @_;
+
+ #my $source_plugin = PVE::Storage::Plugin->lookup($source_scfg->{type});
+ my $target_plugin = PVE::Storage::Plugin->lookup($target_scfg->{type});
+ my $info = PVE::Storage::archive_info($source_path);
+ die "moving non-VMA backups to a Proxmox Backup Server is not supported\n"
+ if $info->{format} ne 'vma';
+
+ my $repo = PVE::PBSClient::get_repository($target_scfg);
+ my $fingerprint = $target_scfg->{fingerprint};
+ my $password = PVE::Storage::PBSPlugin::pbs_password_file_name($target_scfg, $target_storeid);
+ my $namespace = $target_scfg->{namespace};
+ my $keyfile =
+ PVE::Storage::PBSPlugin::pbs_encryption_key_file_name($target_scfg, $target_storeid);
+ my $master_keyfile =
+ PVE::Storage::PBSPlugin::pbs_master_pubkey_file_name($target_scfg, $target_storeid);
+
+ my $comp = $info->{compression};
+ my $backup_time = $info->{ctime};
+ my $source_dirname = dirname($source_path);
+ my $log_file_path = "$source_dirname/$info->{logfilename}";
+ my $notes_file_path = "$source_dirname/$info->{notesfilename}";
+
+ my $vma_to_pbs_cmd = [
+ "vma-to-pbs",
+ "--repository",
+ $repo,
+ "--vmid",
+ $vmid,
+ "--fingerprint",
+ $fingerprint,
+ "--password-file",
+ $password,
+ "--backup-time",
+ $backup_time,
+ "--compress",
+ ];
+
+ push @$vma_to_pbs_cmd, "--ns", $namespace if $namespace;
+ push @$vma_to_pbs_cmd, "--log-file", $log_file_path if -e $log_file_path;
+ push @$vma_to_pbs_cmd, "--notes-file", $notes_file_path if -e $notes_file_path;
+ push @$vma_to_pbs_cmd, "--encrypt", "--keyfile", $keyfile if -e $keyfile;
+ push @$vma_to_pbs_cmd, "--master-keyfile", $master_keyfile if -e $master_keyfile;
+
+ if ($comp) {
+ PVE::Storage::decompress_archive_into_pipe($source_path, $vma_to_pbs_cmd);
+ } else {
+ push @$vma_to_pbs_cmd, $source_path;
+ run_command($vma_to_pbs_cmd);
+ }
+
+ if ($protected) {
+ my $target_volid =
+ PVE::Storage::PBSPlugin::print_volid($target_storeid, 'vm', $vmid, $backup_time);
+ my (undef, $target_volname) = PVE::Storage::parse_volume_id($target_volid, 0);
+ $target_plugin->update_volume_attribute(
+ $target_scfg, $target_storeid, $target_volname, 'protected', 1,
+ );
+ }
+
+ return;
+}
+
1;
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH storage v8 9/9] pvesm: add a copy-volume command
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
` (7 preceding siblings ...)
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 8/9] support copying VMA backups to PBS Filip Schauer
@ 2025-09-16 12:32 ` Filip Schauer
2025-11-13 23:19 ` [pve-devel] applied: [PATCH storage v8 0/9] support copying volumes between storages Thomas Lamprecht
9 siblings, 0 replies; 12+ messages in thread
From: Filip Schauer @ 2025-09-16 12:32 UTC (permalink / raw)
To: pve-devel
The method can be called from the PVE shell with `pvesm copy-volume`:
```
pvesm copy-volume <source volume> <target storage> [--target-node <node>] [--delete]
```
For example to copy a VMA backup to a Proxmox Backup Server:
```
pvesm copy-volume \
local:backup/vzdump-qemu-100-2024_06_25-13_08_56.vma.zst pbs
```
Or copy a container template to another node and delete the source:
```
pvesm copy-volume \
local:vztmpl/devuan-4.0-standard_4.0_amd64.tar.gz local \
--target-node pvenode2 --delete
```
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/CLI/pvesm.pm | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/PVE/CLI/pvesm.pm b/src/PVE/CLI/pvesm.pm
index 860e46f..a9a0765 100755
--- a/src/PVE/CLI/pvesm.pm
+++ b/src/PVE/CLI/pvesm.pm
@@ -769,6 +769,12 @@ our $cmddef = {
print "APIAGE $res->{apiage}\n";
},
],
+ 'copy-volume' => [
+ "PVE::API2::Storage::Content",
+ 'copy',
+ ['volume', 'target-storage'],
+ { node => $nodename },
+ ],
'prune-backups' => [
__PACKAGE__,
'prunebackups',
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] applied: [PATCH storage v8 0/9] support copying volumes between storages
2025-09-16 12:32 [pve-devel] [PATCH storage v8 0/9] support copying volumes between storages Filip Schauer
` (8 preceding siblings ...)
2025-09-16 12:32 ` [pve-devel] [PATCH storage v8 9/9] pvesm: add a copy-volume command Filip Schauer
@ 2025-11-13 23:19 ` Thomas Lamprecht
9 siblings, 0 replies; 12+ messages in thread
From: Thomas Lamprecht @ 2025-11-13 23:19 UTC (permalink / raw)
To: pve-devel, Filip Schauer
On Tue, 16 Sep 2025 14:32:45 +0200, Filip Schauer wrote:
> Add the ability to copy a backup, ISO, container template, snippet, or
> OVA/OVF between storages and nodes via an API method. Copying a VMA
> backup to a Proxmox Backup Server requires the proxmox-vma-to-pbs
> package to be installed. Currently only VMA backups can be copied to a
> Proxmox Backup Server and copying backups from a Proxmox Backup Server
> is currently not supported.
>
> [...]
Applied the first three patches already, thanks!
[1/9] storage migrate: remove remnant from rsync-based migration
commit: e55653445919d7e776778127d0fc94fbd02d9f5a
[2/9] storage migrate: avoid ssh when moving a volume locally
commit: 829b0cd728488ec70d6af7dac5b6f1c2d811f08e
[3/9] plugin: allow volume import of iso, snippets, vztmpl and import
commit: 0ba0739f69bc836d1468c1f62be4f2aa29991133
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 12+ messages in thread