public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Fiona Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH storage 4/6] rbd plugin: implement volume import/export
Date: Fri, 13 Dec 2024 17:30:35 +0100	[thread overview]
Message-ID: <20241213163037.91508-5-f.ebner@proxmox.com> (raw)
In-Reply-To: <20241213163037.91508-1-f.ebner@proxmox.com>

For now, only 'raw+size' is supported and it's not possible to
export/import with snapshots. The volume or snapshot is mapped using
krbd and then the data is read via 'dd'.

Introducing an 'rbd' transport format might be feasible for more
complete (i.e. with snapshots, incremental) transfer between two RBD
storages.

The code is mostly copied from the LVM plugin.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/Storage/RBDPlugin.pm | 115 +++++++++++++++++++++++++++++++++++
 1 file changed, 115 insertions(+)

diff --git a/src/PVE/Storage/RBDPlugin.pm b/src/PVE/Storage/RBDPlugin.pm
index 301918c..0e53888 100644
--- a/src/PVE/Storage/RBDPlugin.pm
+++ b/src/PVE/Storage/RBDPlugin.pm
@@ -864,6 +864,121 @@ sub volume_has_feature {
     return undef;
 }
 
+sub volume_export_formats {
+    my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
+
+    return $class->volume_import_formats(
+	$scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots);
+}
+
+sub volume_export {
+    my (
+	$class,
+	$scfg,
+	$storeid,
+	$fh,
+	$volname,
+	$format,
+	$snapshot,
+	$base_snapshot,
+	$with_snapshots,
+    ) = @_;
+
+    die "volume export format $format not available for $class\n" if $format ne 'raw+size';
+    die "cannot export volumes together with their snapshots in $class\n" if $with_snapshots;
+    die "cannot export an incremental stream in $class\n" if defined($base_snapshot);
+
+    my $file = $class->map_volume($storeid, $scfg, $volname, $snapshot);
+    eval {
+
+	my $size;
+	# should be faster than querying RBD, also checks for the device file's availability
+	run_command(['/sbin/blockdev', '--getsize64', $file], outfunc => sub {
+	    my ($line) = @_;
+	    die "unexpected output from /sbin/blockdev: $line\n" if $line !~ /^(\d+)$/;
+	    $size = int($1);
+	});
+	PVE::Storage::Plugin::write_common_header($fh, $size);
+	run_command(['dd', "if=$file", "bs=64k", "status=progress"], output => '>&'.fileno($fh));
+    };
+    my $err = $@;
+
+    eval { $class->unmap_volume($storeid, $scfg, $volname); };
+    warn $@ if $@;
+
+    die $err if $err;
+
+    return;
+}
+
+sub volume_import_formats {
+    my ($class, $scfg, $storeid, $volname, $snapshot, $base_snapshot, $with_snapshots) = @_;
+    return () if $with_snapshots; # not supported
+    return () if defined($base_snapshot); # not supported
+    return ('raw+size');
+}
+
+sub volume_import {
+    my (
+	$class,
+	$scfg,
+	$storeid,
+	$fh,
+	$volname,
+	$format,
+	$snapshot,
+	$base_snapshot,
+	$with_snapshots,
+	$allow_rename,
+    ) = @_;
+
+    die "volume import format $format not available for $class\n" if $format ne 'raw+size';
+    die "cannot import volumes together with their snapshots in $class\n" if $with_snapshots;
+    die "cannot import an incremental stream in $class\n" if defined($base_snapshot);
+
+    my (undef, $name, $vmid, undef, undef, undef, $file_format) = $class->parse_volname($volname);
+    die "cannot import format $format into a volume of format $file_format\n"
+	if $file_format ne 'raw';
+
+    if (rbd_volume_exists($scfg, $storeid, $name)) {
+	die "volume $name already exists\n" if !$allow_rename;
+	warn "volume $name already exists - importing with a different name\n";
+	$name = undef;
+    }
+
+    my ($size) = PVE::Storage::Plugin::read_common_header($fh);
+    $size = int($size/1024);
+
+    my $mapped;
+    eval {
+	my $allocname = $class->alloc_image($storeid, $scfg, $vmid, 'raw', $name, $size);
+	my $oldname = $volname;
+	$volname = $allocname;
+	if (defined($name) && $allocname ne $oldname) {
+	    die "internal error: unexpected allocated name: '$allocname' != '$oldname'\n";
+	}
+
+	my $file = $class->map_volume($storeid, $scfg, $volname, undef);
+	$mapped = 1;
+
+	run_command(['dd', "of=$file", 'bs=64k'], input => '<&'.fileno($fh));
+    };
+    my $err = $@;
+
+    if ($mapped) {
+	eval { $class->unmap_volume($storeid, $scfg, $volname); };
+	warn $@ if $@;
+    }
+
+    if ($err) {
+	eval { $class->free_image($storeid, $scfg, $volname, 0, $file_format); };
+	warn $@ if $@;
+	die $err;
+    }
+
+    return "$storeid:$volname";
+}
+
 sub rename_volume {
     my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


  parent reply	other threads:[~2024-12-13 16:30 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-12-13 16:30 [pve-devel] [PATCH-SERIES storage 0/6] basic RBD import/export Fiona Ebner
2024-12-13 16:30 ` [pve-devel] [PATCH storage 1/6] rbd plugin: schema: document default value for 'krbd' setting Fiona Ebner
2024-12-13 16:30 ` [pve-devel] [PATCH storage 2/6] export: redirect stdout to avoid any unrelated messages ending up in the export stream Fiona Ebner
2024-12-13 16:30 ` [pve-devel] [PATCH storage 3/6] rbd plugin: factor out helper to check if volume already exists Fiona Ebner
2024-12-13 16:30 ` Fiona Ebner [this message]
2024-12-13 16:34   ` [pve-devel] [PATCH storage 4/6] rbd plugin: implement volume import/export Fiona Ebner
2024-12-17 12:10     ` Fiona Ebner
2024-12-13 16:30 ` [pve-devel] [PATCH storage 5/6] common: introduce common module Fiona Ebner
2024-12-13 16:30 ` [pve-devel] [PATCH storage 6/6] plugins: volume import: align size up to 1KiB Fiona Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241213163037.91508-5-f.ebner@proxmox.com \
    --to=f.ebner@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal