[pve-devel] [RFC v1 pve-storage 1/2] (rfc) example: sshfs plugin: add custom storage plugin for sshfs

* [pve-devel] [RFC v1 pve-storage 1/2] (rfc) example: sshfs plugin: add custom storage plugin for sshfs
@ 2025-03-28 17:12 Max Carrara
  2025-03-28 17:12 ` [pve-devel] [RFC v1 pve-storage 2/2] (rfc) example: sshfs plugin: package SSHFSPlugin.pm Max Carrara
  2025-03-31  8:06 ` [pve-devel] [RFC v1 pve-storage 1/2] (rfc) example: sshfs plugin: add custom storage plugin for sshfs Fiona Ebner
  0 siblings, 2 replies; 4+ messages in thread
From: Max Carrara @ 2025-03-28 17:12 UTC (permalink / raw)
  To: pve-devel

This commit adds a rudimentary implementation of a custom storage
plugin that uses sshfs [1] as the underlying filesystem.

The implementation is very similar to that of the NFS plugin; as a
prerequisite, it is currently necessary to use pubkey auth and have
the host's root user's public key deployed to the remote host like so:

  ssh-copy-id -o UserKnownHostsFile=/etc/pve/priv/known_hosts [USER]@[HOST]

Then, the storage can be added as follows:

  pvesm add sshfs [STOREID] \
    --server [HOST] \
    --sshfs-remote-path [ABS PATH ON REMOTE] \
    --username [USER] \
    --path [LOCAL MOUNTPOINT]

The cluster-wide known_hosts file is used by the plugin in
anticipation of potentially marking the plugin as "shared", i.e. as
shared storage, once there's an easier way to do this via the plugin
API. However, there are a couple more questions that need to be
addressed first, before this can be made shared:

- What would be the preferred way to allow specifying whether a
  (custom) plugin is shared or not via our API?

  E.g. some external plugins do the following, which (I suppose)
  wasn't originally part of the API, but is now, due it being used in
  the wild:

    push @PVE::Storage::Plugin::SHARED_STORAGE, 'some-custom-plugin';

  Would be open for any suggestions on how to support this properly!
  Perhaps as a flag in `plugindata()`?

- Should we allow custom plugins to define sensitive properties for
  their own purposes? If so, how?

  Currently, sensitive props are hardcoded [2] which is sub-optimal,
  but gets the job done. However, should third-party plugin authors
  need additional / different properties, there's currently no way to
  support this. This would perhaps also be useful for this plugin
  here, as one could e.g. provide a path to a password file to use for
  something like sshpass [3] or similar, but I'm not really sure about
  this yet.

The reason why I'm bringing this up is because the upcoming guide in
the wiki could benefit from a demonstration on how to implement /
handle both cases. Network storages are quite common, can be shared
among nodes in most cases, and may also require one to handle
authentication.

Please let me know what you think!

[1]: https://github.com/libfuse/sshfs
[2]: https://git.proxmox.com/?p=pve-storage.git;a=blob;f=src/PVE/API2/Storage/Config.pm;h=e04b6ab93a2081e2f8d253188a3d0056bedfccec;hb=refs/heads/master#l193
[3]: https://packages.debian.org/bookworm/sshpass

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 .../lib/PVE/Storage/Custom/SSHFSPlugin.pm     | 201 ++++++++++++++++++
 1 file changed, 201 insertions(+)
 create mode 100644 example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm

diff --git a/example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm b/example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm
new file mode 100644
index 0000000..326c8f0
--- /dev/null
+++ b/example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm
@@ -0,0 +1,201 @@
+package PVE::Storage::Custom::SSHFSPlugin;
+
+use strict;
+use warnings;
+
+use feature 'signatures';
+
+use PVE::ProcFSTools;
+use PVE::Tools qw(run_command);
+
+use parent qw(PVE::Storage::Plugin);
+
+my $CLUSTER_KNOWN_HOSTS = "/etc/pve/priv/known_hosts";
+
+my @COMMON_SSH_OPTS = (
+    '-o', "UserKnownHostsFile=${CLUSTER_KNOWN_HOSTS}",
+);
+
+# Plugin Definition
+
+sub api {
+    return 10;
+}
+
+sub type {
+    return 'sshfs';
+}
+
+sub plugindata {
+    return {
+	content => [
+	    {
+		images => 1,
+		rootdir => 1,
+		vztmpl => 1,
+		iso => 1,
+		backup => 1,
+		snippets => 1,
+		import => 1,
+		none => 1,
+	    },
+	    {
+		images => 1,
+		rootdir => 1,
+	    },
+	],
+	format => [
+	    {
+		raw => 1,
+		qcow2 => 1,
+		vmdk => 1,
+	    },
+	    'qcow2',
+	],
+    };
+}
+
+sub properties {
+    return {
+	'sshfs-remote-path' => {
+	    description => "Path on the remote filesystem used for SSHFS. Must be absolute.",
+	    type => 'string',
+	    format => 'pve-storage-path',
+	},
+    };
+}
+
+sub options {
+    return {
+	path => { fixed => 1 },
+	'content-dirs' => { optional => 1 },
+	nodes => { optional => 1 },
+	shared => { optional => 1 },
+	disable => { optional => 1 },
+	'prune-backups' => { optional => 1 },
+	'max-protected-backups' => { optional => 1 },
+	content => { optional => 1 },
+	format => { optional => 1 },
+	'create-base-path' => { optional => 1 },
+	'create-subdirs' => { optional => 1 },
+	bwlimit => { optional => 1 },
+	preallocation => { optional => 1 },
+	# SSH Options
+	username => {},
+	server => {},
+	'sshfs-remote-path' => {},
+	port => { optional => 1 },
+   };
+}
+
+# SSHFS Helpers
+
+my sub sshfs_is_mounted: prototype($$) ($scfg, $cache) {
+    my ($user, $host, $remote_path) = $scfg->@{qw(username server sshfs-remote-path)};
+    my $mountpoint = $scfg->{path};
+
+    $cache->{mountdata} = PVE::ProcFSTools::parse_proc_mounts()
+	if !$cache->{mountdata};
+
+    my $ssh_url = "${user}\@${host}:${remote_path}";
+
+    my $has_found_mountpoint = grep {
+	$_->[0] =~ m|^\Q${ssh_url}\E$|
+	&& $_->[1] eq $mountpoint
+	&& $_->[2] eq 'fuse.sshfs'
+    } $cache->{mountdata}->@*;
+
+    return $has_found_mountpoint != 0;
+}
+
+my sub sshfs_mount: prototype($) ($scfg) {
+    my ($user, $host, $remote_path, $port) = $scfg->@{qw(username server sshfs-remote-path port)};
+    my $mountpoint = $scfg->{path};
+
+    my $remote = "${user}\@${host}:${remote_path}";
+
+    my $cmd = [
+	'/usr/bin/sshfs',
+	@COMMON_SSH_OPTS,
+    ];
+
+    push($cmd->@*, '-p', $port) if $port;
+    push($cmd->@*, $remote, $mountpoint);
+
+    eval {
+	run_command(
+	    $cmd,
+	    timeout => 10,
+	    errfunc => sub { warn "$_[0]\n"; },
+	);
+    };
+    if (my $err = $@) {
+	die "failed to mount sshfs storage '$remote' at '$mountpoint': $@\n";
+    }
+
+    die "sshfs storage '$remote' not mounted at '$mountpoint' despite reported success\n"
+	if ! sshfs_is_mounted($scfg, {});
+
+    return;
+}
+
+# Storage Implementation
+
+sub check_connection ($class, $storeid, $scfg) {
+    my ($user, $host, $port) = $scfg->@{qw(username server port)};
+
+    my $cmd = [
+	'/usr/bin/ssh',
+	'-T',
+	@COMMON_SSH_OPTS,
+	'-o', 'BatchMode=yes',
+	'-o', 'ConnectTimeout=5',
+    ];
+
+    push($cmd->@*, "-p", $port) if $port;
+    push($cmd->@*, "${user}\@${host}", 'exit 0');
+
+    eval {
+	run_command(
+	    $cmd,
+	    timeout => 10,
+	    errfunc => sub { warn "$_[0]\n"; },
+	);
+    };
+    if (my $err = $@) {
+	warn "$err";
+	return 0;
+    }
+
+    return 1;
+}
+
+sub activate_storage ($class, $storeid, $scfg, $cache) {
+    my $mountpoint = $scfg->{path};
+
+    if (!sshfs_is_mounted($scfg, $cache)) {
+	$class->config_aware_base_mkdir($scfg, $mountpoint);
+
+	die "unable to activate storage '$storeid' - directory '$mountpoint' does not exist\n"
+	    if ! -d $mountpoint;
+
+	sshfs_mount($scfg);
+    }
+
+    $class->SUPER::activate_storage($storeid, $scfg, $cache);
+    return;
+}
+
+sub get_volume_attribute {
+    return PVE::Storage::DirPlugin::get_volume_attribute(@_);
+}
+
+sub update_volume_attribute {
+    return PVE::Storage::DirPlugin::update_volume_attribute(@_);
+}
+
+sub get_import_metadata {
+    return PVE::Storage::DirPlugin::get_import_metadata(@_);
+}
+
+1;
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 4+ messages in thread