From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id 9D5611FF164
for <inbox@lore.proxmox.com>; Fri, 28 Mar 2025 18:12:50 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 197897F6C;
Fri, 28 Mar 2025 18:12:44 +0100 (CET)
From: Max Carrara <m.carrara@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Fri, 28 Mar 2025 18:12:08 +0100
Message-Id: <20250328171209.503132-1-m.carrara@proxmox.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.076 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [RFC v1 pve-storage 1/2] (rfc) example: sshfs plugin:
add custom storage plugin for sshfs
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
This commit adds a rudimentary implementation of a custom storage
plugin that uses sshfs [1] as the underlying filesystem.
The implementation is very similar to that of the NFS plugin; as a
prerequisite, it is currently necessary to use pubkey auth and have
the host's root user's public key deployed to the remote host like so:
ssh-copy-id -o UserKnownHostsFile=/etc/pve/priv/known_hosts [USER]@[HOST]
Then, the storage can be added as follows:
pvesm add sshfs [STOREID] \
--server [HOST] \
--sshfs-remote-path [ABS PATH ON REMOTE] \
--username [USER] \
--path [LOCAL MOUNTPOINT]
The cluster-wide known_hosts file is used by the plugin in
anticipation of potentially marking the plugin as "shared", i.e. as
shared storage, once there's an easier way to do this via the plugin
API. However, there are a couple more questions that need to be
addressed first, before this can be made shared:
- What would be the preferred way to allow specifying whether a
(custom) plugin is shared or not via our API?
E.g. some external plugins do the following, which (I suppose)
wasn't originally part of the API, but is now, due it being used in
the wild:
push @PVE::Storage::Plugin::SHARED_STORAGE, 'some-custom-plugin';
Would be open for any suggestions on how to support this properly!
Perhaps as a flag in `plugindata()`?
- Should we allow custom plugins to define sensitive properties for
their own purposes? If so, how?
Currently, sensitive props are hardcoded [2] which is sub-optimal,
but gets the job done. However, should third-party plugin authors
need additional / different properties, there's currently no way to
support this. This would perhaps also be useful for this plugin
here, as one could e.g. provide a path to a password file to use for
something like sshpass [3] or similar, but I'm not really sure about
this yet.
The reason why I'm bringing this up is because the upcoming guide in
the wiki could benefit from a demonstration on how to implement /
handle both cases. Network storages are quite common, can be shared
among nodes in most cases, and may also require one to handle
authentication.
Please let me know what you think!
[1]: https://github.com/libfuse/sshfs
[2]: https://git.proxmox.com/?p=pve-storage.git;a=blob;f=src/PVE/API2/Storage/Config.pm;h=e04b6ab93a2081e2f8d253188a3d0056bedfccec;hb=refs/heads/master#l193
[3]: https://packages.debian.org/bookworm/sshpass
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
.../lib/PVE/Storage/Custom/SSHFSPlugin.pm | 201 ++++++++++++++++++
1 file changed, 201 insertions(+)
create mode 100644 example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm
diff --git a/example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm b/example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm
new file mode 100644
index 0000000..326c8f0
--- /dev/null
+++ b/example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm
@@ -0,0 +1,201 @@
+package PVE::Storage::Custom::SSHFSPlugin;
+
+use strict;
+use warnings;
+
+use feature 'signatures';
+
+use PVE::ProcFSTools;
+use PVE::Tools qw(run_command);
+
+use parent qw(PVE::Storage::Plugin);
+
+my $CLUSTER_KNOWN_HOSTS = "/etc/pve/priv/known_hosts";
+
+my @COMMON_SSH_OPTS = (
+ '-o', "UserKnownHostsFile=${CLUSTER_KNOWN_HOSTS}",
+);
+
+# Plugin Definition
+
+sub api {
+ return 10;
+}
+
+sub type {
+ return 'sshfs';
+}
+
+sub plugindata {
+ return {
+ content => [
+ {
+ images => 1,
+ rootdir => 1,
+ vztmpl => 1,
+ iso => 1,
+ backup => 1,
+ snippets => 1,
+ import => 1,
+ none => 1,
+ },
+ {
+ images => 1,
+ rootdir => 1,
+ },
+ ],
+ format => [
+ {
+ raw => 1,
+ qcow2 => 1,
+ vmdk => 1,
+ },
+ 'qcow2',
+ ],
+ };
+}
+
+sub properties {
+ return {
+ 'sshfs-remote-path' => {
+ description => "Path on the remote filesystem used for SSHFS. Must be absolute.",
+ type => 'string',
+ format => 'pve-storage-path',
+ },
+ };
+}
+
+sub options {
+ return {
+ path => { fixed => 1 },
+ 'content-dirs' => { optional => 1 },
+ nodes => { optional => 1 },
+ shared => { optional => 1 },
+ disable => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
+ 'max-protected-backups' => { optional => 1 },
+ content => { optional => 1 },
+ format => { optional => 1 },
+ 'create-base-path' => { optional => 1 },
+ 'create-subdirs' => { optional => 1 },
+ bwlimit => { optional => 1 },
+ preallocation => { optional => 1 },
+ # SSH Options
+ username => {},
+ server => {},
+ 'sshfs-remote-path' => {},
+ port => { optional => 1 },
+ };
+}
+
+# SSHFS Helpers
+
+my sub sshfs_is_mounted: prototype($$) ($scfg, $cache) {
+ my ($user, $host, $remote_path) = $scfg->@{qw(username server sshfs-remote-path)};
+ my $mountpoint = $scfg->{path};
+
+ $cache->{mountdata} = PVE::ProcFSTools::parse_proc_mounts()
+ if !$cache->{mountdata};
+
+ my $ssh_url = "${user}\@${host}:${remote_path}";
+
+ my $has_found_mountpoint = grep {
+ $_->[0] =~ m|^\Q${ssh_url}\E$|
+ && $_->[1] eq $mountpoint
+ && $_->[2] eq 'fuse.sshfs'
+ } $cache->{mountdata}->@*;
+
+ return $has_found_mountpoint != 0;
+}
+
+my sub sshfs_mount: prototype($) ($scfg) {
+ my ($user, $host, $remote_path, $port) = $scfg->@{qw(username server sshfs-remote-path port)};
+ my $mountpoint = $scfg->{path};
+
+ my $remote = "${user}\@${host}:${remote_path}";
+
+ my $cmd = [
+ '/usr/bin/sshfs',
+ @COMMON_SSH_OPTS,
+ ];
+
+ push($cmd->@*, '-p', $port) if $port;
+ push($cmd->@*, $remote, $mountpoint);
+
+ eval {
+ run_command(
+ $cmd,
+ timeout => 10,
+ errfunc => sub { warn "$_[0]\n"; },
+ );
+ };
+ if (my $err = $@) {
+ die "failed to mount sshfs storage '$remote' at '$mountpoint': $@\n";
+ }
+
+ die "sshfs storage '$remote' not mounted at '$mountpoint' despite reported success\n"
+ if ! sshfs_is_mounted($scfg, {});
+
+ return;
+}
+
+# Storage Implementation
+
+sub check_connection ($class, $storeid, $scfg) {
+ my ($user, $host, $port) = $scfg->@{qw(username server port)};
+
+ my $cmd = [
+ '/usr/bin/ssh',
+ '-T',
+ @COMMON_SSH_OPTS,
+ '-o', 'BatchMode=yes',
+ '-o', 'ConnectTimeout=5',
+ ];
+
+ push($cmd->@*, "-p", $port) if $port;
+ push($cmd->@*, "${user}\@${host}", 'exit 0');
+
+ eval {
+ run_command(
+ $cmd,
+ timeout => 10,
+ errfunc => sub { warn "$_[0]\n"; },
+ );
+ };
+ if (my $err = $@) {
+ warn "$err";
+ return 0;
+ }
+
+ return 1;
+}
+
+sub activate_storage ($class, $storeid, $scfg, $cache) {
+ my $mountpoint = $scfg->{path};
+
+ if (!sshfs_is_mounted($scfg, $cache)) {
+ $class->config_aware_base_mkdir($scfg, $mountpoint);
+
+ die "unable to activate storage '$storeid' - directory '$mountpoint' does not exist\n"
+ if ! -d $mountpoint;
+
+ sshfs_mount($scfg);
+ }
+
+ $class->SUPER::activate_storage($storeid, $scfg, $cache);
+ return;
+}
+
+sub get_volume_attribute {
+ return PVE::Storage::DirPlugin::get_volume_attribute(@_);
+}
+
+sub update_volume_attribute {
+ return PVE::Storage::DirPlugin::update_volume_attribute(@_);
+}
+
+sub get_import_metadata {
+ return PVE::Storage::DirPlugin::get_import_metadata(@_);
+}
+
+1;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel