From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 911CF1FF172
for <inbox@lore.proxmox.com>; Wed, 16 Apr 2025 14:48:10 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 5F55836643;
Wed, 16 Apr 2025 14:48:08 +0200 (CEST)
From: Max Carrara <m.carrara@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Wed, 16 Apr 2025 14:47:33 +0200
Message-Id: <20250416124735.320256-1-m.carrara@proxmox.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.078 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [proxmox.com, sshfsplugin.pm]
Subject: [pve-devel] [PATCH v1 pve-storage 0/2] SSHFS Example Storage Plugin
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
SSHFS Example Storage Plugin - v1
=================================
Add a custom storage plugin based on SSHFS [0] to serve as an example
for an upcoming storage plugin development guide. This plugin should
also be ready for production usage, though it would be nice to get some
more testing (and potentially performance-tuning) done.
The plugin demonstrates how the following things should be handled:
- handling sensitive properties (shout-out to Fiona [1])
- the private key is copied to /etc/pve/priv/storage/${PRIV_KEY}.key
instead of being put into /etc/pve/storage.cfg
- on removal of the storage, the private key is also removed
- though not really that nicely supported via the CLI, updating the
private key file is also handled
- making use of /etc/pve/priv/known_hosts for easy setup on a cluster
and marking the storage as shared automatically
- connection checking (since it's a network storage)
- conditionally creating the mountpoint and mounting the storage if not
already mounted
- unmounting the storage on removal (not deactivation!)
- handling "notes" and "protected" attributes for backups
Furthermore, the plugin lives in its own subdir inside `pve-storage`,
complete with its own `debian/` dir and `Makefile` in order to
demonstrate how to package a plugin for Debian.
NOTE: The plugin is *not* built via the top-level `Makefile` at the
moment.
How to Use & Test
-----------------
You'll need some kind guest you can SSH into with pubkey auth.
On that host, it's best to create some directory for SSHFS, e.g.:
mkdir -p /srv/sshfs && chmod 750 /srv/sshfs
The plugin can easily be built via `make deb` inside
`example/sshfs-plugin`. Once built and installed on a different PVE host
of your choice, you should first deploy a private key for SSHFS, e.g.:
ssh-copy-id -i ~/.ssh/id_my_private_key \
-o UserKnownHostsFile=/etc/pve/priv/known_hosts [USER]@[HOST]
Then, using the deployed key, the storage can be added as follows:
pvesm add sshfs [STOREID] \
--username [USER] \
--server [HOST] \
--sshfs-remote-path /srv/sshfs \
--path /mnt/path/to/storage \
--sshfs-private-key ~/.ssh/id_my_private_key
The storage should then pop up in the UI. It should be functionally
equivalent to most other dir-based network storages, except that imports
aren't supported in order to keep the example from becoming too complex.
Would be nice if somebody could give this a spin! I think there's
definitely room for some performance improvements, but so far the
storage was doing *alright*. IOPS are terrible, though.
References
----------
[0]: https://github.com/libfuse/sshfs
[1]: https://lore.proxmox.com/pve-devel/20250404133204.239783-6-f.ebner@proxmox.com/
Older Versions
--------------
rfc-v1: https://lore.proxmox.com/pve-devel/20250328171209.503132-1-m.carrara@proxmox.com/
Summary of Changes
------------------
Max Carrara (2):
example: sshfs plugin: add custom storage plugin for SSHFS
example: sshfs plugin: package SSHFSPlugin.pm
example/sshfs-plugin/Makefile | 71 ++++
example/sshfs-plugin/debian/changelog | 5 +
example/sshfs-plugin/debian/control | 22 +
example/sshfs-plugin/debian/copyright | 21 +
example/sshfs-plugin/debian/rules | 26 ++
example/sshfs-plugin/debian/source/format | 1 +
example/sshfs-plugin/debian/triggers | 1 +
.../lib/PVE/Storage/Custom/SSHFSPlugin.pm | 398 ++++++++++++++++++
8 files changed, 545 insertions(+)
create mode 100644 example/sshfs-plugin/Makefile
create mode 100644 example/sshfs-plugin/debian/changelog
create mode 100644 example/sshfs-plugin/debian/control
create mode 100644 example/sshfs-plugin/debian/copyright
create mode 100755 example/sshfs-plugin/debian/rules
create mode 100644 example/sshfs-plugin/debian/source/format
create mode 100644 example/sshfs-plugin/debian/triggers
create mode 100644 example/sshfs-plugin/lib/PVE/Storage/Custom/SSHFSPlugin.pm
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel