From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id F08C1737C1 for ; Tue, 6 Jul 2021 18:45:31 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E6B842576F for ; Tue, 6 Jul 2021 18:45:31 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id ECB2025766 for ; Tue, 6 Jul 2021 18:45:30 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id B6DF340E9B for ; Tue, 6 Jul 2021 18:45:30 +0200 (CEST) From: Stoiko Ivanov To: pve-devel@lists.proxmox.com Date: Tue, 6 Jul 2021 18:45:18 +0200 Message-Id: <20210706164518.3517966-1-s.ivanov@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.104 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_ASCII_DIVIDERS 0.8 Spam that uses ascii formatting tricks KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: [pve-devel] [PATCH docs] storage: add minimal zfs over iscsi doc X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 16:45:32 -0000 mostly copied from the wiki-page[0], and adapted to include LIO as target provider. Additionally I added a note to explain that the plugin needs ZFS on the target side (and does not make your SAN speak ZFS) Tested during the PVE 7.0 tests for the plugin I did. [0] https://pve.proxmox.com/wiki/Storage:_ZFS_over_iSCSI Signed-off-by: Stoiko Ivanov --- I plan on adding this about once per Debian release (while testing and wondering why we don't have that in our reference docs) - but the plan usually gets replaced by something more urgent (and fun). pve-storage-zfs.adoc | 139 +++++++++++++++++++++++++++++++++++++++++++ pvesm.adoc | 2 + 2 files changed, 141 insertions(+) create mode 100644 pve-storage-zfs.adoc diff --git a/pve-storage-zfs.adoc b/pve-storage-zfs.adoc new file mode 100644 index 0000000..6801873 --- /dev/null +++ b/pve-storage-zfs.adoc @@ -0,0 +1,139 @@ +[[storage_zfs]] +ZFS over ISCSI Backend +---------------------- +ifdef::wiki[] +:pve-toplevel: +:title: Storage: ZFS over ISCSI +endif::wiki[] + +Storage pool type: `zfs` + +This backend accesses a remote machine having a ZFS pool as storage and an iSCSI +target implementation via `ssh`. For each guest disk it creates a ZVOL and, +exports it as iSCSI LUN. This LUN is used by {pve} for the guest disk. + +The following iSCSI target implementations are supported: + +* LIO (Linux) +* IET (Linux) +* ISTGT (FreeBSD) +* Comstar (Solaris) + +NOTE: This plugin needs a ZFS capable remote storage appliance, you cannot use +it to create a ZFS Pool on a regular Storage Appliance/SAN + + +Configuration +~~~~~~~~~~~~~ + +In order to use the ZFS over iSCSI plugin you need to configure the remote +machine (target) to accept `ssh` connections from the {pve} node. {pve} connects to the target for creating the ZVOLs and exporting them via iSCSI. +Authentication is done through a ssh-key (without password protection) stored in +`/etc/pve/priv/zfs/_id_rsa` + +The following steps create a ssh-key and distribute it to the storage machine +with IP 192.0.2.1: + +---- +mkdir /etc/pve/priv/zfs +ssh-keygen -f /etc/pve/priv/zfs/192.0.2.1_id_rsa +ssh-copy-id -i /etc/pve/priv/zfs/192.0.2.1_id_rsa.pub root@192.0.2.1 +ssh -i /etc/pve/priv/zfs/192.0.2.1_id_rsa root@192.0.2.1 +---- + +The backend supports the common storage properties `content`, `nodes`, +`disable`, and the following ZFS over ISCSI specific properties: + +pool:: + +The ZFS pool/filesystem on the iSCSI target. All allocations are done within that +pool. + +portal:: + +iSCSI portal (IP or DNS name with optional port). + +target:: + +iSCSI target. + +iscsiprovider:: + +The iSCSI target implementation used on the remote machine + +comstar_tg:: + +target group for comstar views. + +comstar_hg:: + +host group for comstar views. + +lio_tpg:: + +target portal group for Linux LIO targets + +nowritecache:: + +disable write caching on the target + +blocksize:: + +Set ZFS blocksize parameter. + +sparse:: + +Use ZFS thin-provisioning. A sparse volume is a volume whose +reservation is not equal to the volume size. + + +.Configuration Examples (`/etc/pve/storage.cfg`) +---- +zfs: lio + blocksize 4k + iscsiprovider LIO + pool tank + portal 192.0.2.111 + target iqn.2003-01.org.linux-iscsi.lio.x8664:sn.xxxxxxxxxxxx + content images + lio_tpg tpg1 + sparse 1 + +zfs: solaris + blocksize 4k + target iqn.2010-08.org.illumos:02:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:tank1 + pool tank + iscsiprovider comstar + portal 192.0.2.112 + content images + +zfs: freebsd + blocksize 4k + target iqn.2007-09.jp.ne.peach.istgt:tank1 + pool tank + iscsiprovider istgt + portal 192.0.2.113 + content images + +zfs: iet + blocksize 4k + target iqn.2001-04.com.example:tank1 + pool tank + iscsiprovider iet + portal 192.0.2.114 + content images +---- + +Storage Features +~~~~~~~~~~~~~~~~ + +The ZFS over iSCSI plugin provides a shared storage, which is capable of +snapshots. You need to make sure that the ZFS appliance does not become a single +point of failure in your deployment. + +.Storage features for backend `iscsi` +[width="100%",cols="m,m,3*d",options="header"] +|============================================================================== +|Content types |Image formats |Shared |Snapshots |Clones +|images |raw |yes |yes |no +|============================================================================== diff --git a/pvesm.adoc b/pvesm.adoc index c8e2347..98c8c44 100644 --- a/pvesm.adoc +++ b/pvesm.adoc @@ -436,6 +436,8 @@ include::pve-storage-cephfs.adoc[] include::pve-storage-btrfs.adoc[] +include::pve-storage-zfs.adoc[] + ifdef::manvolnum[] include::pve-copyright.adoc[] -- 2.30.2