From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <s.ivanov@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id F08C1737C1
 for <pve-devel@lists.proxmox.com>; Tue,  6 Jul 2021 18:45:31 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id E6B842576F
 for <pve-devel@lists.proxmox.com>; Tue,  6 Jul 2021 18:45:31 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [94.136.29.106])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS id ECB2025766
 for <pve-devel@lists.proxmox.com>; Tue,  6 Jul 2021 18:45:30 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id B6DF340E9B
 for <pve-devel@lists.proxmox.com>; Tue,  6 Jul 2021 18:45:30 +0200 (CEST)
From: Stoiko Ivanov <s.ivanov@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Tue,  6 Jul 2021 18:45:18 +0200
Message-Id: <20210706164518.3517966-1-s.ivanov@proxmox.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results:  0
 AWL 0.104 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 KAM_ASCII_DIVIDERS        0.8 Spam that uses ascii formatting tricks
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
 information. [proxmox.com]
Subject: [pve-devel] [PATCH docs] storage: add minimal zfs over iscsi doc
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Tue, 06 Jul 2021 16:45:32 -0000

mostly copied from the wiki-page[0], and adapted to include LIO as
target provider.

Additionally I added a note to explain that the plugin needs ZFS on
the target side (and does not make your SAN speak ZFS)

Tested during the PVE 7.0 tests for the plugin I did.

[0] https://pve.proxmox.com/wiki/Storage:_ZFS_over_iSCSI

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
I plan on adding this about once per Debian release (while testing and
wondering why we don't have that in our reference docs) - but the plan
usually gets replaced by something more urgent (and fun).

 pve-storage-zfs.adoc | 139 +++++++++++++++++++++++++++++++++++++++++++
 pvesm.adoc           |   2 +
 2 files changed, 141 insertions(+)
 create mode 100644 pve-storage-zfs.adoc

diff --git a/pve-storage-zfs.adoc b/pve-storage-zfs.adoc
new file mode 100644
index 0000000..6801873
--- /dev/null
+++ b/pve-storage-zfs.adoc
@@ -0,0 +1,139 @@
+[[storage_zfs]]
+ZFS over ISCSI Backend
+----------------------
+ifdef::wiki[]
+:pve-toplevel:
+:title: Storage: ZFS over ISCSI
+endif::wiki[]
+
+Storage pool type: `zfs`
+
+This backend accesses a remote machine having a ZFS pool as storage and an iSCSI
+target implementation via `ssh`. For each guest disk it creates a ZVOL and,
+exports it as iSCSI LUN. This LUN is used by {pve} for the guest disk.
+
+The following iSCSI target implementations are supported:
+
+* LIO (Linux)
+* IET (Linux)
+* ISTGT (FreeBSD)
+* Comstar (Solaris)
+
+NOTE: This plugin needs a ZFS capable remote storage appliance, you cannot use
+it to create a ZFS Pool on a regular Storage Appliance/SAN
+
+
+Configuration
+~~~~~~~~~~~~~
+
+In order to use the ZFS over iSCSI plugin you need to configure the remote
+machine (target) to accept `ssh` connections from the {pve} node. {pve} connects to the target for creating the ZVOLs and exporting them via iSCSI.
+Authentication is done through a ssh-key (without password protection) stored in
+`/etc/pve/priv/zfs/<target_ip>_id_rsa`
+
+The following steps create a ssh-key and distribute it to the storage machine
+with IP 192.0.2.1:
+
+----
+mkdir /etc/pve/priv/zfs
+ssh-keygen -f /etc/pve/priv/zfs/192.0.2.1_id_rsa
+ssh-copy-id -i /etc/pve/priv/zfs/192.0.2.1_id_rsa.pub root@192.0.2.1
+ssh -i /etc/pve/priv/zfs/192.0.2.1_id_rsa root@192.0.2.1
+----
+
+The backend supports the common storage properties `content`, `nodes`,
+`disable`, and the following ZFS over ISCSI specific properties:
+
+pool::
+
+The ZFS pool/filesystem on the iSCSI target. All allocations are done within that
+pool.
+
+portal::
+
+iSCSI portal (IP or DNS name with optional port).
+
+target::
+
+iSCSI target.
+
+iscsiprovider::
+
+The iSCSI target implementation used on the remote machine
+
+comstar_tg::
+
+target group for comstar views.
+
+comstar_hg::
+
+host group for comstar views.
+
+lio_tpg::
+
+target portal group for Linux LIO targets
+
+nowritecache::
+
+disable write caching on the target
+
+blocksize::
+
+Set ZFS blocksize parameter.
+
+sparse::
+
+Use ZFS thin-provisioning. A sparse volume is a volume whose
+reservation is not equal to the volume size.
+
+
+.Configuration Examples (`/etc/pve/storage.cfg`)
+----
+zfs: lio
+   blocksize 4k
+   iscsiprovider LIO
+   pool tank
+   portal 192.0.2.111
+   target iqn.2003-01.org.linux-iscsi.lio.x8664:sn.xxxxxxxxxxxx
+   content images
+   lio_tpg tpg1
+   sparse 1
+
+zfs: solaris
+   blocksize 4k
+   target iqn.2010-08.org.illumos:02:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:tank1
+   pool tank
+   iscsiprovider comstar
+   portal 192.0.2.112
+   content images
+
+zfs: freebsd
+   blocksize 4k
+   target iqn.2007-09.jp.ne.peach.istgt:tank1
+   pool tank
+   iscsiprovider istgt
+   portal 192.0.2.113
+   content images
+
+zfs: iet
+   blocksize 4k
+   target iqn.2001-04.com.example:tank1
+   pool tank
+   iscsiprovider iet
+   portal 192.0.2.114
+   content images
+----
+
+Storage Features
+~~~~~~~~~~~~~~~~
+
+The ZFS over iSCSI plugin provides a shared storage, which is capable of
+snapshots. You need to make sure that the ZFS appliance does not become a single
+point of failure in your deployment.
+
+.Storage features for backend `iscsi`
+[width="100%",cols="m,m,3*d",options="header"]
+|==============================================================================
+|Content types  |Image formats  |Shared |Snapshots |Clones
+|images         |raw            |yes    |yes        |no
+|==============================================================================
diff --git a/pvesm.adoc b/pvesm.adoc
index c8e2347..98c8c44 100644
--- a/pvesm.adoc
+++ b/pvesm.adoc
@@ -436,6 +436,8 @@ include::pve-storage-cephfs.adoc[]
 
 include::pve-storage-btrfs.adoc[]
 
+include::pve-storage-zfs.adoc[]
+
 
 ifdef::manvolnum[]
 include::pve-copyright.adoc[]
-- 
2.30.2