public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Ciro Iriarte <cyruspy@gmail.com>
To: pve-devel@lists.proxmox.com
Subject: [RFC PATCH docs] pvesm: document copy-offload and add it to the storage capability table
Date: Mon, 07 Sep 2026 02:06:38 -0300	[thread overview]
Message-ID: <20260907.docs.copyoffload@cyruspy.gmail.com> (raw)
In-Reply-To: <20260720.0.copyoffload@cyruspy.gmail.com>

The capability table is where people look to decide which storage to use, so a
full-clone offload belongs in it next to Snapshots rather than only in the option
list. Adds a "Clone offload" column and documents both `copy-offload` and
`copy-offload-timeout` among the common storage properties.

The column says what is implemented today, not what is theoretically possible, and
the footnote says so explicitly -- NFS and CIFS can both do server-side copies and
are marked `no` only because nothing calls them yet. Overstating it would send people
looking for a switch that does nothing.

Two caveats are called out because both are silent failures otherwise:

- dir only qualifies when the underlying filesystem supports reflinks, which is a
  property of how the filesystem was created (XFS needs reflink=1) rather than of the
  storage configuration, and is not visible from the PVE side at all.
- a qcow2 with a backing file is excluded, because a copy of an overlay keeps the
  overlay's dependency on its base. It would look like a working clone right up until
  the base is removed.

The timeout description follows the schema rather than paraphrasing it: the default is
one day, and the timer restarts on reported progress, so it bounds a stalled copy
rather than a slow one. A large disk on a busy backend will not trip it.

Documents the storage-side series on the matching `copy-image-offload` branch.

Generated-By: Claude (https://claude.ai)
Signed-off-by: Ciro Iriarte <ciro.iriarte+software@gmail.com>
Co-Authored-By: Claude <noreply@anthropic.com>
---
 pvesm.adoc | 68 +++++++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 52 insertions(+), 16 deletions(-)

diff --git a/pvesm.adoc b/pvesm.adoc
index 5bd24b2..daa6984 100644
--- a/pvesm.adoc
+++ b/pvesm.adoc
@@ -65,23 +65,23 @@ nodes that can be accessed as RBD (RADOS Block Device).
 
 
 .Available storage types
-[width="100%",cols="<2d,1*m,4*d",options="header"]
+[width="100%",cols="<2d,1*m,5*d",options="header"]
 |========================================================================
-|Description    |Plugin type |Level  |Shared|Snapshots|Stable
-|ZFS (local)    |zfspool     |both^1^|no    |yes      |yes
-|Directory      |dir         |file   |no    |yes^2^   |yes
-|BTRFS          |btrfs       |file   |no    |yes      |TP^5^
-|NFS            |nfs         |file   |yes   |yes^2^   |yes
-|CIFS           |cifs        |file   |yes   |yes^2^   |yes
-|Proxmox Backup |pbs         |both   |yes   |n/a      |yes
-|CephFS         |cephfs      |file   |yes   |yes      |yes
-|LVM            |lvm         |block  |no^3^ |yes^4^   |yes
-|LVM-thin       |lvmthin     |block  |no    |yes      |yes
-|iSCSI/kernel   |iscsi       |block  |yes^3^|yes^4^   |yes
-|iSCSI/libiscsi |iscsidirect |block  |yes^3^|yes^4^   |yes
-|FC/SAS         |native^6^   |block  |yes^3^|yes^4^   |yes
-|Ceph/RBD       |rbd         |block  |yes   |yes      |yes
-|ZFS over iSCSI |zfs         |block  |yes   |yes      |yes
+|Description    |Plugin type |Level  |Shared|Snapshots|Clone offload^7^|Stable
+|ZFS (local)    |zfspool     |both^1^|no    |yes      |no              |yes
+|Directory      |dir         |file   |no    |yes^2^   |yes^8^          |yes
+|BTRFS          |btrfs       |file   |no    |yes      |yes             |TP^5^
+|NFS            |nfs         |file   |yes   |yes^2^   |no              |yes
+|CIFS           |cifs        |file   |yes   |yes^2^   |no              |yes
+|Proxmox Backup |pbs         |both   |yes   |n/a      |n/a             |yes
+|CephFS         |cephfs      |file   |yes   |yes      |no              |yes
+|LVM            |lvm         |block  |no^3^ |yes^4^   |no              |yes
+|LVM-thin       |lvmthin     |block  |no    |yes      |yes             |yes
+|iSCSI/kernel   |iscsi       |block  |yes^3^|yes^4^   |no              |yes
+|iSCSI/libiscsi |iscsidirect |block  |yes^3^|yes^4^   |no              |yes
+|FC/SAS         |native^6^   |block  |yes^3^|yes^4^   |n/a             |yes
+|Ceph/RBD       |rbd         |block  |yes   |yes      |yes             |yes
+|ZFS over iSCSI |zfs         |block  |yes   |yes      |no              |yes
 |========================================================================
 
 ^1^: Disk images for VMs are stored in ZFS volume (zvol) datasets, which provide
@@ -112,6 +112,17 @@ xref:pvesm_lvm_config[LVM configuration] section.
 ^6^ Fibre Channel (FC) and SAS block storage is handled directly by the host
 without a dedicated storage plugin.
 
+^7^ Whether a *full* clone can be handed to the storage instead of being copied
+byte by byte by the host. Off by default; enable it per storage with the
+`copy-offload` option. This only applies within one storage backend -- copying
+between two different backends always goes through the host. Storages marked
+`no` are not necessarily incapable, they simply have no implementation yet.
+
+^8^ Only when the underlying filesystem supports reflinks (XFS created with
+`reflink=1`, btrfs, or ZFS with block cloning enabled), and not for a 'qcow2'
+image that has a backing file, since copying such an image would reproduce its
+dependency on the base rather than a standalone disk.
+
 
 Thin Provisioning
 ~~~~~~~~~~~~~~~~~
@@ -277,6 +288,31 @@ file-based storages. The default is `metadata`, which is treated like `off` for
 `raw` images. When using network storages in combination with large `qcow2`
 images, using `off` can help to avoid timeouts.
 
+copy-offload::
+
+Let the storage perform full clones itself instead of the host copying the image
+byte by byte with `qemu-img convert`. Disabled by default. Depending on the
+backend this can make a full clone close to instant and cost little or no extra
+space, because the copy shares blocks with its source copy-on-write. The result
+is still a normal, independent disk: the source can be deleted afterwards.
+
+It only takes effect when both storages are instances of the same plugin and
+that plugin supports it -- see the ``Clone offload'' column above. Anything else
+falls back to the usual host-side copy, so turning this on is safe even where it
+cannot be used.
+
+copy-offload-timeout::
+
+How long to wait, in seconds, for an offloaded copy to become independent of its
+source before giving up and cleaning up the unfinished target (default: 86400,
+one day). The timer restarts every time the backend reports progress, so this
+bounds a copy that has stalled rather than one that is merely slow -- a large
+disk on a busy backend will not trip it.
+
+Only relevant for backends that copy in the background, such as Ceph/RBD, where
+the clone is readable immediately but has to be flattened before it stops
+depending on its source. Backends that finish instantly never reach it.
+
 WARNING: It is not advisable to use the same storage pool on different
 {pve} clusters. Some storage operation need exclusive access to the
 storage, so proper locking is required. While this is implemented
-- 
2.54.0




      parent reply	other threads:[~2026-09-07  5:09 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-20 14:29 [RFC PATCH storage, qemu-server 0/5] offload full clone to the storage backend Ciro Iriarte
2026-07-20 14:29 ` [RFC PATCH storage 1/5] storage: add asynchronous copy-offload hook for full copies Ciro Iriarte
2026-07-20 14:29 ` [RFC PATCH storage 2/5] rbd: implement copy-offload via snapshot + clone + flatten Ciro Iriarte
2026-07-20 14:30 ` [RFC PATCH storage 3/5] dir: implement copy-offload via reflink (FICLONE) Ciro Iriarte
2026-07-20 14:30 ` [RFC PATCH storage 4/5] btrfs, lvmthin: implement copy-offload (atomic class) Ciro Iriarte
2026-07-20 14:30 ` [RFC PATCH qemu-server 5/5] use storage copy-offload for full clone Ciro Iriarte
2026-07-22 15:34 ` [RFC PATCH storage] rbd: fix copy-offload flatten status race on fast small copies Ciro Iriarte
2026-09-07  5:06 ` Ciro Iriarte [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260907.docs.copyoffload@cyruspy.gmail.com \
    --to=cyruspy@gmail.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal