From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id AA56E1FF13E for ; Wed, 01 Jul 2026 10:05:40 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id E1DF4214DE; Wed, 01 Jul 2026 10:04:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1782847222; x=1783452022; darn=lists.proxmox.com; h=content-type:subject:to:from:date:message-id:from:to:cc:subject :date:message-id:reply-to:content-type; bh=IxXoN2QNN2C4qK7yExwG120SwSQo3xjuqqUo1fa6wnU=; b=DbAMWbEX4Xc2b4D8GFDjp+XEb8NGKyKmi0dKGqOwbAvTFY9c/kFayTU6vA5l93e55o P5f6e9Tm6m3WyOTQq9ZZ+Z2Od+lgfX1S6udVJ+rZz/R1VdlFDx9gn+bfLaZ6OQ2Hi5+5 neg7v/X4gRZR/Pgl2WXSQWRWpVp411d3OaQ9/a00imBS+XqFwoE+qPuYqzYeCFGXYCu2 GWe0Ai8Y54xB9wBaSIPOrZp52GIU2nn5/vOKF//qpLc5OuKIsaYS2UdXgXKn8ozkpSia d0dLYlgfzfNGUL8pBeFwHGeRsQxb5UIjvN8+qzOgCW+BEnALJNwCjw8RE8I1HsPQYuV6 lDpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782847222; x=1783452022; h=content-type:subject:to:from:date:message-id:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=IxXoN2QNN2C4qK7yExwG120SwSQo3xjuqqUo1fa6wnU=; b=lq4zKEM9b+/+GlkH6nk/NNJI2/EQuiYoKiOK8QH7hRicNGUlY61GexjDzNKkfamhSN Nl/9AQVFzjpvjqdtwPKaqllhTT1FMOOKYPgrOShpDo/fUoq6K0Rmcs+F+cAkRq4QK5yT XN+1Gybb28ZIREUndxB+TAZzNI0tp8a4AQXsKuxzZsdB+7yD/Lj1FN01+qpugpxPJz/4 QWbGYp+uN5+5kEpT6zS26bgOseOm4K7ckL+TYywidTDAEnAyk4duOCDVKe2bjU4LJJ5E 5ydlgUdm4YcS/7AbQLr4eM9tpuukJu/SO6DVqM6BX1/w+Ga4D4pQpNJFxtuTXgc0Vk9a b9hw== X-Gm-Message-State: AOJu0Yys2I1KznXv5n9DikHvVlcPUbpqFpF/L9hbRjcf/zl8fUrAUa2G SoD34ki1WQE2yUJmXBBwJ93jN8LN9WuZcL4uZHJDveVaQ2+SeXrbBp4nAkP1sDoDC6M= X-Gm-Gg: AfdE7cmGp9eNm425G/f9pJ1Q3mP5APxSq0Kg8vXV5UtilCsXdv3U1a4jlK/uypgsa2N yErHgF0XL2U3iamIEQ4JWBYgaOg5JStj1FLdjmUNZuuqhKJjzmfPh20zSE3al00vdFakkvyMRnm +ltVusR2A3as66NH1hEldMb5QyBaRG7LPDm8FHEqQHx0Tg3qKr9u4c+N2q+wHEGr/vKK28Gz+4U I11PeQ5SXBFAJWT7UDtpFOh5za9++dJyK/tUSQRn1dYOP8LXQTaehc3P2mA1nuvi/pquhSV+pZ+ ZSpKSWHoOWBJ7S9qkMuYF9LFR+O4zohGnjMoZIE6nWUsL5ZNvnVlgvxs9HB5Gy+P64sps7DxWmu g7Nx1YHWiL2qVhb+qqjtWxRQS9RApw01CI2j3PMb5hvYi01IEh8dAwBQfNYbuTp1KwXCck1t+AP SdmLkGPd8= X-Received: by 2002:a05:7300:df45:b0:30c:ab4d:382b with SMTP id 5a478bee46e88-30ee154009bmr3161381eec.45.1782847221965; Tue, 30 Jun 2026 12:20:21 -0700 (PDT) Message-ID: <6a4416f5.01f0a1da.1533d8.2fdf@mx.google.com> Date: Tue, 30 Jun 2026 12:20:21 -0700 (PDT) From: Ciro Iriarte To: pve-devel@lists.proxmox.com Subject: [RFC storage, qemu-server] offload full/live clone to storage backend Content-Type: text/plain; charset=UTF-8 X-SPAM-LEVEL: Spam detection results: 0 DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider KAM_ASCII_DIVIDERS 0.8 Email that uses ascii formatting dividers and possible spam tricks RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record X-MailFrom: cyruspy@gmail.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation Message-ID-Hash: EC4LGFF4CV3YKM3RCHS5UNU5JYB66AB4 X-Message-ID-Hash: EC4LGFF4CV3YKM3RCHS5UNU5JYB66AB4 X-Mailman-Approved-At: Wed, 01 Jul 2026 10:04:53 +0200 X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hello, I would like feedback on a storage-plugin contract that lets a backend perform full-clone copies itself, instead of PVE moving every block through the host. Decision I am asking about -------------------------- For a full clone where source and destination live on the same offload-capable storage, should PVE delegate the copy to the storage plugin (array-internal copy) rather than run qemu-img convert / drive-mirror on the host? If yes, I propose the minimal contract below. Background ---------- clone_image already offloads LINKED clones (CoW). Full clones do not come through it: clone_disk() does vdisk_alloc + qemu-img convert (offline) or drive-mirror (running), reading and writing the whole volume over the host even when the array could copy it internally in near-constant time with zero host I/O. volume_has_feature returns copy => 1 today, but that only means "may be copied", not "the storage can copy it itself" -- there is no hook for the storage to do the copy. Proposed contract (pve-storage) ------------------------------- - Add an optional plugin method, copy_image($scfg, $storeid, $src_volname, $vmid, $dst_format, $opts), that allocates and copies on the backend atomically and returns the new volname. Base class returns undef (unsupported), so existing plugins are unaffected -- the driver is the sole source of the capability. - Add a copy-offload feature to volume_has_feature, negotiated with the target storeid/format, so the driver also decides per copy whether this specific src->dst pair can be offloaded. - Add a per-storage copy-offload option as a policy override (default on), exposed only in the options of drivers that advertise the capability, so it can disable offload but never enable it where unsupported. Setting it to 0 routes that storage back to the host-assisted path with no other change. - Bump the storage APIVER/APIAGE. Consumer (qemu-server) ---------------------- - In clone_disk(), in the offline full path, when src and dst are the same storage, formats match, the disk is not a special disk (cloudinit/efidisk0/tpmstate0), and the plugin advertises copy-offload, call the new copy_image hook instead of qemu-img convert. Otherwise run the existing path unchanged. - A failed offload aborts the clone (existing rollback frees the target); it does not silently fall back to a host copy, so misbehaviour surfaces as a clean error rather than a bad volume. Live clone (running VM) ----------------------- A clone is not a migration: the source keeps running on its own volume, so the target only needs a consistent point-in-time copy, not drive-mirror convergence. For a running VM, quiesce the guest once around the whole disk set (guest-fsfreeze-freeze) at the clone loop level, flush QEMU's caches, run the array copy per disk, then thaw. Online offload runs only when the freeze succeeds, giving an FS-consistent copy; if there is no agent or the freeze fails, fall back to drive-mirror. This needs no separate switch -- the same copy-offload option governs it, and the freeze requirement keeps it from ever producing a crash-consistent copy silently. Out of scope ------------ - Cross-storage and format-converting clones (inherently host-mediated). - VM move-disk needs no separate work: it already calls clone_disk(full) and its "same storage, same format" case is rejected, so it inherits this hook without ever mis-firing. - Container volumes (storage_migrate path) -- a possible follow-up. If the contract looks acceptable, I will follow up with [RFC PATCH] series against pve-storage and then qemu-server. My CLA is on file. Tracking/details: https://github.com/ciroiriarte/pve-FCLUPlugin/issues/11 Thanks, Ciro Iriarte