From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 63D061FF0B7 for ; Fri, 16 Oct 2026 14:26:52 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id EB7DE214C9; Fri, 16 Oct 2026 14:26:50 +0200 (CEST) From: Erik Fastermann To: pve-devel@lists.proxmox.com Subject: [PATCH docs/manager/qemu-server v3 0/6] partially fix #1989: add QEMU qcow2 cache options Date: Fri, 16 Oct 2026 14:25:56 +0200 Message-ID: <20261016122602.183911-1-e.fastermann@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1792153606229 X-SPAM-LEVEL: Spam detection results: 0 AWL 1.686 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: DGQZTFMJ5RX56SPKOPIMI7VTRUPRNMLF X-Message-ID-Hash: DGQZTFMJ5RX56SPKOPIMI7VTRUPRNMLF X-MailFrom: e.fastermann@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Erik Fastermann X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Add multiple options to configure the qcow2 L2/refcount cache. This can provide significant performance gains in some cases, mainly for large disks. The pve-manager patch requires a qemu-server version that includes patch 4, since older versions reject the new qcow2-* drive properties. A v2 of the manager patch was applied and reverted (43b51f49) for that reason. Patches 1-3 are new and do not depend on the cache options: 1: re-run proxmox-perltidy, which reformats two calls in QemuServer.pm on current master. No functional change. 2: fix the bps, bps_rd and bps_wr throttle options, which were converted to 0 mbps (unlimited) instead of their actual value. 3: make the API report why parse_drive rejects a drive. Previously, the web UI only showed "unable to parse drive options" when one of the new qcow2 checks, or one of the existing checks, failed. Changes since v2: qemu-server (patch 4): * Legacy -drive commandline (machine version < 10.0): prefix the options with "file." during a live restore, so they are set on the qcow2 node. Previously they ended up on the alloc-track node, which rejects them, so the live restore could not start. * Handle the undefined format of empty CD-ROM drives. This caused an uninitialized value warning on every start and broke some cfg2cmd tests. * Automatic sizing uses the virtual size reported by qemu-img info instead of the optional size= property. Without size=, the cache fell back to its minimum regardless of the disk size. * Call qemu-img info with the qcow2 format and a 10 second timeout. * Limit qcow2-cache-clean-interval to 2^32 - 1, since QEMU refuses to start with larger values, and describe the option in more detail. * Report validation errors via the mechanism from patch 3 and use the full property names in the messages. * The QEMU default L2 cache is up to 32 MiB, not always 32 MiB. * Drop an unused log_warn import from Blockdev.pm. * Rebase onto current master. pve-manager (patch 5): * Keep the underscore in the drive parser's key regex. Without it, disks with throttling options (mbps_rd, iops_wr_max, ...) could no longer be edited. * Enabling automatic sizing removes qcow2-l2-cache-size and qcow2-refcount-cache-size if they were set via the API or CLI, since the backend rejects that combination. * Parse the automatic sizing flag with PVE.Parser.parseBoolean like the neighboring fields. * Tooltip: the QEMU default L2 cache is up to 32 MiB. * Only accept whole numbers up to 10240 in the qcow2 cache size field, matching what the backend accepts. pve-docs (patch 6): * Name the options that are not exposed in the web UI and mention qm set. * Note that QEMU allocates the cache in addition to the guest memory. * Mention that the options mainly help with large disks. Known limitations: * Snapshots as volume chain: every qcow2 node of the backing chain gets the same cache options. With automatic sizing, every layer gets the cache size computed for the whole disk, so the worst-case memory use grows with the number of snapshots (for a 2 TiB disk with three snapshots, 4 x 320 MiB instead of 320 MiB), and qemu-img info runs once per layer. I would be inclined to leave it this way for now. * Moving a disk to a storage that does not use qcow2 keeps the qcow2-* options, so the move and every later VM start log a warning for each of them. The options could be dropped silently on move instead, but the warning is also useful to have. The downside is that the warning cannot be cleared from the web UI, since the qcow2 fields are disabled for non-qcow2 disks, so the options can only be removed via the API or CLI. * The disk dialog only exposes the total cache size. If qcow2-l2-cache-size or qcow2-refcount-cache-size were set via the API or CLI and are not smaller than the entered total, or both are set, saving fails. I think this is acceptable for now, and with patch 3 the error message states which options conflict. * With patch 3, problems for which parse_drive previously returned undef silently are now logged as warnings for callers that do not collect the errors. This includes the early returns for an invalid drive key and for a drive string that cannot be parsed. The warning adds useful information, so I would keep it. Thanks to Fiona and Dominik for the feedback on the previous versions! qemu-server: Erik Fastermann (4): re-tidy source code drive: fix conversion of bps throttle limits to mbps api: report why drive options are rejected partially fix #1989: disk: add qcow2 cache options src/PVE/API2/Qemu.pm | 9 +- src/PVE/QemuServer.pm | 47 +++++- src/PVE/QemuServer/Blockdev.pm | 7 + src/PVE/QemuServer/Drive.pm | 263 ++++++++++++++++++++++++++++++--- 4 files changed, 302 insertions(+), 24 deletions(-) pve-manager: Erik Fastermann (1): partially fix #1989: ui: qemu: disk: add qcow2 cache size config www/manager6/Parser.js | 2 +- www/manager6/qemu/HDEdit.js | 81 +++++++++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+), 1 deletion(-) pve-docs: Erik Fastermann (1): partially fix #1989: qm: document qcow2 cache size config qm.adoc | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) Summary over all repositories: 7 files changed, 400 insertions(+), 25 deletions(-) -- Generated by murpp 0.12.1