public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Jonas Theisen <j.theisen@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH storage 1/2] rbd: only set image config if the value differs
Date: Fri, 20 Nov 2026 15:10:06 +0100	[thread overview]
Message-ID: <20261120141020.243160-2-j.theisen@proxmox.com> (raw)
In-Reply-To: <20261120141020.243160-1-j.theisen@proxmox.com>

Starting a VM with an EFI disk on RBD sets the rbd_cache_policy image
config on every start, including the target side of a live migration.
At that point, the source VM still holds the exclusive lock, so librbd
forwards the metadata write to the lock owner, waits for it to report
completion, and re-sends the request every 30 seconds until it does.
The 'rbd config image set' command can then hang indefinitely if:

- the lock owner never reports completion, or
- its completion arrives just after a re-send timed out, but before
  the request is sent again. The requester drops the completion, and
  the lock owner answers every further re-send of the finished request
  with its result, 0 on success. That is the same answer as for a
  request that is still in progress, so the requester keeps waiting
  for a completion that is never sent again.

The VM start on the target, and thus the migration, then hangs until
the rbd process is killed.

Reading the image config does not involve the lock owner, and after
the first start of the VM the value is already set. Check it first and
only write it if it differs, so migrations do not need the lock owner
anymore in the common case.

Signed-off-by: Jonas Theisen <j.theisen@proxmox.com>
---
 src/PVE/Storage/RBDPlugin.pm | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/src/PVE/Storage/RBDPlugin.pm b/src/PVE/Storage/RBDPlugin.pm
index 26a1cf8..1b2caf8 100644
--- a/src/PVE/Storage/RBDPlugin.pm
+++ b/src/PVE/Storage/RBDPlugin.pm
@@ -398,9 +398,26 @@ my sub rbd_volume_exists {
 }
 
 # Needs to be public, so qemu-server can mock it for cfg2cmd.
+#
+# Skips the write if the value is already set. Writing the image config is forwarded to the
+# exclusive lock owner, e.g. the source VM during live migration, and can block indefinitely if
+# the owner does not complete the request. Reading it does not involve the lock owner.
 sub rbd_volume_config_set {
     my ($scfg, $storeid, $volname, $key, $value) = @_;
 
+    my $current;
+    my $get_cmd = $rbd_cmd->($scfg, $storeid, 'config', 'image', 'get', $volname, $key);
+    # fails if the key is not set yet, so just fall back to setting it on any error
+    eval {
+        run_rbd_command(
+            $get_cmd,
+            errmsg => "rbd config image get $volname $key error",
+            errfunc => sub { },
+            outfunc => sub { $current = shift },
+        );
+    };
+    return if !$@ && defined($current) && $current eq $value;
+
     my $cmd = $rbd_cmd->($scfg, $storeid, 'config', 'image', 'set', $volname, $key, $value);
     run_rbd_command($cmd, errmsg => "rbd config image set $volname $key $value error");
 
@@ -616,9 +633,6 @@ sub qemu_blockdev_options {
 
     # SPI flash does lots of read-modify-write OPs, without writeback this gets really slow #3329
     if ($options->{hints}->{'efi-disk'}) {
-        # Querying the value would just cost more and the 'rbd image config get' command will just
-        # fail if the config has not been set yet, so it's not even straight-forward to do so.
-        # Simply set the value (possibly again).
         rbd_volume_config_set($scfg, $storeid, $name, 'rbd_cache_policy', 'writeback');
     }
 
-- 
2.47.3





  reply	other threads:[~2026-11-20 14:10 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-20 14:10 [PATCH storage 0/2] rbd: avoid hanging VM start on EFI disk cache policy Jonas Theisen
2026-11-20 14:10 ` Jonas Theisen [this message]
2026-11-20 14:10 ` [PATCH storage 2/2] rbd: do not let setting the EFI disk cache policy block VM start Jonas Theisen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120141020.243160-2-j.theisen@proxmox.com \
    --to=j.theisen@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal