[PATCH v2 manager 0/1] ceph: mds: reimplement hotstandby via ceph fs set allow_standby_replay

[PATCH v2 manager 0/1] ceph: mds: reimplement hotstandby via ceph fs set allow_standby_replay

[Kefu Chai]

Changes since v1:
- Rebase on current master.
- Revamp commit message.

Kefu Chai (1):
  ceph: mds: reimplement hotstandby via ceph fs set allow_standby_replay

 PVE/API2/Ceph/MDS.pm | 39 ++++++++++++++++++++++++++++++++-------
 1 file changed, 32 insertions(+), 7 deletions(-)

-- 
2.47.3

[PATCH v2 manager 1/1] ceph: mds: reimplement hotstandby via ceph fs set allow_standby_replay

[Kefu Chai]

PVE was writing two per-MDS config options into ceph.conf on every MDS
creation:

  [mds.<id>]
  mds_standby_for_name = pve
  mds_standby_replay = true   (when hotstandby=1)

Neither exists in Ceph Squid or Tentacle; both are absent from
src/common/options/mds.yaml.in and silently ignored.

mds_standby_for_name = 'pve' was always wrong: the PVE default
filesystem name is 'cephfs', not 'pve' (FS.pm: $fs_name =
$param->{name} // 'cephfs'), so the option always pointed at a
nonexistent filesystem. The option is a no-op in modern Ceph regardless.

mds_standby_replay was the old per-daemon standby replay knob. The
feature still exists in Squid/Tentacle but moved to a per-filesystem
setting: 'ceph fs set <fsname> allow_standby_replay true'. So the old
key has had no effect since Squid.

Fix:
- Drop the unconditional mds_standby_for_name write.
- When 'hotstandby' is set, call 'ceph fs set <filesystem>
  allow_standby_replay true' instead. A new optional 'filesystem'
  parameter (defaults to 'cephfs') names the target filesystem.
- If the mon command fails, warn and continue: the MDS can serve as a
  standby regardless, and standby replay can be enabled later.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
---
 PVE/API2/Ceph/MDS.pm | 39 ++++++++++++++++++++++++++++++++-------
 1 file changed, 32 insertions(+), 7 deletions(-)

diff --git a/PVE/API2/Ceph/MDS.pm b/PVE/API2/Ceph/MDS.pm
index 31b6fb7e..9025802b 100644
--- a/PVE/API2/Ceph/MDS.pm
+++ b/PVE/API2/Ceph/MDS.pm
@@ -151,8 +151,18 @@ __PACKAGE__->register_method({
                 optional => 1,
                 default => 0,
                 description =>
-                    "Determines whether a ceph-mds daemon should poll and replay the log of an active MDS. "
-                    . "Faster switch on MDS failure, but needs more idle resources.",
+                    "Use together with 'filesystem' to enable standby-replay "
+                    . "for the given CephFS. Keeps a standby MDS replaying the "
+                    . "active MDS journal for faster failover.",
+            },
+            filesystem => {
+                type => 'string',
+                optional => 1,
+                default => 'cephfs',
+                pattern => qr|^[^:/\s]+$|,
+                description =>
+                    "The name of the CephFS filesystem to enable standby replay "
+                    . "for when 'hotstandby' is set. Defaults to 'cephfs'.",
             },
         },
     },
@@ -194,11 +204,6 @@ __PACKAGE__->register_method({
             }
 
             $cfg->{$section}->{host} = $nodename;
-            $cfg->{$section}->{'mds_standby_for_name'} = 'pve';
-
-            if ($param->{hotstandby}) {
-                $cfg->{$section}->{'mds_standby_replay'} = 'true';
-            }
 
             cfs_write_file('ceph.conf', $cfg);
 
@@ -214,6 +219,26 @@ __PACKAGE__->register_method({
 
                 die "$err\n";
             }
+
+            if ($param->{hotstandby}) {
+                my $fs_name = $param->{filesystem} // 'cephfs';
+                print "Enabling standby replay for filesystem '$fs_name'...\n";
+                eval {
+                    $rados->mon_command({
+                        prefix => 'fs set',
+                        fs_name => $fs_name,
+                        var => 'allow_standby_replay',
+                        val => 'true',
+                        format => 'plain',
+                    });
+                };
+                if (my $err = $@) {
+                    chomp $err;
+                    warn "Could not enable standby replay for '$fs_name': $err\n"
+                        . "Run 'ceph fs set $fs_name allow_standby_replay true'"
+                        . " manually once the filesystem exists.\n";
+                }
+            }
         };
 
         return $rpcenv->fork_worker('cephcreatemds', "mds.$mds_id", $authuser, $worker);
-- 
2.47.3