[pve-devel] [PATCH manager 0/5] pve6to7 ceph fixes

[pve-devel] [PATCH manager 0/5] pve6to7 ceph fixes

[Fabian Grünbichler]

reduce checks, adapt version guards, make the whole thing work with
pve-manager 7.x

last patch is stable-6 only, rest is for both branches.

Fabian Grünbichler (5):
  pve6to7: use new flags API
  pve6to7: remove PASS noise for ceph
  pve6to7: check for >= Octopus
  pve6to7: dont guard noout check on Ceph version
  pve6to7: enable noout before upgrade

 PVE/CLI/pve6to7.pm | 39 +++++++++------------------------------
 1 file changed, 9 insertions(+), 30 deletions(-)

-- 
2.30.2

[pve-devel] [PATCH manager 1/5] pve6to7: use new flags API

[Fabian Grünbichler]

the old one is not available post-upgrade, let's use a single codepath
for this.

the new API only allows querying user-settable flags, but the only flags
we check besides 'noout' are not relevant for an upgrade of PVE 6.x to
7.x (PVE 6.x only supports Nautilus+ which requires these flags to be
set in order to work) so we can just drop those outdated checks instead
of extending/refactoring the API.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 PVE/CLI/pve6to7.pm | 19 ++++++-------------
 1 file changed, 6 insertions(+), 13 deletions(-)

diff --git a/PVE/CLI/pve6to7.pm b/PVE/CLI/pve6to7.pm
index 60edac11..c693e0d4 100644
--- a/PVE/CLI/pve6to7.pm
+++ b/PVE/CLI/pve6to7.pm
@@ -8,6 +8,7 @@ use PVE::API2::Ceph;
 use PVE::API2::LXC;
 use PVE::API2::Qemu;
 use PVE::API2::Certificates;
+use PVE::API2::Cluster::Ceph;
 
 use PVE::AccessControl;
 use PVE::Ceph::Tools;
@@ -389,9 +390,12 @@ sub check_ceph {
 
     log_info("getting Ceph status/health information..");
     my $ceph_status = eval { PVE::API2::Ceph->status({ node => $nodename }); };
-    my $osd_flags = eval { PVE::API2::Ceph->get_flags({ node => $nodename }); };
+    my $noout = eval { PVE::API2::Cluster::Ceph->get_flag({ flag => "noout" }); };
+    if ($@) {
+	log_fail("failed to get 'noout' flag status - $@");
+    }
+
     my $noout_wanted = 1;
-    my $noout = $osd_flags && $osd_flags =~ m/noout/;
 
     if (!$ceph_status || !$ceph_status->{health}) {
 	log_fail("unable to determine Ceph status!");
@@ -409,17 +413,6 @@ sub check_ceph {
 	}
     }
 
-    log_info("getting Ceph OSD flags..");
-    eval {
-	if (!$osd_flags) {
-	    log_fail("unable to get Ceph OSD flags!");
-	} else {
-	    if (!($osd_flags =~ m/recovery_deletes/ && $osd_flags =~ m/purged_snapdirs/)) {
-		log_fail("missing 'recovery_deletes' and/or 'purged_snapdirs' flag, scrub of all PGs required before upgrading to Nautilus!");
-	    }
-	}
-    };
-
     # TODO: check OSD min-required version, if to low it breaks stuff!
 
     log_info("getting Ceph daemon versions..");
-- 
2.30.2

[pve-devel] [PATCH manager 2/5] pve6to7: remove PASS noise for ceph

[Fabian Grünbichler]

these were mostly relevant for the Luminous -> Nautilus upgrade, and we
don't need to list all the default passing states that our tooling sets
up anyway.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 PVE/CLI/pve6to7.pm | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/PVE/CLI/pve6to7.pm b/PVE/CLI/pve6to7.pm
index c693e0d4..65ee5a66 100644
--- a/PVE/CLI/pve6to7.pm
+++ b/PVE/CLI/pve6to7.pm
@@ -473,8 +473,6 @@ sub check_ceph {
 	my $global_monhost = $global->{mon_host} // $global->{"mon host"} // $global->{"mon-host"};
 	if (!defined($global_monhost)) {
 	    log_warn("No 'mon_host' entry found in ceph config.\n  It's recommended to add mon_host with all monitor addresses (without ports) to the global section.");
-	} else {
-	    log_pass("Found 'mon_host' entry.");
 	}
 
 	my $ipv6 = $global->{ms_bind_ipv6} // $global->{"ms bind ipv6"} // $global->{"ms-bind-ipv6"};
@@ -482,17 +480,11 @@ sub check_ceph {
 	    my $ipv4 = $global->{ms_bind_ipv4} // $global->{"ms bind ipv4"} // $global->{"ms-bind-ipv4"};
 	    if ($ipv6 eq 'true' && (!defined($ipv4) || $ipv4 ne 'false')) {
 		log_warn("'ms_bind_ipv6' is enabled but 'ms_bind_ipv4' is not disabled.\n  Make sure to disable 'ms_bind_ipv4' for ipv6 only clusters, or add an ipv4 network to public/cluster network.");
-	    } else {
-		log_pass("'ms_bind_ipv6' is enabled and 'ms_bind_ipv4' disabled");
 	    }
-	} else {
-	    log_pass("'ms_bind_ipv6' not enabled");
 	}
 
 	if (defined($global->{keyring})) {
 	    log_warn("[global] config section contains 'keyring' option, which will prevent services from starting with Nautilus.\n Move 'keyring' option to [client] section instead.");
-	} else {
-	    log_pass("no 'keyring' option in [global] section found.");
 	}
 
     } else {
-- 
2.30.2

[pve-devel] [PATCH manager 3/5] pve6to7: check for >= Octopus

[Fabian Grünbichler]

and drop the Nautilus OSD upgrade check while we are at it..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 PVE/CLI/pve6to7.pm | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/PVE/CLI/pve6to7.pm b/PVE/CLI/pve6to7.pm
index 65ee5a66..00f922bb 100644
--- a/PVE/CLI/pve6to7.pm
+++ b/PVE/CLI/pve6to7.pm
@@ -493,12 +493,8 @@ sub check_ceph {
 
     my $local_ceph_ver = PVE::Ceph::Tools::get_local_version(1);
     if (defined($local_ceph_ver)) {
-	if ($local_ceph_ver == 14) {
-	    my $ceph_volume_osds = PVE::Ceph::Tools::ceph_volume_list();
-	    my $scanned_osds = PVE::Tools::dir_glob_regex('/etc/ceph/osd', '^.*\.json$');
-	    if (-e '/var/lib/ceph/osd/' && !defined($scanned_osds) && !(keys %$ceph_volume_osds)) {
-		log_warn("local Ceph version is Nautilus, local OSDs detected, but no conversion from ceph-disk to ceph-volume done (yet).");
-	    }
+	if ($local_ceph_ver <= 14) {
+	    log_fail("local Ceph version too low, at least Octopus required..");
 	}
     } else {
 	log_fail("unable to determine local Ceph version.");
-- 
2.30.2

[pve-devel] [PATCH manager 4/5] pve6to7: dont guard noout check on Ceph version

[Fabian Grünbichler]

we don't have a mandatory Ceph major version upgrade this time around,
so this check does not make sense. instead, we want noout until the full
cluster is upgraded. let's use the simple approach and just flip the
switch to "turn off noout if all of Ceph is a single version" in the PVE
7.x branch.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---

Notes:
    next patch sets it for the stable-6 branch

 PVE/CLI/pve6to7.pm | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/PVE/CLI/pve6to7.pm b/PVE/CLI/pve6to7.pm
index 00f922bb..36e6676f 100644
--- a/PVE/CLI/pve6to7.pm
+++ b/PVE/CLI/pve6to7.pm
@@ -447,9 +447,7 @@ sub check_ceph {
 	    log_warn("unable to determine overall Ceph daemon versions!");
 	} elsif (keys %$overall_versions == 1) {
 	    log_pass("single running overall version detected for all Ceph daemon types.");
-	    if ((keys %$overall_versions)[0] =~ /^ceph version 15\./) {
-		$noout_wanted = 0;
-	    }
+	    $noout_wanted = 0; # off post-upgrade, on pre-upgrade
 	} else {
 	    log_warn("overall version mismatch detected, check 'ceph versions' output for details!");
 	}
-- 
2.30.2

[pve-devel] [PATCH stable-6 manager 5/5] pve6to7: enable noout before upgrade

[Fabian Grünbichler]

even if the cluster-wide Ceph versions are uniform.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 PVE/CLI/pve6to7.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/PVE/CLI/pve6to7.pm b/PVE/CLI/pve6to7.pm
index 36e6676f..db93fa68 100644
--- a/PVE/CLI/pve6to7.pm
+++ b/PVE/CLI/pve6to7.pm
@@ -447,7 +447,7 @@ sub check_ceph {
 	    log_warn("unable to determine overall Ceph daemon versions!");
 	} elsif (keys %$overall_versions == 1) {
 	    log_pass("single running overall version detected for all Ceph daemon types.");
-	    $noout_wanted = 0; # off post-upgrade, on pre-upgrade
+	    $noout_wanted = 1; # off post-upgrade, on pre-upgrade
 	} else {
 	    log_warn("overall version mismatch detected, check 'ceph versions' output for details!");
 	}
-- 
2.30.2

[pve-devel] applied-series: [PATCH manager 0/5] pve6to7 ceph fixes

[Thomas Lamprecht]

On 06.07.21 14:13, Fabian Grünbichler wrote:
> reduce checks, adapt version guards, make the whole thing work with
> pve-manager 7.x
> 
> last patch is stable-6 only, rest is for both branches.
> 
> Fabian Grünbichler (5):
>   pve6to7: use new flags API
>   pve6to7: remove PASS noise for ceph
>   pve6to7: check for >= Octopus
>   pve6to7: dont guard noout check on Ceph version
>   pve6to7: enable noout before upgrade
> 
>  PVE/CLI/pve6to7.pm | 39 +++++++++------------------------------
>  1 file changed, 9 insertions(+), 30 deletions(-)
> 



applied series to master and stable-6 respectively, thanks!