[pve-devel] [PATCH v2 quincy-stable-8 ceph 0/2] Backport Patches for Reef for Quincy

[pve-devel] [PATCH v2 quincy-stable-8 ceph 0/2] Backport Patches for Reef for Quincy

[Max Carrara]

Updated series that includes an unstaged change I had missed to amend
regarding the corrected typo mentioned in the recent patch for master [0].
Woops!

[0]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061499.html

Max Carrara (2):
  patches: include patches regarding RocksDB and dashboard from master
  patch: fix `ceph dashboard` subcommand becoming unavailable on crash

 ...hboard-simplify-authentication-proto.patch | 279 ++++++++++++++++++
 patches/0021-debian-rules-fix-buildtype.patch |  22 --
 ...move-ability-to-create-and-check-TLS.patch | 128 ++++++++
 ...cksb-inherit-parent-cmake-cxx-flags.patch} |   0
 patches/series                                |   5 +-
 5 files changed, 410 insertions(+), 24 deletions(-)
 create mode 100644 patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch
 delete mode 100644 patches/0021-debian-rules-fix-buildtype.patch
 create mode 100644 patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
 rename patches/{0022-rocksb-inherit-parent-cmake-cxx-flags.patch => 0023-rocksb-inherit-parent-cmake-cxx-flags.patch} (100%)

-- 
2.39.2

[pve-devel] [PATCH v2 quincy-stable-8 ceph 1/2] patches: include patches regarding RocksDB and dashboard from master

[Max Carrara]

This commit essentially contains all changes from the following
commits (most recent last):
  * f35168f6713d5f3fa1aaa8c572d754b61c458d91
  * 86a553d66e69176940959530d4fedcbcbbab54d9
  * ab5c03b44d78c4e4b233ff5a310888592dbb9bb4

The series file and the patches' prefixed numbers have been updated
correspondingly.

A very minor adaptation has been made to the patch added by commit
f35168f6713d5f3fa1aaa8c572d754b61c458d91 in order to get it to apply
correctly.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 Changes v1 --> v2:
  * None

 ...hboard-simplify-authentication-proto.patch | 279 ++++++++++++++++++
 patches/0021-debian-rules-fix-buildtype.patch |  22 --
 ...move-ability-to-create-and-check-TLS.patch | 101 +++++++
 ...cksb-inherit-parent-cmake-cxx-flags.patch} |   0
 patches/series                                |   5 +-
 5 files changed, 383 insertions(+), 24 deletions(-)
 create mode 100644 patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch
 delete mode 100644 patches/0021-debian-rules-fix-buildtype.patch
 create mode 100644 patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
 rename patches/{0022-rocksb-inherit-parent-cmake-cxx-flags.patch => 0023-rocksb-inherit-parent-cmake-cxx-flags.patch} (100%)

diff --git a/patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch b/patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch
new file mode 100644
index 000000000..88dbcd04a
--- /dev/null
+++ b/patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch
@@ -0,0 +1,279 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Max Carrara <m.carrara@proxmox.com>
+Date: Tue, 2 Jan 2024 13:02:51 +0000
+Subject: [PATCH] backport: mgr/dashboard: simplify authentication protocol
+
+This is a backport of https://github.com/ceph/ceph/pull/54710 which
+fixes the Ceph Dashboard not being able to launch on Ceph Reef running
+on Debian Bookworm.
+
+This is achieved by removing the dependency on `PyJWT` (Python) and thus
+transitively also removing the dependency on `cryptography` (Python).
+For more information, see the original pull request.
+
+Note that the Ceph Dashboard still cannot be used if TLS is activated,
+because `pyOpenSSL` is used to verify certs during launch. Disabling
+TLS via `ceph config set mgr mgr/dashboard/ssl false` and using e.g.
+a reverse proxy can be used as a workaround.
+
+A separate patch is required to allow the dashboard to run with TLS
+enabled.
+
+Fixes: https://forum.proxmox.com/threads/ceph-warning-post-upgrade-to-v8.129371
+Signed-off-by: Daniel Persson <mailto.woden@gmail.com>
+Signed-off-by: Max Carrara <m.carrara@proxmox.com>
+---
+ ceph.spec.in                                  |  4 --
+ debian/control                                |  1 -
+ src/pybind/mgr/dashboard/constraints.txt      |  1 -
+ src/pybind/mgr/dashboard/exceptions.py        | 12 ++++
+ .../mgr/dashboard/requirements-lint.txt       |  1 +
+ .../mgr/dashboard/requirements-test.txt       |  1 +
+ src/pybind/mgr/dashboard/requirements.txt     |  1 -
+ src/pybind/mgr/dashboard/services/auth.py     | 70 ++++++++++++++++---
+ 8 files changed, 75 insertions(+), 16 deletions(-)
+
+diff --git a/ceph.spec.in b/ceph.spec.in
+index e4cbbef6943..457d7a6dc56 100644
+--- a/ceph.spec.in
++++ b/ceph.spec.in
+@@ -403,7 +403,6 @@ BuildRequires:	xmlsec1-nss
+ BuildRequires:	xmlsec1-openssl
+ BuildRequires:	xmlsec1-openssl-devel
+ BuildRequires:	python%{python3_pkgversion}-cherrypy
+-BuildRequires:	python%{python3_pkgversion}-jwt
+ BuildRequires:	python%{python3_pkgversion}-routes
+ BuildRequires:	python%{python3_pkgversion}-scipy
+ BuildRequires:	python%{python3_pkgversion}-werkzeug
+@@ -416,7 +415,6 @@ BuildRequires:	libxmlsec1-1
+ BuildRequires:	libxmlsec1-nss1
+ BuildRequires:	libxmlsec1-openssl1
+ BuildRequires:	python%{python3_pkgversion}-CherryPy
+-BuildRequires:	python%{python3_pkgversion}-PyJWT
+ BuildRequires:	python%{python3_pkgversion}-Routes
+ BuildRequires:	python%{python3_pkgversion}-Werkzeug
+ BuildRequires:	python%{python3_pkgversion}-numpy-devel
+@@ -608,7 +606,6 @@ Requires:       ceph-prometheus-alerts = %{_epoch_prefix}%{version}-%{release}
+ Requires:       python%{python3_pkgversion}-setuptools
+ %if 0%{?fedora} || 0%{?rhel}
+ Requires:       python%{python3_pkgversion}-cherrypy
+-Requires:       python%{python3_pkgversion}-jwt
+ Requires:       python%{python3_pkgversion}-routes
+ Requires:       python%{python3_pkgversion}-werkzeug
+ %if 0%{?weak_deps}
+@@ -617,7 +614,6 @@ Recommends:     python%{python3_pkgversion}-saml
+ %endif
+ %if 0%{?suse_version}
+ Requires:       python%{python3_pkgversion}-CherryPy
+-Requires:       python%{python3_pkgversion}-PyJWT
+ Requires:       python%{python3_pkgversion}-Routes
+ Requires:       python%{python3_pkgversion}-Werkzeug
+ Recommends:     python%{python3_pkgversion}-python3-saml
+diff --git a/debian/control b/debian/control
+index e3f2073ce8f..ba34a163f54 100644
+--- a/debian/control
++++ b/debian/control
+@@ -87,7 +87,6 @@ Build-Depends: automake,
+                python3-all-dev,
+                python3-cherrypy3,
+                python3-natsort,
+-               python3-jwt <pkg.ceph.check>,
+                python3-pecan <pkg.ceph.check>,
+                python3-bcrypt <pkg.ceph.check>,
+                tox <pkg.ceph.check>,
+diff --git a/src/pybind/mgr/dashboard/constraints.txt b/src/pybind/mgr/dashboard/constraints.txt
+index 55f81c92dec..fd614104880 100644
+--- a/src/pybind/mgr/dashboard/constraints.txt
++++ b/src/pybind/mgr/dashboard/constraints.txt
+@@ -1,6 +1,5 @@
+ CherryPy~=13.1
+ more-itertools~=8.14
+-PyJWT~=2.0
+ bcrypt~=3.1
+ python3-saml~=1.4
+ requests~=2.26
+diff --git a/src/pybind/mgr/dashboard/exceptions.py b/src/pybind/mgr/dashboard/exceptions.py
+index 96cbc523356..d396a38d2c3 100644
+--- a/src/pybind/mgr/dashboard/exceptions.py
++++ b/src/pybind/mgr/dashboard/exceptions.py
+@@ -121,3 +121,15 @@ class GrafanaError(Exception):
+ 
+ class PasswordPolicyException(Exception):
+     pass
++
++
++class ExpiredSignatureError(Exception):
++    pass
++
++
++class InvalidTokenError(Exception):
++    pass
++
++
++class InvalidAlgorithmError(Exception):
++    pass
+diff --git a/src/pybind/mgr/dashboard/requirements-lint.txt b/src/pybind/mgr/dashboard/requirements-lint.txt
+index d82fa1ace1d..5fe9957c32a 100644
+--- a/src/pybind/mgr/dashboard/requirements-lint.txt
++++ b/src/pybind/mgr/dashboard/requirements-lint.txt
+@@ -9,3 +9,4 @@ autopep8==1.5.7
+ pyfakefs==4.5.0
+ isort==5.5.3
+ jsonschema==4.16.0
++PyJWT~=2.0
+diff --git a/src/pybind/mgr/dashboard/requirements-test.txt b/src/pybind/mgr/dashboard/requirements-test.txt
+index 4e925e8616f..78fd1d5b742 100644
+--- a/src/pybind/mgr/dashboard/requirements-test.txt
++++ b/src/pybind/mgr/dashboard/requirements-test.txt
+@@ -2,3 +2,4 @@ pytest-cov
+ pytest-instafail
+ pyfakefs==4.5.0
+ jsonschema==4.16.0
++PyJWT~=2.0
+diff --git a/src/pybind/mgr/dashboard/requirements.txt b/src/pybind/mgr/dashboard/requirements.txt
+index 8003d62a552..292971819c9 100644
+--- a/src/pybind/mgr/dashboard/requirements.txt
++++ b/src/pybind/mgr/dashboard/requirements.txt
+@@ -1,7 +1,6 @@
+ bcrypt
+ CherryPy
+ more-itertools
+-PyJWT
+ pyopenssl
+ requests
+ Routes
+diff --git a/src/pybind/mgr/dashboard/services/auth.py b/src/pybind/mgr/dashboard/services/auth.py
+index f13963abffd..3c600231252 100644
+--- a/src/pybind/mgr/dashboard/services/auth.py
++++ b/src/pybind/mgr/dashboard/services/auth.py
+@@ -1,17 +1,19 @@
+ # -*- coding: utf-8 -*-
+ 
++import base64
++import hashlib
++import hmac
+ import json
+ import logging
+ import os
+ import threading
+ import time
+ import uuid
+-from base64 import b64encode
+ 
+ import cherrypy
+-import jwt
+ 
+ from .. import mgr
++from ..exceptions import ExpiredSignatureError, InvalidAlgorithmError, InvalidTokenError
+ from .access_control import LocalAuthenticator, UserDoesNotExist
+ 
+ cherrypy.config.update({
+@@ -33,7 +35,7 @@ class JwtManager(object):
+     @staticmethod
+     def _gen_secret():
+         secret = os.urandom(16)
+-        return b64encode(secret).decode('utf-8')
++        return base64.b64encode(secret).decode('utf-8')
+ 
+     @classmethod
+     def init(cls):
+@@ -45,6 +47,54 @@ class JwtManager(object):
+             mgr.set_store('jwt_secret', secret)
+         cls._secret = secret
+ 
++    @classmethod
++    def array_to_base64_string(cls, message):
++        jsonstr = json.dumps(message, sort_keys=True).replace(" ", "")
++        string_bytes = base64.urlsafe_b64encode(bytes(jsonstr, 'UTF-8'))
++        return string_bytes.decode('UTF-8').replace("=", "")
++
++    @classmethod
++    def encode(cls, message, secret):
++        header = {"alg": cls.JWT_ALGORITHM, "typ": "JWT"}
++        base64_header = cls.array_to_base64_string(header)
++        base64_message = cls.array_to_base64_string(message)
++        base64_secret = base64.urlsafe_b64encode(hmac.new(
++            bytes(secret, 'UTF-8'),
++            msg=bytes(base64_header + "." + base64_message, 'UTF-8'),
++            digestmod=hashlib.sha256
++        ).digest()).decode('UTF-8').replace("=", "")
++        return base64_header + "." + base64_message + "." + base64_secret
++
++    @classmethod
++    def decode(cls, message, secret):
++        split_message = message.split(".")
++        base64_header = split_message[0]
++        base64_message = split_message[1]
++        base64_secret = split_message[2]
++
++        decoded_header = json.loads(base64.urlsafe_b64decode(base64_header))
++
++        if decoded_header['alg'] != cls.JWT_ALGORITHM:
++            raise InvalidAlgorithmError()
++
++        incoming_secret = base64.urlsafe_b64encode(hmac.new(
++            bytes(secret, 'UTF-8'),
++            msg=bytes(base64_header + "." + base64_message, 'UTF-8'),
++            digestmod=hashlib.sha256
++        ).digest()).decode('UTF-8').replace("=", "")
++
++        if base64_secret != incoming_secret:
++            raise InvalidTokenError()
++
++        # We add ==== as padding to ignore the requirement to have correct padding in
++        # the urlsafe_b64decode method.
++        decoded_message = json.loads(base64.urlsafe_b64decode(base64_message + "===="))
++        now = int(time.time())
++        if decoded_message['exp'] < now:
++            raise ExpiredSignatureError()
++
++        return decoded_message
++
+     @classmethod
+     def gen_token(cls, username):
+         if not cls._secret:
+@@ -59,13 +109,13 @@ class JwtManager(object):
+             'iat': now,
+             'username': username
+         }
+-        return jwt.encode(payload, cls._secret, algorithm=cls.JWT_ALGORITHM)  # type: ignore
++        return cls.encode(payload, cls._secret)  # type: ignore
+ 
+     @classmethod
+     def decode_token(cls, token):
+         if not cls._secret:
+             cls.init()
+-        return jwt.decode(token, cls._secret, algorithms=cls.JWT_ALGORITHM)  # type: ignore
++        return cls.decode(token, cls._secret)  # type: ignore
+ 
+     @classmethod
+     def get_token_from_header(cls):
+@@ -99,8 +149,8 @@ class JwtManager(object):
+     @classmethod
+     def get_user(cls, token):
+         try:
+-            dtoken = JwtManager.decode_token(token)
+-            if not JwtManager.is_blocklisted(dtoken['jti']):
++            dtoken = cls.decode_token(token)
++            if not cls.is_blocklisted(dtoken['jti']):
+                 user = AuthManager.get_user(dtoken['username'])
+                 if user.last_update <= dtoken['iat']:
+                     return user
+@@ -110,10 +160,12 @@ class JwtManager(object):
+                 )
+             else:
+                 cls.logger.debug('Token is block-listed')  # type: ignore
+-        except jwt.ExpiredSignatureError:
++        except ExpiredSignatureError:
+             cls.logger.debug("Token has expired")  # type: ignore
+-        except jwt.InvalidTokenError:
++        except InvalidTokenError:
+             cls.logger.debug("Failed to decode token")  # type: ignore
++        except InvalidAlgorithmError:
++            cls.logger.debug("Only the HS256 algorithm is supported.")  # type: ignore
+         except UserDoesNotExist:
+             cls.logger.debug(  # type: ignore
+                 "Invalid token: user %s does not exist", dtoken['username']
+-- 
+2.39.2
+
diff --git a/patches/0021-debian-rules-fix-buildtype.patch b/patches/0021-debian-rules-fix-buildtype.patch
deleted file mode 100644
index 8b6ef6b56..000000000
--- a/patches/0021-debian-rules-fix-buildtype.patch
+++ /dev/null
@@ -1,22 +0,0 @@
-From 1f4b106d49fc916994d97e273599f75caa904c3b Mon Sep 17 00:00:00 2001
-From: Mark Nelson <mark.nelson@clyso.com>
-Date: Thu, 14 Dec 2023 05:19:46 +0000
-Subject: [PATCH] debian/rules: Fix build_type for massive performance gain
-
-Signed-off-by: Mark Nelson <mark.nelson@clyso.com>
----
- debian/rules | 1 +
- 1 file changed, 1 insertion(+)
-
-diff --git a/debian/rules b/debian/rules
-index ed7f4a255ed4b..b28abb7d62788 100755
---- a/debian/rules
-+++ b/debian/rules
-@@ -29,6 +29,7 @@ extraopts += -DWITH_PYTHON3=3
- extraopts += -DWITH_CEPHFS_JAVA=ON
- extraopts += -DWITH_CEPHFS_SHELL=ON
- extraopts += -DWITH_SYSTEMD=ON -DCEPH_SYSTEMD_ENV_DIR=/etc/default
-+extraopts += -DCMAKE_BUILD_TYPE=RelWithDebInfo
- extraopts += -DWITH_GRAFANA=ON
- ifeq ($(DEB_HOST_ARCH), amd64)
-   extraopts += -DWITH_RBD_RWL=ON
diff --git a/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
new file mode 100644
index 000000000..59c5263da
--- /dev/null
+++ b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
@@ -0,0 +1,101 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Max Carrara <m.carrara@proxmox.com>
+Date: Thu, 4 Jan 2024 17:37:50 +0100
+Subject: [PATCH] mgr/dashboard: remove ability to create and check TLS
+ key/cert pairs
+
+In order to avoid running into PyO3-related issues [0] with PyOpenSSL,
+the ability to create self-signed certs is disabled - the command
+`ceph dashboard create-self-signed-cert` is made to always return an
+error.
+
+The command's error message contains the manual steps the user may
+follow in order to set the certificate themselves, as well as a link
+to the Ceph Dashboard documentation regarding TLS support. [1]
+
+Furthermore, the check on start-up, that verifies that the configured
+key/cert pair actually match, is also removed. This means that users
+need to ensure themselves that the correct pair is supplied -
+otherwise their browser will complain.
+
+These changes allow the dashboard to launch with TLS enabled again.
+
+[0]: https://tracker.ceph.com/issues/63529
+[1]: https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support
+
+Signed-off-by: Max Carrara <m.carrara@proxmox.com>
+---
+ src/pybind/mgr/dashboard/module.py | 41 ++++++++++++++++++++----------
+ 1 file changed, 27 insertions(+), 14 deletions(-)
+
+diff --git a/src/pybind/mgr/dashboard/module.py b/src/pybind/mgr/dashboard/module.py
+index 68725be6e35..9db55a3ee93 100644
+--- a/src/pybind/mgr/dashboard/module.py
++++ b/src/pybind/mgr/dashboard/module.py
+@@ -23,8 +23,7 @@ if TYPE_CHECKING:
+ 
+ from mgr_module import CLIReadCommand, CLIWriteCommand, HandleCommandResult, \
+     MgrModule, MgrStandbyModule, NotifyType, Option, _get_localized_key
+-from mgr_util import ServerConfigException, build_url, \
+-    create_self_signed_cert, get_default_addr, verify_tls_files
++from mgr_util import ServerConfigException, build_url, get_default_addr
+ 
+ from . import mgr
+ from .controllers import Router, json_error_page
+@@ -172,11 +171,14 @@ class CherryPyConfig(object):
+             else:
+                 pkey_fname = self.get_localized_module_option('key_file')  # type: ignore
+ 
+-            verify_tls_files(cert_fname, pkey_fname)
+-
+             # Create custom SSL context to disable TLS 1.0 and 1.1.
+             context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
+-            context.load_cert_chain(cert_fname, pkey_fname)
++
++            try:
++                context.load_cert_chain(cert_fname, pkey_fname)
++            except ssl.SSLError:
++                raise ServerConfigException("No certificate configured")
++
+             if sys.version_info >= (3, 7):
+                 if Settings.UNSAFE_TLS_v1_2:
+                     context.minimum_version = ssl.TLSVersion.TLSv1_2
+@@ -473,15 +475,26 @@ class Module(MgrModule, CherryPyConfig):
+ 
+     @CLIWriteCommand("dashboard create-self-signed-cert")
+     def set_mgr_created_self_signed_cert(self):
+-        cert, pkey = create_self_signed_cert('IT', 'ceph-dashboard')
+-        result = HandleCommandResult(*self.set_ssl_certificate(inbuf=cert))
+-        if result.retval != 0:
+-            return result
+-
+-        result = HandleCommandResult(*self.set_ssl_certificate_key(inbuf=pkey))
+-        if result.retval != 0:
+-            return result
+-        return 0, 'Self-signed certificate created', ''
++        from textwrap import dedent
++
++        err = """
++        Creating self-signed certificates is currently not available.
++        However, you can still set a key and certificate pair manually:
++
++        1. Generate a private key and self-signed certificate:
++          # openssl req -newkey rsa:2048 -nodes -x509 \\
++          -keyout /root/dashboard-key.pem -out /root/dashboard-cert.pem -sha512 \\
++          -days 3650 -subj "/CN=IT/O=ceph-mgr-dashboard" -utf8
++
++        2. Set the corresponding config keys for the key/cert pair:
++          # ceph config-key set mgr/dashboard/key -i /root/dashboard-key.pem
++          # ceph config-key set mgr/dashboard/crt -i /root/dashboard-crt.pem
++
++        For more information on how to configure TLS for the dashboard, visit:
++        https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support
++        """
++
++        return -errno.ENOTSUP, '', dedent(err).strip()
+ 
+     @CLIWriteCommand("dashboard set-rgw-credentials")
+     def set_rgw_credentials(self):
+-- 
+2.39.2
+
diff --git a/patches/0022-rocksb-inherit-parent-cmake-cxx-flags.patch b/patches/0023-rocksb-inherit-parent-cmake-cxx-flags.patch
similarity index 100%
rename from patches/0022-rocksb-inherit-parent-cmake-cxx-flags.patch
rename to patches/0023-rocksb-inherit-parent-cmake-cxx-flags.patch
diff --git a/patches/series b/patches/series
index 73f66396c..ee897a78a 100644
--- a/patches/series
+++ b/patches/series
@@ -13,5 +13,6 @@
 0016-d-rules-fix-no-restart-on-upgrade.patch
 0017-python3.10-pep-620.patch
 0020-fix-4759-run-ceph-crash-daemon-with-www-data-group-f.patch
-0021-debian-rules-fix-buildtype.patch
-0022-rocksb-inherit-parent-cmake-cxx-flags.patch
+0021-backport-mgr-dashboard-simplify-authentication-proto.patch
+0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
+0023-rocksb-inherit-parent-cmake-cxx-flags.patch
-- 
2.39.2

[pve-devel] [PATCH v2 quincy-stable-8 ceph 2/2] patch: fix `ceph dashboard` subcommand becoming unavailable on crash

[Max Carrara]

Adapt the patch that originally disabled certain TLS checks during the
dashboard's startup and fixes the `ceph dashboard` subcommand becoming
unavailable if the dashboard crashes during that time.

This is achieved by re-implementing certain checks and also re-raising
any other unforeseen exceptions that occur in regards to TLS as one of
Ceph's internal exception types, which are then handled by the
dashboard itself. This is akin to how these cases were handled
originally.

Also fixes a typo in the `ceph dashboard create-self-signed-cert`
command output.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 Changes v1 --> v2:
  * Correct typo in `ceph dashboard create-self-signed-cert` command

 ...move-ability-to-create-and-check-TLS.patch | 55 ++++++++++++++-----
 1 file changed, 41 insertions(+), 14 deletions(-)

diff --git a/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
index 59c5263da..9cc2889da 100644
--- a/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
+++ b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
@@ -1,6 +1,6 @@
 From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
 From: Max Carrara <m.carrara@proxmox.com>
-Date: Thu, 4 Jan 2024 17:37:50 +0100
+Date: Fri, 26 Jan 2024 14:04:47 +0100
 Subject: [PATCH] mgr/dashboard: remove ability to create and check TLS
  key/cert pairs
 
@@ -18,6 +18,17 @@ key/cert pair actually match, is also removed. This means that users
 need to ensure themselves that the correct pair is supplied -
 otherwise their browser will complain.
 
+Other checks unrelated to the verification of keypairs are preserved,
+such as checking for the cert's and key's existence on the filesystem.
+
+`ssl.SSLError`s that occur during startup are re-raised with the
+additional information they contain as `ServerConfigException`s, as
+the dashboard handles these in its startup loop. Other exceptions are
+re-raised as well. Otherwise, the dashboard will irrecoverably crash,
+which also causes the `ceph dashboard` subcommand to stop working
+altogether, even if one of its sub-subcommands are unrelated to the
+dashboard itself.
+
 These changes allow the dashboard to launch with TLS enabled again.
 
 [0]: https://tracker.ceph.com/issues/63529
@@ -25,11 +36,11 @@ These changes allow the dashboard to launch with TLS enabled again.
 
 Signed-off-by: Max Carrara <m.carrara@proxmox.com>
 ---
- src/pybind/mgr/dashboard/module.py | 41 ++++++++++++++++++++----------
- 1 file changed, 27 insertions(+), 14 deletions(-)
+ src/pybind/mgr/dashboard/module.py | 58 ++++++++++++++++++++++--------
+ 1 file changed, 43 insertions(+), 15 deletions(-)
 
 diff --git a/src/pybind/mgr/dashboard/module.py b/src/pybind/mgr/dashboard/module.py
-index 68725be6e35..9db55a3ee93 100644
+index 68725be6e35..c8b263d9786 100644
 --- a/src/pybind/mgr/dashboard/module.py
 +++ b/src/pybind/mgr/dashboard/module.py
 @@ -23,8 +23,7 @@ if TYPE_CHECKING:
@@ -42,25 +53,41 @@ index 68725be6e35..9db55a3ee93 100644
  
  from . import mgr
  from .controllers import Router, json_error_page
-@@ -172,11 +171,14 @@ class CherryPyConfig(object):
+@@ -172,11 +171,29 @@ class CherryPyConfig(object):
              else:
                  pkey_fname = self.get_localized_module_option('key_file')  # type: ignore
  
 -            verify_tls_files(cert_fname, pkey_fname)
--
-             # Create custom SSL context to disable TLS 1.0 and 1.1.
-             context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
--            context.load_cert_chain(cert_fname, pkey_fname)
++            if not cert_fname or not pkey_fname:
++                raise ServerConfigException('no certificate configured')
++
++            if not os.path.isfile(cert_fname):
++                raise ServerConfigException(f"Certificate {cert_fname} does not exist")
++
++            if not os.path.isfile(pkey_fname):
++                raise ServerConfigException(f"private key {pkey_fname} does not exist")
 +
 +            try:
++                # Create custom SSL context to disable TLS 1.0 and 1.1.
++                context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
 +                context.load_cert_chain(cert_fname, pkey_fname)
-+            except ssl.SSLError:
-+                raise ServerConfigException("No certificate configured")
-+
++            except ssl.SSLError as e:
++                raise ServerConfigException(
++                    "Encountered unexpected error while creating SSL context"
++                    f" - library: {e.library}, reason: {e.reason}"
++                )
++            except Exception as e:
++                raise ServerConfigException(
++                    f"Encountered unexpected error while creating SSL context: {e}"
++                )
+ 
+-            # Create custom SSL context to disable TLS 1.0 and 1.1.
+-            context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
+-            context.load_cert_chain(cert_fname, pkey_fname)
              if sys.version_info >= (3, 7):
                  if Settings.UNSAFE_TLS_v1_2:
                      context.minimum_version = ssl.TLSVersion.TLSv1_2
-@@ -473,15 +475,26 @@ class Module(MgrModule, CherryPyConfig):
+@@ -473,15 +490,26 @@ class Module(MgrModule, CherryPyConfig):
  
      @CLIWriteCommand("dashboard create-self-signed-cert")
      def set_mgr_created_self_signed_cert(self):
@@ -81,7 +108,7 @@ index 68725be6e35..9db55a3ee93 100644
 +
 +        1. Generate a private key and self-signed certificate:
 +          # openssl req -newkey rsa:2048 -nodes -x509 \\
-+          -keyout /root/dashboard-key.pem -out /root/dashboard-cert.pem -sha512 \\
++          -keyout /root/dashboard-key.pem -out /root/dashboard-crt.pem -sha512 \\
 +          -days 3650 -subj "/CN=IT/O=ceph-mgr-dashboard" -utf8
 +
 +        2. Set the corresponding config keys for the key/cert pair:
-- 
2.39.2

[pve-devel] applied-series: [PATCH v2 quincy-stable-8 ceph 0/2] Backport Patches for Reef for Quincy

[Thomas Lamprecht]

Am 26/01/2024 um 16:44 schrieb Max Carrara:
> Updated series that includes an unstaged change I had missed to amend
> regarding the corrected typo mentioned in the recent patch for master [0].
> Woops!
> 
> [0]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061499.html
> 
> Max Carrara (2):
>   patches: include patches regarding RocksDB and dashboard from master
>   patch: fix `ceph dashboard` subcommand becoming unavailable on crash
> 
>  ...hboard-simplify-authentication-proto.patch | 279 ++++++++++++++++++
>  patches/0021-debian-rules-fix-buildtype.patch |  22 --
>  ...move-ability-to-create-and-check-TLS.patch | 128 ++++++++
>  ...cksb-inherit-parent-cmake-cxx-flags.patch} |   0
>  patches/series                                |   5 +-
>  5 files changed, 410 insertions(+), 24 deletions(-)
>  create mode 100644 patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch
>  delete mode 100644 patches/0021-debian-rules-fix-buildtype.patch
>  create mode 100644 patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
>  rename patches/{0022-rocksb-inherit-parent-cmake-cxx-flags.patch => 0023-rocksb-inherit-parent-cmake-cxx-flags.patch} (100%)
> 


applied, thanks!

Re: [pve-devel] [PATCH v2 quincy-stable-8 ceph 1/2] patches: include patches regarding RocksDB and dashboard from master

[Thomas Lamprecht]

Am 26/01/2024 um 16:44 schrieb Max Carrara:
> This commit essentially contains all changes from the following
> commits (most recent last):
>   * f35168f6713d5f3fa1aaa8c572d754b61c458d91
>   * 86a553d66e69176940959530d4fedcbcbbab54d9
>   * ab5c03b44d78c4e4b233ff5a310888592dbb9bb4

FWIW, I got a git alias named show-short that resolves to:

show --no-patch --pretty='format: %C(auto)%h ("%s")'

Which I use for such stuff, as with the subject inlcuded ones gets a bit
more telling commit references.

> 
> The series file and the patches' prefixed numbers have been updated
> correspondingly.
> 
> A very minor adaptation has been made to the patch added by commit
> f35168f6713d5f3fa1aaa8c572d754b61c458d91 in order to get it to apply
> correctly.
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  Changes v1 --> v2:
>   * None
> 

Re: [pve-devel] [PATCH v2 quincy-stable-8 ceph 1/2] patches: include patches regarding RocksDB and dashboard from master

[Thomas Lamprecht]

Am 26/01/2024 um 16:44 schrieb Max Carrara:
> This commit essentially contains all changes from the following
> commits (most recent last):
>   * f35168f6713d5f3fa1aaa8c572d754b61c458d91
>   * 86a553d66e69176940959530d4fedcbcbbab54d9
>   * ab5c03b44d78c4e4b233ff5a310888592dbb9bb4
> 
> The series file and the patches' prefixed numbers have been updated
> correspondingly.
> 
> A very minor adaptation has been made to the patch added by commit
> f35168f6713d5f3fa1aaa8c572d754b61c458d91 in order to get it to apply
> correctly.
> 
> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> ---
>  Changes v1 --> v2:
>   * None
> 

>  delete mode 100644 patches/0021-debian-rules-fix-buildtype.patch

this deletion is not mentioned in the commit message though.

While it actually is fine, as it was a extra patch for the rocksdb build that
is already fixed with rocksb-inherit-parent-cmake-cxx-flags.patch one, it's
still odd to just delete it here without any mentioning..

> diff --git a/patches/0021-debian-rules-fix-buildtype.patch b/patches/0021-debian-rules-fix-buildtype.patch
> deleted file mode 100644
> index 8b6ef6b56..000000000
> --- a/patches/0021-debian-rules-fix-buildtype.patch
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -From 1f4b106d49fc916994d97e273599f75caa904c3b Mon Sep 17 00:00:00 2001
> -From: Mark Nelson <mark.nelson@clyso.com>
> -Date: Thu, 14 Dec 2023 05:19:46 +0000
> -Subject: [PATCH] debian/rules: Fix build_type for massive performance gain
> -
> -Signed-off-by: Mark Nelson <mark.nelson@clyso.com>
> ----
> - debian/rules | 1 +
> - 1 file changed, 1 insertion(+)
> -
> -diff --git a/debian/rules b/debian/rules
> -index ed7f4a255ed4b..b28abb7d62788 100755
> ---- a/debian/rules
> -+++ b/debian/rules
> -@@ -29,6 +29,7 @@ extraopts += -DWITH_PYTHON3=3
> - extraopts += -DWITH_CEPHFS_JAVA=ON
> - extraopts += -DWITH_CEPHFS_SHELL=ON
> - extraopts += -DWITH_SYSTEMD=ON -DCEPH_SYSTEMD_ENV_DIR=/etc/default
> -+extraopts += -DCMAKE_BUILD_TYPE=RelWithDebInfo
> - extraopts += -DWITH_GRAFANA=ON
> - ifeq ($(DEB_HOST_ARCH), amd64)
> -   extraopts += -DWITH_RBD_RWL=ON
> diff --git a/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
> new file mode 100644
> index 000000000..59c5263da
> --- /dev/null
> +++ b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
> @@ -0,0 +1,101 @@
> +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
> +From: Max Carrara <m.carrara@proxmox.com>
> +Date: Thu, 4 Jan 2024 17:37:50 +0100
> +Subject: [PATCH] mgr/dashboard: remove ability to create and check TLS
> + key/cert pairs
> +
> +In order to avoid running into PyO3-related issues [0] with PyOpenSSL,
> +the ability to create self-signed certs is disabled - the command
> +`ceph dashboard create-self-signed-cert` is made to always return an
> +error.
> +
> +The command's error message contains the manual steps the user may
> +follow in order to set the certificate themselves, as well as a link
> +to the Ceph Dashboard documentation regarding TLS support. [1]
> +
> +Furthermore, the check on start-up, that verifies that the configured
> +key/cert pair actually match, is also removed. This means that users
> +need to ensure themselves that the correct pair is supplied -
> +otherwise their browser will complain.
> +
> +These changes allow the dashboard to launch with TLS enabled again.
> +
> +[0]: https://tracker.ceph.com/issues/63529
> +[1]: https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support
> +
> +Signed-off-by: Max Carrara <m.carrara@proxmox.com>
> +---
> + src/pybind/mgr/dashboard/module.py | 41 ++++++++++++++++++++----------
> + 1 file changed, 27 insertions(+), 14 deletions(-)
> +

> diff --git a/patches/0022-rocksb-inherit-parent-cmake-cxx-flags.patch b/patches/0023-rocksb-inherit-parent-cmake-cxx-flags.patch
> similarity index 100%
> rename from patches/0022-rocksb-inherit-parent-cmake-cxx-flags.patch
> rename to patches/0023-rocksb-inherit-parent-cmake-cxx-flags.patch
> diff --git a/patches/series b/patches/series
> index 73f66396c..ee897a78a 100644
> --- a/patches/series
> +++ b/patches/series
> @@ -13,5 +13,6 @@
>  0016-d-rules-fix-no-restart-on-upgrade.patch
>  0017-python3.10-pep-620.patch
>  0020-fix-4759-run-ceph-crash-daemon-with-www-data-group-f.patch
> -0021-debian-rules-fix-buildtype.patch

Re: [pve-devel] [PATCH v2 quincy-stable-8 ceph 1/2] patches: include patches regarding RocksDB and dashboard from master

[Max Carrara]

On 2/15/24 14:09, Thomas Lamprecht wrote:
> Am 26/01/2024 um 16:44 schrieb Max Carrara:
>> This commit essentially contains all changes from the following
>> commits (most recent last):
>>   * f35168f6713d5f3fa1aaa8c572d754b61c458d91
>>   * 86a553d66e69176940959530d4fedcbcbbab54d9
>>   * ab5c03b44d78c4e4b233ff5a310888592dbb9bb4
>>
>> The series file and the patches' prefixed numbers have been updated
>> correspondingly.
>>
>> A very minor adaptation has been made to the patch added by commit
>> f35168f6713d5f3fa1aaa8c572d754b61c458d91 in order to get it to apply
>> correctly.
>>
>> Signed-off-by: Max Carrara <m.carrara@proxmox.com>
>> ---
>>  Changes v1 --> v2:
>>   * None
>>
> 
>>  delete mode 100644 patches/0021-debian-rules-fix-buildtype.patch
> 
> this deletion is not mentioned in the commit message though.
> 
> While it actually is fine, as it was a extra patch for the rocksdb build that
> is already fixed with rocksb-inherit-parent-cmake-cxx-flags.patch one, it's
> still odd to just delete it here without any mentioning..

Mea culpa; should've mentioned this! I essentially just applied the changes listed
above, which includes the deletion of that patch. Will include a more thorough
summary next time.

> 
>> diff --git a/patches/0021-debian-rules-fix-buildtype.patch b/patches/0021-debian-rules-fix-buildtype.patch
>> deleted file mode 100644
>> index 8b6ef6b56..000000000
>> --- a/patches/0021-debian-rules-fix-buildtype.patch
>> +++ /dev/null
>> @@ -1,22 +0,0 @@
>> -From 1f4b106d49fc916994d97e273599f75caa904c3b Mon Sep 17 00:00:00 2001
>> -From: Mark Nelson <mark.nelson@clyso.com>
>> -Date: Thu, 14 Dec 2023 05:19:46 +0000
>> -Subject: [PATCH] debian/rules: Fix build_type for massive performance gain
>> -
>> -Signed-off-by: Mark Nelson <mark.nelson@clyso.com>
>> ----
>> - debian/rules | 1 +
>> - 1 file changed, 1 insertion(+)
>> -
>> -diff --git a/debian/rules b/debian/rules
>> -index ed7f4a255ed4b..b28abb7d62788 100755
>> ---- a/debian/rules
>> -+++ b/debian/rules
>> -@@ -29,6 +29,7 @@ extraopts += -DWITH_PYTHON3=3
>> - extraopts += -DWITH_CEPHFS_JAVA=ON
>> - extraopts += -DWITH_CEPHFS_SHELL=ON
>> - extraopts += -DWITH_SYSTEMD=ON -DCEPH_SYSTEMD_ENV_DIR=/etc/default
>> -+extraopts += -DCMAKE_BUILD_TYPE=RelWithDebInfo
>> - extraopts += -DWITH_GRAFANA=ON
>> - ifeq ($(DEB_HOST_ARCH), amd64)
>> -   extraopts += -DWITH_RBD_RWL=ON
>> diff --git a/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
>> new file mode 100644
>> index 000000000..59c5263da
>> --- /dev/null
>> +++ b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
>> @@ -0,0 +1,101 @@
>> +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
>> +From: Max Carrara <m.carrara@proxmox.com>
>> +Date: Thu, 4 Jan 2024 17:37:50 +0100
>> +Subject: [PATCH] mgr/dashboard: remove ability to create and check TLS
>> + key/cert pairs
>> +
>> +In order to avoid running into PyO3-related issues [0] with PyOpenSSL,
>> +the ability to create self-signed certs is disabled - the command
>> +`ceph dashboard create-self-signed-cert` is made to always return an
>> +error.
>> +
>> +The command's error message contains the manual steps the user may
>> +follow in order to set the certificate themselves, as well as a link
>> +to the Ceph Dashboard documentation regarding TLS support. [1]
>> +
>> +Furthermore, the check on start-up, that verifies that the configured
>> +key/cert pair actually match, is also removed. This means that users
>> +need to ensure themselves that the correct pair is supplied -
>> +otherwise their browser will complain.
>> +
>> +These changes allow the dashboard to launch with TLS enabled again.
>> +
>> +[0]: https://tracker.ceph.com/issues/63529
>> +[1]: https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support
>> +
>> +Signed-off-by: Max Carrara <m.carrara@proxmox.com>
>> +---
>> + src/pybind/mgr/dashboard/module.py | 41 ++++++++++++++++++++----------
>> + 1 file changed, 27 insertions(+), 14 deletions(-)
>> +
> 
>> diff --git a/patches/0022-rocksb-inherit-parent-cmake-cxx-flags.patch b/patches/0023-rocksb-inherit-parent-cmake-cxx-flags.patch
>> similarity index 100%
>> rename from patches/0022-rocksb-inherit-parent-cmake-cxx-flags.patch
>> rename to patches/0023-rocksb-inherit-parent-cmake-cxx-flags.patch
>> diff --git a/patches/series b/patches/series
>> index 73f66396c..ee897a78a 100644
>> --- a/patches/series
>> +++ b/patches/series
>> @@ -13,5 +13,6 @@
>>  0016-d-rules-fix-no-restart-on-upgrade.patch
>>  0017-python3.10-pep-620.patch
>>  0020-fix-4759-run-ceph-crash-daemon-with-www-data-group-f.patch
>> -0021-debian-rules-fix-buildtype.patch