[pve-devel] [PATCH v3 ceph 0/2] Complete Workaround For Ceph Dashboard

[pve-devel] [PATCH v3 ceph 0/2] Complete Workaround For Ceph Dashboard

[Max Carrara]

These two patches allow the Ceph Dashboard to work again as intended.

The first patch provides a backport of a pull request [0] which removes
the dependency on PyJWT. [1]

The second patch removes two usages of functions that use PyOpenSSL, [2]
which prevents running into sub-interpreter-related issues with
PyO3. [3] This is achieved by removing a check during the dashboard's
launch as well as prohibiting users from creating self-signed TLS certs
via the `ceph` CLI. Users may still provide a key/cert pair manually
and are instructed to do so once they try to execute the respective
command. [4]


[0]: https://github.com/ceph/ceph/pull/54710
[1]: https://github.com/jpadilla/pyjwt
[2]: https://github.com/pyca/pyopenssl
[3]: https://tracker.ceph.com/issues/63529
[4]: `ceph dashboard create-self-signed-cert`

Max Carrara (2):
  mgr/dashboard: add backport that allows the dashboard to work again
  mgr/dashboard: add patch that removes PyOpenSSL-related usages

 ...hboard-simplify-authentication-proto.patch | 279 ++++++++++++++++++
 ...move-ability-to-create-and-check-TLS.patch | 101 +++++++
 patches/series                                |   2 +
 3 files changed, 382 insertions(+)
 create mode 100644 patches/0012-backport-mgr-dashboard-simplify-authentication-proto.patch
 create mode 100644 patches/0013-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch

-- 
2.39.2

[pve-devel] [PATCH v3 ceph 1/2] mgr/dashboard: add backport that allows the dashboard to work again

[Max Carrara]

After upgrading from PVE 7 to PVE 8, some users noted that the Ceph
Dashboard does not work anymore. [0] A user from our community
provided a pull request [1] which removes a dependency to `PyJWT`
(Python). This commit adds a backport of this PR as a single patch.

This patch by itself however does not yet allow the dashboard to run
with TLS enabled.

[0]: https://forum.proxmox.com/threads/ceph-warning-post-upgrade-to-v8.129371/
[1]: https://github.com/ceph/ceph/pull/54710

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 ...hboard-simplify-authentication-proto.patch | 279 ++++++++++++++++++
 patches/series                                |   1 +
 2 files changed, 280 insertions(+)
 create mode 100644 patches/0012-backport-mgr-dashboard-simplify-authentication-proto.patch

diff --git a/patches/0012-backport-mgr-dashboard-simplify-authentication-proto.patch b/patches/0012-backport-mgr-dashboard-simplify-authentication-proto.patch
new file mode 100644
index 000000000..bed0e20ba
--- /dev/null
+++ b/patches/0012-backport-mgr-dashboard-simplify-authentication-proto.patch
@@ -0,0 +1,279 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Max Carrara <m.carrara@proxmox.com>
+Date: Tue, 2 Jan 2024 13:02:51 +0000
+Subject: [PATCH] backport: mgr/dashboard: simplify authentication protocol
+
+This is a backport of https://github.com/ceph/ceph/pull/54710 which
+fixes the Ceph Dashboard not being able to launch on Ceph Reef running
+on Debian Bookworm.
+
+This is achieved by removing the dependency on `PyJWT` (Python) and thus
+transitively also removing the dependency on `cryptography` (Python).
+For more information, see the original pull request.
+
+Note that the Ceph Dashboard still cannot be used if TLS is activated,
+because `pyOpenSSL` is used to verify certs during launch. Disabling
+TLS via `ceph config set mgr mgr/dashboard/ssl false` and using e.g.
+a reverse proxy can be used as a workaround.
+
+A separate patch is required to allow the dashboard to run with TLS
+enabled.
+
+Fixes: https://forum.proxmox.com/threads/ceph-warning-post-upgrade-to-v8.129371
+Signed-off-by: Daniel Persson <mailto.woden@gmail.com>
+Signed-off-by: Max Carrara <m.carrara@proxmox.com>
+---
+ ceph.spec.in                                  |  4 --
+ debian/control                                |  1 -
+ src/pybind/mgr/dashboard/constraints.txt      |  1 -
+ src/pybind/mgr/dashboard/exceptions.py        | 12 ++++
+ .../mgr/dashboard/requirements-lint.txt       |  1 +
+ .../mgr/dashboard/requirements-test.txt       |  1 +
+ src/pybind/mgr/dashboard/requirements.txt     |  1 -
+ src/pybind/mgr/dashboard/services/auth.py     | 70 ++++++++++++++++---
+ 8 files changed, 75 insertions(+), 16 deletions(-)
+
+diff --git a/ceph.spec.in b/ceph.spec.in
+index f0dd8e8a941..6fb61aed8d2 100644
+--- a/ceph.spec.in
++++ b/ceph.spec.in
+@@ -412,7 +412,6 @@ BuildRequires:	xmlsec1-nss
+ BuildRequires:	xmlsec1-openssl
+ BuildRequires:	xmlsec1-openssl-devel
+ BuildRequires:	python%{python3_pkgversion}-cherrypy
+-BuildRequires:	python%{python3_pkgversion}-jwt
+ BuildRequires:	python%{python3_pkgversion}-routes
+ BuildRequires:	python%{python3_pkgversion}-scipy
+ BuildRequires:	python%{python3_pkgversion}-werkzeug
+@@ -425,7 +424,6 @@ BuildRequires:	libxmlsec1-1
+ BuildRequires:	libxmlsec1-nss1
+ BuildRequires:	libxmlsec1-openssl1
+ BuildRequires:	python%{python3_pkgversion}-CherryPy
+-BuildRequires:	python%{python3_pkgversion}-PyJWT
+ BuildRequires:	python%{python3_pkgversion}-Routes
+ BuildRequires:	python%{python3_pkgversion}-Werkzeug
+ BuildRequires:	python%{python3_pkgversion}-numpy-devel
+@@ -617,7 +615,6 @@ Requires:       ceph-prometheus-alerts = %{_epoch_prefix}%{version}-%{release}
+ Requires:       python%{python3_pkgversion}-setuptools
+ %if 0%{?fedora} || 0%{?rhel}
+ Requires:       python%{python3_pkgversion}-cherrypy
+-Requires:       python%{python3_pkgversion}-jwt
+ Requires:       python%{python3_pkgversion}-routes
+ Requires:       python%{python3_pkgversion}-werkzeug
+ %if 0%{?weak_deps}
+@@ -626,7 +623,6 @@ Recommends:     python%{python3_pkgversion}-saml
+ %endif
+ %if 0%{?suse_version}
+ Requires:       python%{python3_pkgversion}-CherryPy
+-Requires:       python%{python3_pkgversion}-PyJWT
+ Requires:       python%{python3_pkgversion}-Routes
+ Requires:       python%{python3_pkgversion}-Werkzeug
+ Recommends:     python%{python3_pkgversion}-python3-saml
+diff --git a/debian/control b/debian/control
+index 32e7bb45ce4..289b28877a8 100644
+--- a/debian/control
++++ b/debian/control
+@@ -91,7 +91,6 @@ Build-Depends: automake,
+                python3-all-dev,
+                python3-cherrypy3,
+                python3-natsort,
+-               python3-jwt <pkg.ceph.check>,
+                python3-pecan <pkg.ceph.check>,
+                python3-bcrypt <pkg.ceph.check>,
+                tox <pkg.ceph.check>,
+diff --git a/src/pybind/mgr/dashboard/constraints.txt b/src/pybind/mgr/dashboard/constraints.txt
+index 55f81c92dec..fd614104880 100644
+--- a/src/pybind/mgr/dashboard/constraints.txt
++++ b/src/pybind/mgr/dashboard/constraints.txt
+@@ -1,6 +1,5 @@
+ CherryPy~=13.1
+ more-itertools~=8.14
+-PyJWT~=2.0
+ bcrypt~=3.1
+ python3-saml~=1.4
+ requests~=2.26
+diff --git a/src/pybind/mgr/dashboard/exceptions.py b/src/pybind/mgr/dashboard/exceptions.py
+index 96cbc523356..d396a38d2c3 100644
+--- a/src/pybind/mgr/dashboard/exceptions.py
++++ b/src/pybind/mgr/dashboard/exceptions.py
+@@ -121,3 +121,15 @@ class GrafanaError(Exception):
+ 
+ class PasswordPolicyException(Exception):
+     pass
++
++
++class ExpiredSignatureError(Exception):
++    pass
++
++
++class InvalidTokenError(Exception):
++    pass
++
++
++class InvalidAlgorithmError(Exception):
++    pass
+diff --git a/src/pybind/mgr/dashboard/requirements-lint.txt b/src/pybind/mgr/dashboard/requirements-lint.txt
+index d82fa1ace1d..5fe9957c32a 100644
+--- a/src/pybind/mgr/dashboard/requirements-lint.txt
++++ b/src/pybind/mgr/dashboard/requirements-lint.txt
+@@ -9,3 +9,4 @@ autopep8==1.5.7
+ pyfakefs==4.5.0
+ isort==5.5.3
+ jsonschema==4.16.0
++PyJWT~=2.0
+diff --git a/src/pybind/mgr/dashboard/requirements-test.txt b/src/pybind/mgr/dashboard/requirements-test.txt
+index d2566bab59f..5066c7a59b6 100644
+--- a/src/pybind/mgr/dashboard/requirements-test.txt
++++ b/src/pybind/mgr/dashboard/requirements-test.txt
+@@ -2,3 +2,4 @@ pytest-cov
+ pytest-instafail
+ pyfakefs==4.5.0
+ jsonschema
++PyJWT~=2.0
+diff --git a/src/pybind/mgr/dashboard/requirements.txt b/src/pybind/mgr/dashboard/requirements.txt
+index 8003d62a552..292971819c9 100644
+--- a/src/pybind/mgr/dashboard/requirements.txt
++++ b/src/pybind/mgr/dashboard/requirements.txt
+@@ -1,7 +1,6 @@
+ bcrypt
+ CherryPy
+ more-itertools
+-PyJWT
+ pyopenssl
+ requests
+ Routes
+diff --git a/src/pybind/mgr/dashboard/services/auth.py b/src/pybind/mgr/dashboard/services/auth.py
+index f13963abffd..3c600231252 100644
+--- a/src/pybind/mgr/dashboard/services/auth.py
++++ b/src/pybind/mgr/dashboard/services/auth.py
+@@ -1,17 +1,19 @@
+ # -*- coding: utf-8 -*-
+ 
++import base64
++import hashlib
++import hmac
+ import json
+ import logging
+ import os
+ import threading
+ import time
+ import uuid
+-from base64 import b64encode
+ 
+ import cherrypy
+-import jwt
+ 
+ from .. import mgr
++from ..exceptions import ExpiredSignatureError, InvalidAlgorithmError, InvalidTokenError
+ from .access_control import LocalAuthenticator, UserDoesNotExist
+ 
+ cherrypy.config.update({
+@@ -33,7 +35,7 @@ class JwtManager(object):
+     @staticmethod
+     def _gen_secret():
+         secret = os.urandom(16)
+-        return b64encode(secret).decode('utf-8')
++        return base64.b64encode(secret).decode('utf-8')
+ 
+     @classmethod
+     def init(cls):
+@@ -45,6 +47,54 @@ class JwtManager(object):
+             mgr.set_store('jwt_secret', secret)
+         cls._secret = secret
+ 
++    @classmethod
++    def array_to_base64_string(cls, message):
++        jsonstr = json.dumps(message, sort_keys=True).replace(" ", "")
++        string_bytes = base64.urlsafe_b64encode(bytes(jsonstr, 'UTF-8'))
++        return string_bytes.decode('UTF-8').replace("=", "")
++
++    @classmethod
++    def encode(cls, message, secret):
++        header = {"alg": cls.JWT_ALGORITHM, "typ": "JWT"}
++        base64_header = cls.array_to_base64_string(header)
++        base64_message = cls.array_to_base64_string(message)
++        base64_secret = base64.urlsafe_b64encode(hmac.new(
++            bytes(secret, 'UTF-8'),
++            msg=bytes(base64_header + "." + base64_message, 'UTF-8'),
++            digestmod=hashlib.sha256
++        ).digest()).decode('UTF-8').replace("=", "")
++        return base64_header + "." + base64_message + "." + base64_secret
++
++    @classmethod
++    def decode(cls, message, secret):
++        split_message = message.split(".")
++        base64_header = split_message[0]
++        base64_message = split_message[1]
++        base64_secret = split_message[2]
++
++        decoded_header = json.loads(base64.urlsafe_b64decode(base64_header))
++
++        if decoded_header['alg'] != cls.JWT_ALGORITHM:
++            raise InvalidAlgorithmError()
++
++        incoming_secret = base64.urlsafe_b64encode(hmac.new(
++            bytes(secret, 'UTF-8'),
++            msg=bytes(base64_header + "." + base64_message, 'UTF-8'),
++            digestmod=hashlib.sha256
++        ).digest()).decode('UTF-8').replace("=", "")
++
++        if base64_secret != incoming_secret:
++            raise InvalidTokenError()
++
++        # We add ==== as padding to ignore the requirement to have correct padding in
++        # the urlsafe_b64decode method.
++        decoded_message = json.loads(base64.urlsafe_b64decode(base64_message + "===="))
++        now = int(time.time())
++        if decoded_message['exp'] < now:
++            raise ExpiredSignatureError()
++
++        return decoded_message
++
+     @classmethod
+     def gen_token(cls, username):
+         if not cls._secret:
+@@ -59,13 +109,13 @@ class JwtManager(object):
+             'iat': now,
+             'username': username
+         }
+-        return jwt.encode(payload, cls._secret, algorithm=cls.JWT_ALGORITHM)  # type: ignore
++        return cls.encode(payload, cls._secret)  # type: ignore
+ 
+     @classmethod
+     def decode_token(cls, token):
+         if not cls._secret:
+             cls.init()
+-        return jwt.decode(token, cls._secret, algorithms=cls.JWT_ALGORITHM)  # type: ignore
++        return cls.decode(token, cls._secret)  # type: ignore
+ 
+     @classmethod
+     def get_token_from_header(cls):
+@@ -99,8 +149,8 @@ class JwtManager(object):
+     @classmethod
+     def get_user(cls, token):
+         try:
+-            dtoken = JwtManager.decode_token(token)
+-            if not JwtManager.is_blocklisted(dtoken['jti']):
++            dtoken = cls.decode_token(token)
++            if not cls.is_blocklisted(dtoken['jti']):
+                 user = AuthManager.get_user(dtoken['username'])
+                 if user.last_update <= dtoken['iat']:
+                     return user
+@@ -110,10 +160,12 @@ class JwtManager(object):
+                 )
+             else:
+                 cls.logger.debug('Token is block-listed')  # type: ignore
+-        except jwt.ExpiredSignatureError:
++        except ExpiredSignatureError:
+             cls.logger.debug("Token has expired")  # type: ignore
+-        except jwt.InvalidTokenError:
++        except InvalidTokenError:
+             cls.logger.debug("Failed to decode token")  # type: ignore
++        except InvalidAlgorithmError:
++            cls.logger.debug("Only the HS256 algorithm is supported.")  # type: ignore
+         except UserDoesNotExist:
+             cls.logger.debug(  # type: ignore
+                 "Invalid token: user %s does not exist", dtoken['username']
+-- 
+2.39.2
+
diff --git a/patches/series b/patches/series
index 8804134cc..93354a011 100644
--- a/patches/series
+++ b/patches/series
@@ -9,3 +9,4 @@
 0009-fix-4759-run-ceph-crash-daemon-with-www-data-group-f.patch
 0010-debian-add-missing-bcrypt-to-manager-.requires.patch
 0011-fix-compatibility-with-CPUs-not-supporting-SSE-4.1-i.patch
+0012-backport-mgr-dashboard-simplify-authentication-proto.patch
-- 
2.39.2

[pve-devel] [PATCH v3 ceph 2/2] mgr/dashboard: add patch that removes PyOpenSSL-related usages

[Max Carrara]

This patch allows the dashboard to work again with TLS enabled; it
however disables the possibility to create self-signed certs via the
`ceph` CLI. This means that users will have to supply the correct
key/cert pair themselves, which are just a few extra steps instead. [0]

Users that try to generate a self-signed cert via the `ceph` CLI are
instead provided with instructions on how to generate and configure a
key/cert pair themselves.

Additionally, the check whether the cert and key match is removed during
the dashboard's launch.

See the patch for additional details.

[0]: https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
 ...move-ability-to-create-and-check-TLS.patch | 101 ++++++++++++++++++
 patches/series                                |   1 +
 2 files changed, 102 insertions(+)
 create mode 100644 patches/0013-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch

diff --git a/patches/0013-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch b/patches/0013-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
new file mode 100644
index 000000000..59c5263da
--- /dev/null
+++ b/patches/0013-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
@@ -0,0 +1,101 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Max Carrara <m.carrara@proxmox.com>
+Date: Thu, 4 Jan 2024 17:37:50 +0100
+Subject: [PATCH] mgr/dashboard: remove ability to create and check TLS
+ key/cert pairs
+
+In order to avoid running into PyO3-related issues [0] with PyOpenSSL,
+the ability to create self-signed certs is disabled - the command
+`ceph dashboard create-self-signed-cert` is made to always return an
+error.
+
+The command's error message contains the manual steps the user may
+follow in order to set the certificate themselves, as well as a link
+to the Ceph Dashboard documentation regarding TLS support. [1]
+
+Furthermore, the check on start-up, that verifies that the configured
+key/cert pair actually match, is also removed. This means that users
+need to ensure themselves that the correct pair is supplied -
+otherwise their browser will complain.
+
+These changes allow the dashboard to launch with TLS enabled again.
+
+[0]: https://tracker.ceph.com/issues/63529
+[1]: https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support
+
+Signed-off-by: Max Carrara <m.carrara@proxmox.com>
+---
+ src/pybind/mgr/dashboard/module.py | 41 ++++++++++++++++++++----------
+ 1 file changed, 27 insertions(+), 14 deletions(-)
+
+diff --git a/src/pybind/mgr/dashboard/module.py b/src/pybind/mgr/dashboard/module.py
+index 68725be6e35..9db55a3ee93 100644
+--- a/src/pybind/mgr/dashboard/module.py
++++ b/src/pybind/mgr/dashboard/module.py
+@@ -23,8 +23,7 @@ if TYPE_CHECKING:
+ 
+ from mgr_module import CLIReadCommand, CLIWriteCommand, HandleCommandResult, \
+     MgrModule, MgrStandbyModule, NotifyType, Option, _get_localized_key
+-from mgr_util import ServerConfigException, build_url, \
+-    create_self_signed_cert, get_default_addr, verify_tls_files
++from mgr_util import ServerConfigException, build_url, get_default_addr
+ 
+ from . import mgr
+ from .controllers import Router, json_error_page
+@@ -172,11 +171,14 @@ class CherryPyConfig(object):
+             else:
+                 pkey_fname = self.get_localized_module_option('key_file')  # type: ignore
+ 
+-            verify_tls_files(cert_fname, pkey_fname)
+-
+             # Create custom SSL context to disable TLS 1.0 and 1.1.
+             context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
+-            context.load_cert_chain(cert_fname, pkey_fname)
++
++            try:
++                context.load_cert_chain(cert_fname, pkey_fname)
++            except ssl.SSLError:
++                raise ServerConfigException("No certificate configured")
++
+             if sys.version_info >= (3, 7):
+                 if Settings.UNSAFE_TLS_v1_2:
+                     context.minimum_version = ssl.TLSVersion.TLSv1_2
+@@ -473,15 +475,26 @@ class Module(MgrModule, CherryPyConfig):
+ 
+     @CLIWriteCommand("dashboard create-self-signed-cert")
+     def set_mgr_created_self_signed_cert(self):
+-        cert, pkey = create_self_signed_cert('IT', 'ceph-dashboard')
+-        result = HandleCommandResult(*self.set_ssl_certificate(inbuf=cert))
+-        if result.retval != 0:
+-            return result
+-
+-        result = HandleCommandResult(*self.set_ssl_certificate_key(inbuf=pkey))
+-        if result.retval != 0:
+-            return result
+-        return 0, 'Self-signed certificate created', ''
++        from textwrap import dedent
++
++        err = """
++        Creating self-signed certificates is currently not available.
++        However, you can still set a key and certificate pair manually:
++
++        1. Generate a private key and self-signed certificate:
++          # openssl req -newkey rsa:2048 -nodes -x509 \\
++          -keyout /root/dashboard-key.pem -out /root/dashboard-cert.pem -sha512 \\
++          -days 3650 -subj "/CN=IT/O=ceph-mgr-dashboard" -utf8
++
++        2. Set the corresponding config keys for the key/cert pair:
++          # ceph config-key set mgr/dashboard/key -i /root/dashboard-key.pem
++          # ceph config-key set mgr/dashboard/crt -i /root/dashboard-crt.pem
++
++        For more information on how to configure TLS for the dashboard, visit:
++        https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support
++        """
++
++        return -errno.ENOTSUP, '', dedent(err).strip()
+ 
+     @CLIWriteCommand("dashboard set-rgw-credentials")
+     def set_rgw_credentials(self):
+-- 
+2.39.2
+
diff --git a/patches/series b/patches/series
index 93354a011..924f3dadd 100644
--- a/patches/series
+++ b/patches/series
@@ -10,3 +10,4 @@
 0010-debian-add-missing-bcrypt-to-manager-.requires.patch
 0011-fix-compatibility-with-CPUs-not-supporting-SSE-4.1-i.patch
 0012-backport-mgr-dashboard-simplify-authentication-proto.patch
+0013-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
-- 
2.39.2

[pve-devel] applied-series: [PATCH v3 ceph 0/2] Complete Workaround For Ceph Dashboard

[Thomas Lamprecht]

Am 05/01/2024 um 15:07 schrieb Max Carrara:
> These two patches allow the Ceph Dashboard to work again as intended.
> 
> The first patch provides a backport of a pull request [0] which removes
> the dependency on PyJWT. [1]
> 
> The second patch removes two usages of functions that use PyOpenSSL, [2]
> which prevents running into sub-interpreter-related issues with
> PyO3. [3] This is achieved by removing a check during the dashboard's
> launch as well as prohibiting users from creating self-signed TLS certs
> via the `ceph` CLI. Users may still provide a key/cert pair manually
> and are instructed to do so once they try to execute the respective
> command. [4]
> 
> 
> [0]: https://github.com/ceph/ceph/pull/54710
> [1]: https://github.com/jpadilla/pyjwt
> [2]: https://github.com/pyca/pyopenssl
> [3]: https://tracker.ceph.com/issues/63529
> [4]: `ceph dashboard create-self-signed-cert`
> 
> Max Carrara (2):
>   mgr/dashboard: add backport that allows the dashboard to work again
>   mgr/dashboard: add patch that removes PyOpenSSL-related usages
> 
>  ...hboard-simplify-authentication-proto.patch | 279 ++++++++++++++++++
>  ...move-ability-to-create-and-check-TLS.patch | 101 +++++++
>  patches/series                                |   2 +
>  3 files changed, 382 insertions(+)
>  create mode 100644 patches/0012-backport-mgr-dashboard-simplify-authentication-proto.patch
>  create mode 100644 patches/0013-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch
> 


applied, thanks!