From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 2564A9114A for ; Fri, 26 Jan 2024 16:45:18 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id EF9E937F53 for ; Fri, 26 Jan 2024 16:45:17 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 26 Jan 2024 16:45:16 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 3FCCA492E3 for ; Fri, 26 Jan 2024 16:45:16 +0100 (CET) From: Max Carrara To: pve-devel@lists.proxmox.com Date: Fri, 26 Jan 2024 16:44:39 +0100 Message-Id: <20240126154440.657816-2-m.carrara@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240126154440.657816-1-m.carrara@proxmox.com> References: <20240126154440.657816-1-m.carrara@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.168 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LOTSOFHASH 0.25 Emails with lots of hash-like gibberish SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pve-devel] [PATCH v2 quincy-stable-8 ceph 1/2] patches: include patches regarding RocksDB and dashboard from master X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jan 2024 15:45:18 -0000 This commit essentially contains all changes from the following commits (most recent last): * f35168f6713d5f3fa1aaa8c572d754b61c458d91 * 86a553d66e69176940959530d4fedcbcbbab54d9 * ab5c03b44d78c4e4b233ff5a310888592dbb9bb4 The series file and the patches' prefixed numbers have been updated correspondingly. A very minor adaptation has been made to the patch added by commit f35168f6713d5f3fa1aaa8c572d754b61c458d91 in order to get it to apply correctly. Signed-off-by: Max Carrara --- Changes v1 --> v2: * None ...hboard-simplify-authentication-proto.patch | 279 ++++++++++++++++++ patches/0021-debian-rules-fix-buildtype.patch | 22 -- ...move-ability-to-create-and-check-TLS.patch | 101 +++++++ ...cksb-inherit-parent-cmake-cxx-flags.patch} | 0 patches/series | 5 +- 5 files changed, 383 insertions(+), 24 deletions(-) create mode 100644 patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch delete mode 100644 patches/0021-debian-rules-fix-buildtype.patch create mode 100644 patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch rename patches/{0022-rocksb-inherit-parent-cmake-cxx-flags.patch => 0023-rocksb-inherit-parent-cmake-cxx-flags.patch} (100%) diff --git a/patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch b/patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch new file mode 100644 index 000000000..88dbcd04a --- /dev/null +++ b/patches/0021-backport-mgr-dashboard-simplify-authentication-proto.patch @@ -0,0 +1,279 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Max Carrara +Date: Tue, 2 Jan 2024 13:02:51 +0000 +Subject: [PATCH] backport: mgr/dashboard: simplify authentication protocol + +This is a backport of https://github.com/ceph/ceph/pull/54710 which +fixes the Ceph Dashboard not being able to launch on Ceph Reef running +on Debian Bookworm. + +This is achieved by removing the dependency on `PyJWT` (Python) and thus +transitively also removing the dependency on `cryptography` (Python). +For more information, see the original pull request. + +Note that the Ceph Dashboard still cannot be used if TLS is activated, +because `pyOpenSSL` is used to verify certs during launch. Disabling +TLS via `ceph config set mgr mgr/dashboard/ssl false` and using e.g. +a reverse proxy can be used as a workaround. + +A separate patch is required to allow the dashboard to run with TLS +enabled. + +Fixes: https://forum.proxmox.com/threads/ceph-warning-post-upgrade-to-v8.129371 +Signed-off-by: Daniel Persson +Signed-off-by: Max Carrara +--- + ceph.spec.in | 4 -- + debian/control | 1 - + src/pybind/mgr/dashboard/constraints.txt | 1 - + src/pybind/mgr/dashboard/exceptions.py | 12 ++++ + .../mgr/dashboard/requirements-lint.txt | 1 + + .../mgr/dashboard/requirements-test.txt | 1 + + src/pybind/mgr/dashboard/requirements.txt | 1 - + src/pybind/mgr/dashboard/services/auth.py | 70 ++++++++++++++++--- + 8 files changed, 75 insertions(+), 16 deletions(-) + +diff --git a/ceph.spec.in b/ceph.spec.in +index e4cbbef6943..457d7a6dc56 100644 +--- a/ceph.spec.in ++++ b/ceph.spec.in +@@ -403,7 +403,6 @@ BuildRequires: xmlsec1-nss + BuildRequires: xmlsec1-openssl + BuildRequires: xmlsec1-openssl-devel + BuildRequires: python%{python3_pkgversion}-cherrypy +-BuildRequires: python%{python3_pkgversion}-jwt + BuildRequires: python%{python3_pkgversion}-routes + BuildRequires: python%{python3_pkgversion}-scipy + BuildRequires: python%{python3_pkgversion}-werkzeug +@@ -416,7 +415,6 @@ BuildRequires: libxmlsec1-1 + BuildRequires: libxmlsec1-nss1 + BuildRequires: libxmlsec1-openssl1 + BuildRequires: python%{python3_pkgversion}-CherryPy +-BuildRequires: python%{python3_pkgversion}-PyJWT + BuildRequires: python%{python3_pkgversion}-Routes + BuildRequires: python%{python3_pkgversion}-Werkzeug + BuildRequires: python%{python3_pkgversion}-numpy-devel +@@ -608,7 +606,6 @@ Requires: ceph-prometheus-alerts = %{_epoch_prefix}%{version}-%{release} + Requires: python%{python3_pkgversion}-setuptools + %if 0%{?fedora} || 0%{?rhel} + Requires: python%{python3_pkgversion}-cherrypy +-Requires: python%{python3_pkgversion}-jwt + Requires: python%{python3_pkgversion}-routes + Requires: python%{python3_pkgversion}-werkzeug + %if 0%{?weak_deps} +@@ -617,7 +614,6 @@ Recommends: python%{python3_pkgversion}-saml + %endif + %if 0%{?suse_version} + Requires: python%{python3_pkgversion}-CherryPy +-Requires: python%{python3_pkgversion}-PyJWT + Requires: python%{python3_pkgversion}-Routes + Requires: python%{python3_pkgversion}-Werkzeug + Recommends: python%{python3_pkgversion}-python3-saml +diff --git a/debian/control b/debian/control +index e3f2073ce8f..ba34a163f54 100644 +--- a/debian/control ++++ b/debian/control +@@ -87,7 +87,6 @@ Build-Depends: automake, + python3-all-dev, + python3-cherrypy3, + python3-natsort, +- python3-jwt , + python3-pecan , + python3-bcrypt , + tox , +diff --git a/src/pybind/mgr/dashboard/constraints.txt b/src/pybind/mgr/dashboard/constraints.txt +index 55f81c92dec..fd614104880 100644 +--- a/src/pybind/mgr/dashboard/constraints.txt ++++ b/src/pybind/mgr/dashboard/constraints.txt +@@ -1,6 +1,5 @@ + CherryPy~=13.1 + more-itertools~=8.14 +-PyJWT~=2.0 + bcrypt~=3.1 + python3-saml~=1.4 + requests~=2.26 +diff --git a/src/pybind/mgr/dashboard/exceptions.py b/src/pybind/mgr/dashboard/exceptions.py +index 96cbc523356..d396a38d2c3 100644 +--- a/src/pybind/mgr/dashboard/exceptions.py ++++ b/src/pybind/mgr/dashboard/exceptions.py +@@ -121,3 +121,15 @@ class GrafanaError(Exception): + + class PasswordPolicyException(Exception): + pass ++ ++ ++class ExpiredSignatureError(Exception): ++ pass ++ ++ ++class InvalidTokenError(Exception): ++ pass ++ ++ ++class InvalidAlgorithmError(Exception): ++ pass +diff --git a/src/pybind/mgr/dashboard/requirements-lint.txt b/src/pybind/mgr/dashboard/requirements-lint.txt +index d82fa1ace1d..5fe9957c32a 100644 +--- a/src/pybind/mgr/dashboard/requirements-lint.txt ++++ b/src/pybind/mgr/dashboard/requirements-lint.txt +@@ -9,3 +9,4 @@ autopep8==1.5.7 + pyfakefs==4.5.0 + isort==5.5.3 + jsonschema==4.16.0 ++PyJWT~=2.0 +diff --git a/src/pybind/mgr/dashboard/requirements-test.txt b/src/pybind/mgr/dashboard/requirements-test.txt +index 4e925e8616f..78fd1d5b742 100644 +--- a/src/pybind/mgr/dashboard/requirements-test.txt ++++ b/src/pybind/mgr/dashboard/requirements-test.txt +@@ -2,3 +2,4 @@ pytest-cov + pytest-instafail + pyfakefs==4.5.0 + jsonschema==4.16.0 ++PyJWT~=2.0 +diff --git a/src/pybind/mgr/dashboard/requirements.txt b/src/pybind/mgr/dashboard/requirements.txt +index 8003d62a552..292971819c9 100644 +--- a/src/pybind/mgr/dashboard/requirements.txt ++++ b/src/pybind/mgr/dashboard/requirements.txt +@@ -1,7 +1,6 @@ + bcrypt + CherryPy + more-itertools +-PyJWT + pyopenssl + requests + Routes +diff --git a/src/pybind/mgr/dashboard/services/auth.py b/src/pybind/mgr/dashboard/services/auth.py +index f13963abffd..3c600231252 100644 +--- a/src/pybind/mgr/dashboard/services/auth.py ++++ b/src/pybind/mgr/dashboard/services/auth.py +@@ -1,17 +1,19 @@ + # -*- coding: utf-8 -*- + ++import base64 ++import hashlib ++import hmac + import json + import logging + import os + import threading + import time + import uuid +-from base64 import b64encode + + import cherrypy +-import jwt + + from .. import mgr ++from ..exceptions import ExpiredSignatureError, InvalidAlgorithmError, InvalidTokenError + from .access_control import LocalAuthenticator, UserDoesNotExist + + cherrypy.config.update({ +@@ -33,7 +35,7 @@ class JwtManager(object): + @staticmethod + def _gen_secret(): + secret = os.urandom(16) +- return b64encode(secret).decode('utf-8') ++ return base64.b64encode(secret).decode('utf-8') + + @classmethod + def init(cls): +@@ -45,6 +47,54 @@ class JwtManager(object): + mgr.set_store('jwt_secret', secret) + cls._secret = secret + ++ @classmethod ++ def array_to_base64_string(cls, message): ++ jsonstr = json.dumps(message, sort_keys=True).replace(" ", "") ++ string_bytes = base64.urlsafe_b64encode(bytes(jsonstr, 'UTF-8')) ++ return string_bytes.decode('UTF-8').replace("=", "") ++ ++ @classmethod ++ def encode(cls, message, secret): ++ header = {"alg": cls.JWT_ALGORITHM, "typ": "JWT"} ++ base64_header = cls.array_to_base64_string(header) ++ base64_message = cls.array_to_base64_string(message) ++ base64_secret = base64.urlsafe_b64encode(hmac.new( ++ bytes(secret, 'UTF-8'), ++ msg=bytes(base64_header + "." + base64_message, 'UTF-8'), ++ digestmod=hashlib.sha256 ++ ).digest()).decode('UTF-8').replace("=", "") ++ return base64_header + "." + base64_message + "." + base64_secret ++ ++ @classmethod ++ def decode(cls, message, secret): ++ split_message = message.split(".") ++ base64_header = split_message[0] ++ base64_message = split_message[1] ++ base64_secret = split_message[2] ++ ++ decoded_header = json.loads(base64.urlsafe_b64decode(base64_header)) ++ ++ if decoded_header['alg'] != cls.JWT_ALGORITHM: ++ raise InvalidAlgorithmError() ++ ++ incoming_secret = base64.urlsafe_b64encode(hmac.new( ++ bytes(secret, 'UTF-8'), ++ msg=bytes(base64_header + "." + base64_message, 'UTF-8'), ++ digestmod=hashlib.sha256 ++ ).digest()).decode('UTF-8').replace("=", "") ++ ++ if base64_secret != incoming_secret: ++ raise InvalidTokenError() ++ ++ # We add ==== as padding to ignore the requirement to have correct padding in ++ # the urlsafe_b64decode method. ++ decoded_message = json.loads(base64.urlsafe_b64decode(base64_message + "====")) ++ now = int(time.time()) ++ if decoded_message['exp'] < now: ++ raise ExpiredSignatureError() ++ ++ return decoded_message ++ + @classmethod + def gen_token(cls, username): + if not cls._secret: +@@ -59,13 +109,13 @@ class JwtManager(object): + 'iat': now, + 'username': username + } +- return jwt.encode(payload, cls._secret, algorithm=cls.JWT_ALGORITHM) # type: ignore ++ return cls.encode(payload, cls._secret) # type: ignore + + @classmethod + def decode_token(cls, token): + if not cls._secret: + cls.init() +- return jwt.decode(token, cls._secret, algorithms=cls.JWT_ALGORITHM) # type: ignore ++ return cls.decode(token, cls._secret) # type: ignore + + @classmethod + def get_token_from_header(cls): +@@ -99,8 +149,8 @@ class JwtManager(object): + @classmethod + def get_user(cls, token): + try: +- dtoken = JwtManager.decode_token(token) +- if not JwtManager.is_blocklisted(dtoken['jti']): ++ dtoken = cls.decode_token(token) ++ if not cls.is_blocklisted(dtoken['jti']): + user = AuthManager.get_user(dtoken['username']) + if user.last_update <= dtoken['iat']: + return user +@@ -110,10 +160,12 @@ class JwtManager(object): + ) + else: + cls.logger.debug('Token is block-listed') # type: ignore +- except jwt.ExpiredSignatureError: ++ except ExpiredSignatureError: + cls.logger.debug("Token has expired") # type: ignore +- except jwt.InvalidTokenError: ++ except InvalidTokenError: + cls.logger.debug("Failed to decode token") # type: ignore ++ except InvalidAlgorithmError: ++ cls.logger.debug("Only the HS256 algorithm is supported.") # type: ignore + except UserDoesNotExist: + cls.logger.debug( # type: ignore + "Invalid token: user %s does not exist", dtoken['username'] +-- +2.39.2 + diff --git a/patches/0021-debian-rules-fix-buildtype.patch b/patches/0021-debian-rules-fix-buildtype.patch deleted file mode 100644 index 8b6ef6b56..000000000 --- a/patches/0021-debian-rules-fix-buildtype.patch +++ /dev/null @@ -1,22 +0,0 @@ -From 1f4b106d49fc916994d97e273599f75caa904c3b Mon Sep 17 00:00:00 2001 -From: Mark Nelson -Date: Thu, 14 Dec 2023 05:19:46 +0000 -Subject: [PATCH] debian/rules: Fix build_type for massive performance gain - -Signed-off-by: Mark Nelson ---- - debian/rules | 1 + - 1 file changed, 1 insertion(+) - -diff --git a/debian/rules b/debian/rules -index ed7f4a255ed4b..b28abb7d62788 100755 ---- a/debian/rules -+++ b/debian/rules -@@ -29,6 +29,7 @@ extraopts += -DWITH_PYTHON3=3 - extraopts += -DWITH_CEPHFS_JAVA=ON - extraopts += -DWITH_CEPHFS_SHELL=ON - extraopts += -DWITH_SYSTEMD=ON -DCEPH_SYSTEMD_ENV_DIR=/etc/default -+extraopts += -DCMAKE_BUILD_TYPE=RelWithDebInfo - extraopts += -DWITH_GRAFANA=ON - ifeq ($(DEB_HOST_ARCH), amd64) - extraopts += -DWITH_RBD_RWL=ON diff --git a/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch new file mode 100644 index 000000000..59c5263da --- /dev/null +++ b/patches/0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch @@ -0,0 +1,101 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Max Carrara +Date: Thu, 4 Jan 2024 17:37:50 +0100 +Subject: [PATCH] mgr/dashboard: remove ability to create and check TLS + key/cert pairs + +In order to avoid running into PyO3-related issues [0] with PyOpenSSL, +the ability to create self-signed certs is disabled - the command +`ceph dashboard create-self-signed-cert` is made to always return an +error. + +The command's error message contains the manual steps the user may +follow in order to set the certificate themselves, as well as a link +to the Ceph Dashboard documentation regarding TLS support. [1] + +Furthermore, the check on start-up, that verifies that the configured +key/cert pair actually match, is also removed. This means that users +need to ensure themselves that the correct pair is supplied - +otherwise their browser will complain. + +These changes allow the dashboard to launch with TLS enabled again. + +[0]: https://tracker.ceph.com/issues/63529 +[1]: https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support + +Signed-off-by: Max Carrara +--- + src/pybind/mgr/dashboard/module.py | 41 ++++++++++++++++++++---------- + 1 file changed, 27 insertions(+), 14 deletions(-) + +diff --git a/src/pybind/mgr/dashboard/module.py b/src/pybind/mgr/dashboard/module.py +index 68725be6e35..9db55a3ee93 100644 +--- a/src/pybind/mgr/dashboard/module.py ++++ b/src/pybind/mgr/dashboard/module.py +@@ -23,8 +23,7 @@ if TYPE_CHECKING: + + from mgr_module import CLIReadCommand, CLIWriteCommand, HandleCommandResult, \ + MgrModule, MgrStandbyModule, NotifyType, Option, _get_localized_key +-from mgr_util import ServerConfigException, build_url, \ +- create_self_signed_cert, get_default_addr, verify_tls_files ++from mgr_util import ServerConfigException, build_url, get_default_addr + + from . import mgr + from .controllers import Router, json_error_page +@@ -172,11 +171,14 @@ class CherryPyConfig(object): + else: + pkey_fname = self.get_localized_module_option('key_file') # type: ignore + +- verify_tls_files(cert_fname, pkey_fname) +- + # Create custom SSL context to disable TLS 1.0 and 1.1. + context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) +- context.load_cert_chain(cert_fname, pkey_fname) ++ ++ try: ++ context.load_cert_chain(cert_fname, pkey_fname) ++ except ssl.SSLError: ++ raise ServerConfigException("No certificate configured") ++ + if sys.version_info >= (3, 7): + if Settings.UNSAFE_TLS_v1_2: + context.minimum_version = ssl.TLSVersion.TLSv1_2 +@@ -473,15 +475,26 @@ class Module(MgrModule, CherryPyConfig): + + @CLIWriteCommand("dashboard create-self-signed-cert") + def set_mgr_created_self_signed_cert(self): +- cert, pkey = create_self_signed_cert('IT', 'ceph-dashboard') +- result = HandleCommandResult(*self.set_ssl_certificate(inbuf=cert)) +- if result.retval != 0: +- return result +- +- result = HandleCommandResult(*self.set_ssl_certificate_key(inbuf=pkey)) +- if result.retval != 0: +- return result +- return 0, 'Self-signed certificate created', '' ++ from textwrap import dedent ++ ++ err = """ ++ Creating self-signed certificates is currently not available. ++ However, you can still set a key and certificate pair manually: ++ ++ 1. Generate a private key and self-signed certificate: ++ # openssl req -newkey rsa:2048 -nodes -x509 \\ ++ -keyout /root/dashboard-key.pem -out /root/dashboard-cert.pem -sha512 \\ ++ -days 3650 -subj "/CN=IT/O=ceph-mgr-dashboard" -utf8 ++ ++ 2. Set the corresponding config keys for the key/cert pair: ++ # ceph config-key set mgr/dashboard/key -i /root/dashboard-key.pem ++ # ceph config-key set mgr/dashboard/crt -i /root/dashboard-crt.pem ++ ++ For more information on how to configure TLS for the dashboard, visit: ++ https://docs.ceph.com/en/reef/mgr/dashboard/#ssl-tls-support ++ """ ++ ++ return -errno.ENOTSUP, '', dedent(err).strip() + + @CLIWriteCommand("dashboard set-rgw-credentials") + def set_rgw_credentials(self): +-- +2.39.2 + diff --git a/patches/0022-rocksb-inherit-parent-cmake-cxx-flags.patch b/patches/0023-rocksb-inherit-parent-cmake-cxx-flags.patch similarity index 100% rename from patches/0022-rocksb-inherit-parent-cmake-cxx-flags.patch rename to patches/0023-rocksb-inherit-parent-cmake-cxx-flags.patch diff --git a/patches/series b/patches/series index 73f66396c..ee897a78a 100644 --- a/patches/series +++ b/patches/series @@ -13,5 +13,6 @@ 0016-d-rules-fix-no-restart-on-upgrade.patch 0017-python3.10-pep-620.patch 0020-fix-4759-run-ceph-crash-daemon-with-www-data-group-f.patch -0021-debian-rules-fix-buildtype.patch -0022-rocksb-inherit-parent-cmake-cxx-flags.patch +0021-backport-mgr-dashboard-simplify-authentication-proto.patch +0022-mgr-dashboard-remove-ability-to-create-and-check-TLS.patch +0023-rocksb-inherit-parent-cmake-cxx-flags.patch -- 2.39.2