From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 19EFD93697 for ; Mon, 5 Feb 2024 18:55:03 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A5D711B443 for ; Mon, 5 Feb 2024 18:54:32 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 5 Feb 2024 18:54:31 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 0A3E744478 for ; Mon, 5 Feb 2024 18:54:31 +0100 (CET) From: Max Carrara To: pve-devel@lists.proxmox.com Date: Mon, 5 Feb 2024 18:54:08 +0100 Message-Id: <20240205175419.1271680-1-m.carrara@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.030 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pve-devel] [PATCH v2 master ceph, quincy-stable 8 ceph, pve-storage, pve-manager 00/11] Fix #4759: Configure Permissions for ceph-crash.service X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Feb 2024 17:55:03 -0000 This marks version 02 of the patch series "Fix #4759: Configure Permissions for ceph-crash.service" [0]. Each patch also lists which changes have been made between versions. Other changes not otherwise mentioned in patches: v1 --> v2: * drop patch regarding POSIX compatibility in debian/postinst hook as suggested [1] Regarding `ceph.client.crash.keyring` ------------------------------------- One idea [2] that was mentioned was to have a keyring for each host instead of sharing a keyring on pmxcfs. While that is indeed possible, it would require host-specific `client.crash` keys, each requiring a separate section ('client.crash.$HOSTNAME'). Alternatively, the default search path location may also be changed, however, the Ceph docs do not recommend this [3]: > It is possible to override this search-path location by adding a > `keyring` option in the `[global]` section of your Ceph configuration > file, but this is not recommended. In our case, `ceph-crash` appears to try '/etc/pve/priv/ceph.client.crash.$HOSTNAME.keyring' first (and subsequently logs an authentication error) before attempting to authenticate via other clients/keyrings, which aligns with what Fabian had discovered. It is therefore easier to: * have one keyring file at '/etc/pve/ceph/ceph.client.crash.keyring' * have the configuration point to that file * patch Ceph in order to make `ceph-crash` try to authenticate with 'client.crash' first This gets rid of the unnecessary warnings and errors that otherwise show up in the systemd journal. Furthermore, the Ceph Crash docs use 'client.crash' [4], so I personally doubt that upstream expects that people use the host-specific key; rather, it seems that any of the three keys is fine. Further points I have regarding sharing the key via pmxcfs: * in case the key & keyring file ever need to be rotated, only a single file location and cephx auth entry needs to be updated - instead of having to update every host's key *and* each corresponding cephx auth entry * the key is generated only once, exactly when the first monitor is created - otherwise we would need to ensure that the host-specific keys exist and are reliably added to cephx - e.g. per each use of `pveceph init` while also having to set up the first host-specific key when the first monitor is created via `pveceph mon create` * the keyring file's name doesn't need to be changed if the hostname changes (although that's admittedly a very minor benefit) * overall less administrative friction for the user (again, single location, single cephx auth entry) Therefore, I decided to keep most things as they were with other suggestions incorporated. The order of the identities that `ceph-crash` checks is adapted, in order to "suppress" the warnings in the systemd journal. Though, I will of course consider any feedback in this regard, especially if my reasoning here proves to be erroneous. [0]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061546.html [1]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061561.html [2]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061566.html [3]: https://docs.ceph.com/en/reef/rados/configuration/auth-config-ref/#enabling-cephx [4]: https://docs.ceph.com/en/quincy/mgr/crash/#enabling ceph (master): Max Carrara (2): debian: add patch to fix ceph crash dir permissions in postinst hook patches: add patch that reorders clients used by ceph-crash ...rmissions-of-subdirectories-of-var-l.patch | 50 +++++++++++++++++++ ...h-crash-change-order-of-client-names.patch | 30 +++++++++++ patches/series | 2 + 3 files changed, 82 insertions(+) create mode 100644 patches/0015-debian-adjust-permissions-of-subdirectories-of-var-l.patch create mode 100644 patches/0016-ceph-crash-change-order-of-client-names.patch ceph (quincy-stable-8): Max Carrara (2): debian: add patch to fix ceph crash dir permissions in postinst hook patches: add patch that reorders clients used by ceph-crash ...rmissions-of-subdirectories-of-var-l.patch | 50 +++++++++++++++++++ ...h-crash-change-order-of-client-names.patch | 30 +++++++++++ patches/series | 2 + 3 files changed, 82 insertions(+) create mode 100644 patches/0024-debian-adjust-permissions-of-subdirectories-of-var-l.patch create mode 100644 patches/0025-ceph-crash-change-order-of-client-names.patch pve-storage: Max Carrara (3): cephconfig: align our parser more with Ceph's parser cephconfig: allow writing arbitrary sections amend! cephconfig: allow writing arbitrary sections src/PVE/CephConfig.pm | 42 +++++++++++++++++++++++++++--------------- 1 file changed, 27 insertions(+), 15 deletions(-) pve-manager: Max Carrara (4): ceph: fix edge case of wrong files being deleted on purge fix #4759: ceph: configure keyring for ceph-crash.service ceph: create '/etc/pve/ceph' during `pveceph init` fix #4759: debian/postinst: configure ceph-crash.service and its key PVE/API2/Ceph.pm | 5 ++ PVE/API2/Ceph/MON.pm | 17 ++++++- PVE/Ceph/Tools.pm | 57 +++++++++++++++++++--- debian/postinst | 113 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 185 insertions(+), 7 deletions(-) -- 2.39.2