From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8459791A76 for ; Tue, 30 Jan 2024 19:41:01 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 249C832549 for ; Tue, 30 Jan 2024 19:41:01 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 30 Jan 2024 19:40:59 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 8102349369 for ; Tue, 30 Jan 2024 19:40:59 +0100 (CET) From: Max Carrara To: pve-devel@lists.proxmox.com Date: Tue, 30 Jan 2024 19:40:33 +0100 Message-Id: <20240130184041.1125674-1-m.carrara@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.039 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pve-devel] [PATCH master ceph, quincy-stable-8 ceph, pve-storage, pve-manager 0/8] Fix #4759: Configure Permissions for ceph-crash.service X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jan 2024 18:41:01 -0000 Introduction ------------ This series fixes #4759 [0], an issue where Ceph's crash daemon is unable to post crash logs due to insufficient permissions, through an adaptation of our `pveceph` CLI as well as an accompanying Debian postinst hook. In essence, this series ensures that the crash daemon can authenticate with its Ceph cluster without requiring elevated privileges. For this to work, the following conditions required: 1. A key named 'client.crash' must be stored in the Ceph cluster itself 2. The key must be saved to a '.keyring' file which can be read by the `ceph` user (in order to authenticate with the cluster) 3. A reference to the '.keyring' file's location must be provided in a 'client.crash' section within the '/etc/pve/ceph.conf' file Implementation -------------- When creating a cluster's first monitor via `pveceph create mon`, the 'client.crash' key is automatically generated and saved to '/etc/pve/ceph/ceph.client.crash.keyring'. This file is then referenced via the new '[client.crash]' section in '/etc/pve/ceph.conf'. To allow the crash daemon to actually send its crash logs to the cluster, a postinst hook for both Ceph Reef and Ceph Quincy is provided respectively in patches 1 and 2. In order to support the new '[client.crash]' section within our tooling, the writer for '/etc/pve/ceph.conf' is updated in patch 3. Furthermore, the 'keyring' file's directory, '/etc/pve/ceph/', is added for future non-sensitive configuration files regarding Ceph which the `ceph` user should be allowed to read without requiring elevated privileges (and to avoid clutter in '/etc/pve/'). Updating Existing Clusters' Configuration ----------------------------------------- Existing clusters' configuration is adapted via a Debian postinst hook added in patch 8. This hook ensures that every existing cluster's configuration follows the methodolody introduced in the previous section. Most importantly, the hook does not generate a new key if one is already known to Ceph. However, it will still ensure that the key is saved to '/etc/pve/ceph/ceph.client.crash.keyring' and referenced accordingly in '/etc/pve/ceph.conf'. The hook will also not alter any files if the cluster's configuration already meets the required criteria. Testing ------- The CLI as well as the Debian postinst hook have both been thoroughly tested by going through several scenarios that might exist in the wild. The postinst hook specifically accounts for: * Ceph not being installed or configured * Connection to RADOS failing * An already existing 'client.crash' key in Ceph * An already existing '/etc/pve/ceph/ceph.client.crash.keyring' file with expected or unexpected contents * A missing '[client.crash]' section in '/etc/pve/ceph.conf' * A '[client.crash]' section in '/etc/pve/ceph.conf' which doesn't reference any key or references a different key [0]: https://bugzilla.proxmox.com/show_bug.cgi?id=4759 ceph (master): Max Carrara (1): debian: add patch to fix ceph crash dir permissions in postinst hook ...rmissions-of-subdirectories-of-var-l.patch | 42 +++++++++++++++++++ patches/series | 1 + 2 files changed, 43 insertions(+) create mode 100644 patches/0015-debian-adjust-permissions-of-subdirectories-of-var-l.patch ceph (quincy-stable-8): Max Carrara (1): debian: add patch to fix ceph crash dir permissions in postinst hook ...rmissions-of-subdirectories-of-var-l.patch | 42 +++++++++++++++++++ patches/series | 1 + 2 files changed, 43 insertions(+) create mode 100644 patches/0024-debian-adjust-permissions-of-subdirectories-of-var-l.patch pve-storage: Max Carrara (1): cephconfig: support sections in the format of [client.$NAME] src/PVE/CephConfig.pm | 1 + 1 file changed, 1 insertion(+) pve-manager: Max Carrara (5): ceph: fix edge case of wrong files being deleted on purge fix #4759: ceph: configure keyring for ceph-crash.service ceph: create '/etc/pve/ceph' during `pveceph init` debian/postinst: fix shellcheck warning fix #4759: debian/postinst: configure ceph-crash.service and its key PVE/API2/Ceph.pm | 5 ++ PVE/API2/Ceph/MON.pm | 28 ++++++++++- PVE/Ceph/Services.pm | 12 ++++- PVE/Ceph/Tools.pm | 92 ++++++++++++++++++++++++++++++----- debian/postinst | 111 ++++++++++++++++++++++++++++++++++++++++++- 5 files changed, 232 insertions(+), 16 deletions(-) -- 2.39.2