[pve-devel] [PATCH v2 master ceph, quincy-stable 8 ceph, pve-storage, pve-manager 00/11] Fix #4759: Configure Permissions for ceph-crash.service

[Max Carrara <m.carrara@proxmox.com>]

This marks version 02 of the patch series "Fix #4759: Configure
Permissions for ceph-crash.service" [0]. 

Each patch also lists which changes have been made between versions.

Other changes not otherwise mentioned in patches:
  v1 --> v2:
    * drop patch regarding POSIX compatibility in debian/postinst hook 
      as suggested [1]


Regarding `ceph.client.crash.keyring`
-------------------------------------

One idea [2] that was mentioned was to have a keyring for each host
instead of sharing a keyring on pmxcfs. While that is indeed possible,
it would require host-specific `client.crash` keys, each requiring
a separate section ('client.crash.$HOSTNAME'). Alternatively, the
default search path location may also be changed, however, the Ceph docs
do not recommend this [3]:

> It is possible to override this search-path location by adding a
> `keyring` option in the `[global]` section of your Ceph configuration
> file, but this is not recommended.

In our case, `ceph-crash` appears to try
'/etc/pve/priv/ceph.client.crash.$HOSTNAME.keyring' first (and
subsequently logs an authentication error) before attempting to
authenticate via other clients/keyrings, which aligns with what Fabian
had discovered.

It is therefore easier to:
  * have one keyring file at '/etc/pve/ceph/ceph.client.crash.keyring'
  * have the configuration point to that file
  * patch Ceph in order to make `ceph-crash` try to authenticate with
    'client.crash' first

This gets rid of the unnecessary warnings and errors that otherwise show
up in the systemd journal.

Furthermore, the Ceph Crash docs use 'client.crash' [4], so I personally
doubt that upstream expects that people use the host-specific key;
rather, it seems that any of the three keys is fine.

Further points I have regarding sharing the key via pmxcfs:
  * in case the key & keyring file ever need to be rotated, only a
    single file location and cephx auth entry needs to be updated
    - instead of having to update every host's key *and* each
      corresponding cephx auth entry

  * the key is generated only once, exactly when the first monitor is
    created
    - otherwise we would need to ensure that the host-specific keys
      exist and are reliably added to cephx - e.g. per each use of
      `pveceph init` while also having to set up the first host-specific
      key when the first monitor is created via `pveceph mon create`

  * the keyring file's name doesn't need to be changed if the hostname
    changes (although that's admittedly a very minor benefit)

  * overall less administrative friction for the user (again, single
    location, single cephx auth entry)

Therefore, I decided to keep most things as they were with other
suggestions incorporated. The order of the identities that `ceph-crash`
checks is adapted, in order to "suppress" the warnings in the systemd
journal.

Though, I will of course consider any feedback in this regard,
especially if my reasoning here proves to be erroneous.


[0]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061546.html
[1]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061561.html
[2]: https://lists.proxmox.com/pipermail/pve-devel/2024-January/061566.html
[3]: https://docs.ceph.com/en/reef/rados/configuration/auth-config-ref/#enabling-cephx
[4]: https://docs.ceph.com/en/quincy/mgr/crash/#enabling



ceph (master):

Max Carrara (2):
  debian: add patch to fix ceph crash dir permissions in postinst hook
  patches: add patch that reorders clients used by ceph-crash

 ...rmissions-of-subdirectories-of-var-l.patch | 50 +++++++++++++++++++
 ...h-crash-change-order-of-client-names.patch | 30 +++++++++++
 patches/series                                |  2 +
 3 files changed, 82 insertions(+)
 create mode 100644 patches/0015-debian-adjust-permissions-of-subdirectories-of-var-l.patch
 create mode 100644 patches/0016-ceph-crash-change-order-of-client-names.patch


ceph (quincy-stable-8):

Max Carrara (2):
  debian: add patch to fix ceph crash dir permissions in postinst hook
  patches: add patch that reorders clients used by ceph-crash

 ...rmissions-of-subdirectories-of-var-l.patch | 50 +++++++++++++++++++
 ...h-crash-change-order-of-client-names.patch | 30 +++++++++++
 patches/series                                |  2 +
 3 files changed, 82 insertions(+)
 create mode 100644 patches/0024-debian-adjust-permissions-of-subdirectories-of-var-l.patch
 create mode 100644 patches/0025-ceph-crash-change-order-of-client-names.patch


pve-storage:

Max Carrara (3):
  cephconfig: align our parser more with Ceph's parser
  cephconfig: allow writing arbitrary sections
  amend! cephconfig: allow writing arbitrary sections

 src/PVE/CephConfig.pm | 42 +++++++++++++++++++++++++++---------------
 1 file changed, 27 insertions(+), 15 deletions(-)


pve-manager:

Max Carrara (4):
  ceph: fix edge case of wrong files being deleted on purge
  fix #4759: ceph: configure keyring for ceph-crash.service
  ceph: create '/etc/pve/ceph' during `pveceph init`
  fix #4759: debian/postinst: configure ceph-crash.service and its key

 PVE/API2/Ceph.pm     |   5 ++
 PVE/API2/Ceph/MON.pm |  17 ++++++-
 PVE/Ceph/Tools.pm    |  57 +++++++++++++++++++---
 debian/postinst      | 113 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 185 insertions(+), 7 deletions(-)


-- 
2.39.2