[pve-devel] [PATCH master ceph, quincy-stable-8 ceph, pve-storage, pve-manager 0/8] Fix #4759: Configure Permissions for ceph-crash.service

[Max Carrara <m.carrara@proxmox.com>]

Introduction
------------

This series fixes #4759 [0], an issue where Ceph's crash daemon is
unable to post crash logs due to insufficient permissions, through an
adaptation of our `pveceph` CLI as well as an accompanying Debian
postinst hook.

In essence, this series ensures that the crash daemon can authenticate
with its Ceph cluster without requiring elevated privileges. 

For this to work, the following conditions required:
  1.  A key named 'client.crash' must be stored in the Ceph cluster
      itself
  2.  The key must be saved to a '.keyring' file which can be read by
      the `ceph` user (in order to authenticate with the cluster)
  3.  A reference to the '.keyring' file's location must be provided in
      a 'client.crash' section within the '/etc/pve/ceph.conf' file


Implementation
--------------

When creating a cluster's first monitor via `pveceph create mon`, the
'client.crash' key is automatically generated and saved to
'/etc/pve/ceph/ceph.client.crash.keyring'. This file is then referenced
via the new '[client.crash]' section in '/etc/pve/ceph.conf'.

To allow the crash daemon to actually send its crash logs to the
cluster, a postinst hook for both Ceph Reef and Ceph Quincy is provided
respectively in patches 1 and 2.

In order to support the new '[client.crash]' section within our tooling,
the writer for '/etc/pve/ceph.conf' is updated in patch 3.

Furthermore, the 'keyring' file's directory, '/etc/pve/ceph/', is added
for future non-sensitive configuration files regarding Ceph which the
`ceph` user should be allowed to read without requiring elevated
privileges (and to avoid clutter in '/etc/pve/').


Updating Existing Clusters' Configuration
-----------------------------------------

Existing clusters' configuration is adapted via a Debian postinst hook
added in patch 8. This hook ensures that every existing cluster's
configuration follows the methodolody introduced in the previous
section.

Most importantly, the hook does not generate a new key if one is
already known to Ceph. However, it will still ensure that the key is
saved to '/etc/pve/ceph/ceph.client.crash.keyring' and referenced
accordingly in '/etc/pve/ceph.conf'.

The hook will also not alter any files if the cluster's configuration
already meets the required criteria.


Testing
-------

The CLI as well as the Debian postinst hook have both been thoroughly
tested by going through several scenarios that might exist in the wild.
The postinst hook specifically accounts for:
  * Ceph not being installed or configured
  * Connection to RADOS failing
  * An already existing 'client.crash' key in Ceph
  * An already existing '/etc/pve/ceph/ceph.client.crash.keyring' file
    with expected or unexpected contents
  * A missing '[client.crash]' section in '/etc/pve/ceph.conf'
  * A '[client.crash]' section in '/etc/pve/ceph.conf' which doesn't
    reference any key or references a different key


[0]: https://bugzilla.proxmox.com/show_bug.cgi?id=4759



ceph (master):

Max Carrara (1):
  debian: add patch to fix ceph crash dir permissions in postinst hook

 ...rmissions-of-subdirectories-of-var-l.patch | 42 +++++++++++++++++++
 patches/series                                |  1 +
 2 files changed, 43 insertions(+)
 create mode 100644 patches/0015-debian-adjust-permissions-of-subdirectories-of-var-l.patch


ceph (quincy-stable-8):

Max Carrara (1):
  debian: add patch to fix ceph crash dir permissions in postinst hook

 ...rmissions-of-subdirectories-of-var-l.patch | 42 +++++++++++++++++++
 patches/series                                |  1 +
 2 files changed, 43 insertions(+)
 create mode 100644 patches/0024-debian-adjust-permissions-of-subdirectories-of-var-l.patch


pve-storage:

Max Carrara (1):
  cephconfig: support sections in the format of [client.$NAME]

 src/PVE/CephConfig.pm | 1 +
 1 file changed, 1 insertion(+)


pve-manager:

Max Carrara (5):
  ceph: fix edge case of wrong files being deleted on purge
  fix #4759: ceph: configure keyring for ceph-crash.service
  ceph: create '/etc/pve/ceph' during `pveceph init`
  debian/postinst: fix shellcheck warning
  fix #4759: debian/postinst: configure ceph-crash.service and its key

 PVE/API2/Ceph.pm     |   5 ++
 PVE/API2/Ceph/MON.pm |  28 ++++++++++-
 PVE/Ceph/Services.pm |  12 ++++-
 PVE/Ceph/Tools.pm    |  92 ++++++++++++++++++++++++++++++-----
 debian/postinst      | 111 ++++++++++++++++++++++++++++++++++++++++++-
 5 files changed, 232 insertions(+), 16 deletions(-)

-- 
2.39.2