From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 9CA311FF0B2 for ; Tue, 08 Sep 2026 18:21:13 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id AD6C9215F3; Tue, 08 Sep 2026 18:21:09 +0200 (CEST) Message-ID: Date: Tue, 8 Sep 2026 18:21:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: [PATCH docs v2 3/5] cephx-migration: Move all requirements into prelude To: pve-devel@lists.proxmox.com, Maximiliano Sandoval References: <20260908153932.577051-1-m.sandoval@proxmox.com> <20260908153932.577051-4-m.sandoval@proxmox.com> Content-Language: en-US From: Jonas Theisen In-Reply-To: <20260908153932.577051-4-m.sandoval@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1788884455899 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.508 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: HIUMGZYI6LFVOJ3LNZ4ZJK7QGAPKHRAD X-Message-ID-Hash: HIUMGZYI6LFVOJ3LNZ4ZJK7QGAPKHRAD X-MailFrom: j.theisen@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 9/8/26 17:39, Maximiliano Sandoval wrote: > The migration should start after all requirements are meet. > > Note that the script will warn if it is not being run as root. There is no need > to mention that the script should be run as root. > > Signed-off-by: Maximiliano Sandoval > --- > pveceph.adoc | 51 ++++++++++++++++++++++++++------------------------- > 1 file changed, 26 insertions(+), 25 deletions(-) > > diff --git a/pveceph.adoc b/pveceph.adoc > index 625768f..40ca16b 100644 > --- a/pveceph.adoc > +++ b/pveceph.adoc > @@ -1363,21 +1363,35 @@ Reruns do not rotate completed keys again. After each run, invoke the helper > without options to check the current state and follow the next steps it prints. > > .Before You Begin > -* Upgrade `pve-manager` to 9.2.17 or newer and install the latest Ceph > - packages on every node. Staged client-key rotation requires Ceph > - 19.2.6-pve3, 20.2.4-pve3, or newer on every monitor. Then complete the > - rolling restart of all Ceph services. > +* Upgrade all cluster nodes to the latest {pve} packages. In particular, > + ** `pve-manager` '9.2.17' or newer > + ** Ceph '20.2.4-pve3' or newer if using Tentacle > + ** Ceph '19.2.6-pve3' or newer if using Squid > + ** Kernel 7.0 or newer. > +* Ensure all external clients are running Ceph and Kernel versions supporting `aes256k`. > +* Complete a rolling restart of Ceph services. > * Resolve health warnings and errors not listed in > - xref:pveceph_cephx_health_check_reference[Health Check Reference]. Ensure > - every node is reachable over SSH. > -* Run the helper as `root` on one node only. Do not start another migration, > - a rolling restart, or change Ceph keys with other tools while it runs. The > - helper's lock does not block direct Ceph authentication commands. > + xref:pveceph_cephx_health_check_reference[Health Check Reference]. > +* Ensure every node is reachable over SSH. Small knit: The trailing full stops are probably unnecessary for a list Also i would propose to move the "Ensure all external clients [...]" point to the bottom of the list since this will be the exception. To reflect this the point could also be formulated like: "If external clients are accessing the Ceph cluster ensure they support 'aes256k'" Imho this would reduce the risk of users getting confused. > + > +.Kernel Support Considerations > +A storage client requires support for `aes256k` in the `rbd` kernel module if: > + > +* A container is using a RBD storage. > +* or, A VM is using a RBD storage, if `krbd` is enabled on the storage configuration. > +* or, a CephFS is mounted, unless `fuse` is enabled in the storage configuration. > + > +Clients requiring kernel support should not downgrade to older kernel versions > +after the migration. > > 1. Migrate the Cluster-Owned Keys > +++++++++++++++++++++++++++++++++ > > -Review the cluster-owned migration in the default dry-run mode: > +WARNING: Do not start another migration, a rolling restart, or change Ceph keys > +with other tools during the migration. The helper's lock does not block direct > +Ceph authentication commands. > + > +Review the cluster-owned key migration in dry-run mode: > > [source,bash] > ---- > @@ -1405,21 +1419,8 @@ automatically. > 2. Migrate the Keys of Compatible Ceph Users > ++++++++++++++++++++++++++++++++++++++++++++ > > -[[pveceph_cipher_support]] > -.Check compatibility > -Several workloads can share one Ceph user. Migrate its key only when every > -client supports `aes256k`, including disconnected and external clients. > -Ceph programs from the updated {pve} packages support it; kernel clients > -require a *running* kernel version 7.0 or newer. > - > -[cols="2,3",options="header"] > -|=== > -|Virtual machine with RBD disks |Userspace, unless `krbd` is enabled. > -|Container on RBD |Always the kernel client. > -|CephFS mount |Kernel, unless `fuse` is enabled. > -|=== > - > -Check external systems too. If any affected client is incompatible or > +Migrate a user's key only once all its clients support `aes256k`, including > +disconnected and external clients. If any affected client is incompatible or > unknown, leave its user's key unchanged and postpone this step. > > .Stage the keys