Re: [pve-devel] [PATCH manager] ui: ceph installer: Run ceph/init if joining a cluster

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

On 20/12/2023 10:53, Mira Limbeck wrote:
> On 12/19/23 18:09, Aaron Lauterer wrote:
>> The commit message doesn't explain the actual issue that it is trying to
>> solve.
>> AFAICT we do not need the ceph.conf symlinked right away for normal PVE
>> operations. If it is not present in /etc/ceph/ceph.conf, the RBD and
>> CephFS connections will use the dedicated parameters to connect and
>> authenticate against the Ceph cluster, which are all present in the
>> /etc/pve/priv... directory.
>>
>> And even in a newly set up HCI PVE + Ceph cluster, the symlink for the
>> Ceph config will only be created once you set up a Ceph service (MON,
>> MGR, MDS, OSD) on that node.
>>
>> Even with a valid use case (see first line of my response), I am not
>> sure if the UI is the right place, as a user can still run into the same
>> situation if they use 'pveceph install' on the CLI without running
>> 'pveceph init' afterwards.
>> Therefore, if there is a reason to always have the symlink, wouldn't it
>> be better to add it to the install routine to create the symlink if the
>> /etc/pve/ceph.conf file is detected?
> 
> 
> Yes, you're right. Since we call `pveceph install` even when installing
> from the GUI, that should be the place to add it.
> 
> We've had a few cases where Ceph commands could not be run on the newly
> added host, even though Ceph was installed because of the missing symlink.
> Usually on a hyper-converged I'd assume that Ceph commands can simply be
> run without specifying a Ceph config, since that's how we set it up. But
> for newly added nodes that's currently not the case.
> 
> This also affects the system reports, since we don't pass the Ceph
> config there at all for any of the Ceph commands.
> 

FWIW, we could also create such links on cluster join if ceph.conf is
present in the pmxcfs, as then client-operations would already work
without running pveceph (server) install, e.g., on a cluster with both
ceph server nodes and others, e.g., compute/guest focused, nodes.

Not saying that this is the ultimate way to go, but should be IMO at
least evaluated w.r.t pros/cons.