Re: [PVE-User] Caution: ceph-mon service does not start after today's updates

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

On 26.11.20 16:03, Lindsay Mathieson wrote:
> On 27/11/2020 12:46 am, Thomas Lamprecht wrote:
>> Note, it's always good idea to check if all services are running OK again before
>> continuing with upgrading the next host, not just on this update:-)
>>
>> Also, ceph monitors can be nicely restarted over the web interface, there's a
>> visible status about which services run outdated versions/need a restart.
>>
>>
>> Anyway, do you have any logs which could give more details for possible issues?
> 
> I have a node that is just failing to rejoin the cluster and the ceph mon & mgr fail to start.
> 
> 
> Seeing this repeated in syslog
> 
>    Nov 27 00:58:23 vnh pveproxy[2903]: /etc/pve/local/pve-ssl.key:
>    failed to load local private key (key_file or key) at
>    /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1737.
>    Nov 27 00:58:23 vnh pveproxy[2904]: /etc/pve/local/pve-ssl.key:
>    failed to load local private key (key_file or key) at
>    /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1737.
>    Nov 27 00:58:23 vnh pveproxy[2905]: /etc/pve/local/pve-ssl.key:
>    failed to load local private key (key_file or key) at
>    /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1737.
>    Nov 27 00:58:26 vnh ceph-mon[2073]: 2020-11-27 00:58:26.378
>    7fb182935700 -1 mon.vnh@0(probing) e9 handle_auth_bad_method hmm,
>    they didn't like 2 result (95) Operation not supported
>    Nov 27 00:58:26 vnh ceph-mon[2073]: 2020-11-27 00:58:26.390
>    7fb17d92b700 -1 mon.vnh@0(probing) e9 handle_auth_bad_method hmm,
>    they didn't like 2 result (95) Operation not supported
>    Nov 27 00:58:26 vnh ceph-mon[2073]: 2020-11-27 00:58:26.526
>    7fb183136700 -1 mon.vnh@0(probing) e9 handle_auth_bad_method hmm,
>    they didn't like 2 result (95) Operation not supported
>    Nov 27 00:58:27 vnh ceph-mon[2073]: 2020-11-27 00:58:27.702
>    7fb182935700 -1 mon.vnh@0(probing) e9 handle_auth_request no
>    AuthAuthorizeHandler found for auth method 1
> 

the errors seems like being the result of pve-cluster not coming up,
which seems the actual problem.

> 
> The following gets the node back on the cluster:
> 
> systemctl start pve-cluster.service

Anything of pve-cluster service in the log?


What does:
# systemd-analyze verify default.target

outputs?

cheers,
Thomas