From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 995E01FF184 for ; Thu, 20 Nov 2025 12:29:14 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8FD6C4AA7; Thu, 20 Nov 2025 12:29:14 +0100 (CET) Date: Thu, 20 Nov 2025 12:29:10 +0100 To: pve-user@lists.proxmox.com MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE user list List-Post: From: Rainer Krienke via pve-user Precedence: list Cc: Rainer Krienke X-Mailman-Version: 2.1.29 X-BeenThere: pve-user@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE user list List-Help: Subject: [PVE-User] Keyring problem for external rbds reported by pve8to9 check script Content-Type: multipart/mixed; boundary="===============3598925561395355474==" Errors-To: pve-user-bounces@lists.proxmox.com Sender: "pve-user" --===============3598925561395355474== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-user@lists.proxmox.com Delivered-To: pve-user@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 32566E274B for ; Thu, 20 Nov 2025 12:29:13 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 123A34AF0 for ; Thu, 20 Nov 2025 12:29:13 +0100 (CET) Received: from mout1.uni-koblenz.de (mout1.uni-koblenz.de [IPv6:2001:4c80:50::868]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Thu, 20 Nov 2025 12:29:11 +0100 (CET) Received: from mailhub.uni-koblenz.de (mailhub.uni-koblenz.de [141.26.64.104]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by mout1.uni-koblenz.de (Postfix) with ESMTPS id 4dBx2H1xpKzCrqc for ; Thu, 20 Nov 2025 12:29:11 +0100 (CET) Received: from mailproxy.uni-koblenz.de (mailproxy.uni-koblenz.de [141.26.64.230]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by mailhub.uni-koblenz.de (Postfix) with ESMTPS id 4dBx2H0wvNz3wfK for ; Thu, 20 Nov 2025 12:29:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uni-koblenz.de; s=dkim0001; t=1763638151; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:autocrypt:autocrypt; bh=BFYk00c2iYon9FqD3zRwlgp0dg9a5m6NI+xJCOHxMuE=; b=kxaSTFbGiQJUUyJE7iAAUSvwqt0p8ICLAZs9NdBss+nQiF+UNjmStSRWZ3MUP9ZGCb80Eg R0hCvxgXoqHFit0lxUo+mAVBCqW+xSbtCcGLwtZq4RJ8EINhZLvI8QpNQHIn+PjD+nl6f9 sPqoqt+UBf+e98KPHK6/66ZEipDhh7PoPz3Karxd75UPeFY6vfL4xw1Ematsnq5SLp+nEQ rdZvhSGNoNKzE2FmfAJKx0Ph2gg5L5+VTAo/ob8hogKONhQVAb2LoK+o9SrEJ642t3FaWD 8D4+hg+TvKH0h8ZE9dP7YelEVjQRRJWpL0xZIrwVyHE97YK5JPUr2QiKk4XkzsnuAokhb8 B+Hjx3qIaVOxqC+FjOaePCGbNK41fnnLMXcKlxemBJSoRxgyuPPIhfLWqXqiN1WLX0QrH2 IeTcSu4n0GFcVJndt4a4lG+z6zqu1uHDwh+6cbJNi3lw8hRS9YoJulhy6aBG9kCv0Np/25 jnCMvrjHMsWDQv818Zv3gZiSmJ4U9whyL5esMvy1lQHc12CafSJeHqS7btfRI3vg+3PHA/ wRQBAVS3qseDC7g4+KUSoR01KvdAPr+Kl9ssXXXEfYOZjATuiL8/VIJ/CRzQcoai61fx4W 7XYx+vmB0PCLm5pK2vRz4m/6ewLZgtOfOaxnY7fcBnI96HjAe+uYY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=uni-koblenz.de; s=dkim0001; t=1763638151; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:autocrypt:autocrypt; bh=BFYk00c2iYon9FqD3zRwlgp0dg9a5m6NI+xJCOHxMuE=; b=kY6Ll9Nn3qLQxpbrqTwtHgmdjyqca6xviHwGIpj9DD9PElFbYoK0V3WjQa35LgYTYDFEI4 eKlLLWDnj2KONG2W+u7MTafSCqWK60oOqPK8jhDUUd4kZV4SBppDV/ZhjcSFhikmM/Lb1K oLRRYG/nYRwMsyDWvOhM2Bll6grxK9S1fivWRG4qM9fVA8x7G4vboXRcSqtlv3MrVXDh88 qdvu4XAFJ5scBVrFB8IWdEBXTzz6u2vW7fFmVMhJxuJx580A7e8K34js5RftC5ssfg3mA5 SYGSghmswTuLA71YwO9ecGXpKJ6SeN0pKUc/sdxEWTyglCnIBqVwQ3geoebUB/vVUgVrK6 fjR9tqkwXZ0HeWrZ/EWd+12alK3le9OVX9NL6JXVMumRyzKBA2UiVLtbiFutAMXE9SBVF4 8f9ktmtl23GknEc0Te6eiS+o9imsQF73VKe605HaAmwR96vShUxkdCY38GeIO5ntwsFiTK KsbRzrc7JlBrrOstzFiZqJHYR0kNvDZ67/yElPs/yCDGaMSDjfmLWoXToxgPTtFc6SS2r2 L7BZ/jblfSjWLVx7rrwvq8dT5XiD0HyoS95atwWeXKbti0fhlehN6RArehmaM6FGpC3nt2 2wf5qaeg6fq2mEAnZaJhIWH+qvoCYIG/oawLx9Xh9aUWTqAaXHRPs= ARC-Seal: i=1; s=dkim0001; d=uni-koblenz.de; t=1763638151; a=rsa-sha256; cv=none; b=QRhEBUhhhc2ELsJqB+kCxjGiOOB6fndXfI4BGEwwklJriI7EW3CopGsLj6rsPnlA+xkFUS 1j7oTJCXDvwIzcHfgNiQx7UMP2/SPQc9LQoiiTCMtGX1H9BrKRvj9wAIqaNe7e0vQkUbA8 8rhnx1yfoDarKPNE30G+ealGu1Et8dHxWyOBhIwX5K3T9a6CPBy1tRnc3lMEJHBkqU5XGD 3RYgIfSIHvue/GcpnxHu4Hfv4xaNQ+O6jYvrq5P8DSaEXnMWxPcpDAm+a6kYB8lP8p53Hd PpVTUU1uMwKzZ5Z6dCMoVPfzwKMTYmqIm75nH8aD0jOy3aR0uHu8ddfKtE/f8H6IHWurQF DY8cu7XIn7kSEdpq0i02b5nWgMEMW51XVaIaGySLR3oAeGEYTjzjdmH01PdORSsVh+Wdb/ FybMyzbTI3GuoQpEN8aXt1IBiJc2YmzbfCsqOYme+FwxPQLCxWumP0DHi1GVJp57fgyms8 HMY/yYrH3yN5escRnqDIYzz0KfWhqH06MTH/veuRAbisAM2fXRoVHJ+toYynnZZfpodD/E PG5UFfFZUL6EhPOn54ui02qNjnBQ0s7sZZWwVxIeRM6C4/k04is0B4NTk2KX1KgAkbY4kn SJ9eBrzceAyQYNcf8T7O9jpUL7n6o12sBmiOXHR/g4FKq2TR0jzGY= ARC-Authentication-Results: i=1; incoming_from_mx; dkim=none Received: from [IPV6:2001:4c80:50:127::506] (unknown [IPv6:2001:4c80:50:127::506]) by mailproxy.uni-koblenz.de (Postfix) with ESMTPSA id 02EE7100642 for ; Thu, 20 Nov 2025 12:29:10 +0100 (CET) Message-ID: Date: Thu, 20 Nov 2025 12:29:10 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: de-DE To: pve-user@lists.proxmox.com From: Rainer Krienke Subject: Keyring problem for external rbds reported by pve8to9 check script Autocrypt: addr=krienke@uni-koblenz.de; keydata= xsDiBDyQof8RBADHWiT+yp96Y5DwyFrbRKjuGK0wuhcwGuMYlLMBz+tiTpFxqYn1stFpUfRA fvGTFb9jDYo66z8kOX3NU/5zEBjMK7UgSRBKCXXdDfvL+l2ywyr8wmVNKjAJBjX+GqhNqR/n lr/4RGvRxqQ9blzTYst5EobNkDVfZAzOQSwWTCkRowCg/+jb6lX9hSXJ5MbNtTJkO8NmMQ8D /ihdAVOOGUZJyIQAz6yk1NaRg/ntSoX3j3YT/t4J7zx6RXU7hOgTE4xWacz7pata2KPFc9iF dh7HI5cGc9E64UNXKlcFpAuxBjLEYcExLMhAfMfnng46XL+BBUI3Dk6jF9zSEiMlDqXggWwW bb8IL/3Rsh/O+W0kBZh2qQ6gdwSLA/9LBQtywVB3mc66GVlyarR7fYNj9C18gXSG1XLfXWty ip2VLD2zajc6po4aaGky7KbWZDXBqxcijvFaIjvASLpk41GRKw4ymhaeolqg1oOZYlanyI3S audi3kgGzfnwcriid6LS2vFStZLg2JLrjm8hUt79eRCSl5jg7YPGzdZbPM0nUmFpbmVyIEty aWVua2UgPGtyaWVua2VAdW5pLWtvYmxlbnouZGU+wmgEEBECACAFAlsneTUHCwkIBwMCAQYV CAIJCgsEFgIDAQIeAQIXgAASB2VHUEcAAQEJEGpXbY3PygxKew4AoMWCLv9hZqQ9CfDMUxLW sCgGZLy3AJsFyvbApc7SCNs5/PxPWFZRSlT+pM7BTQRTtnEqARAAufjdJ6QFsHXP1Nbj42EZ CSN/C9SuFIvXwwAMSB9giu/12PtXzDTe8VDfI7Re1I33hm6LnA1HqD2uJi4xcecVGEaHVoJe BE6A6MhvEArCu9T38JQbg6vSubsVYQtcvZO4mGef6F6q1212d7kv4lgBaTHX2RIMvQbdnbSP vHytxGz2tKmcN0nfMYRHvhECD9M+nk0WmoeOOneAedPESL9rNpDS8jQQVyorajh2t42my9wj mAxn1QF4fDnUGmAuN/GqHwCgCapL2YgmMMxncJkQ7aCXpN9cOG/pQn/AkYmIoPTfQ6EvU8Ct ufC2FExq6Hg6xgXZlcmQ36+MUWNeUbL5czkChiu5AKpKX310fBNsiYeRIpiohc5MrzIQz153 RuGPpY7OAztEJMpXu5vFoR2w4sGueag/YzoMgp0aTOBKpflBMTaJqT44l895Meds9KPXqHmV KcieuDFxOREyrk0+zCUj+MettgJwkwr5pRo9kqiFh+xo+CtEnxWeUPFW3w5cPdc4douXsN3O QePv92saM5Ua4C5DvYLckTgRMWqq0qhbvEU9c4E7bAvEl7zBBOCG7+sq3wYwSMo2XVIsbklQ gClVnsuNCCRqBqYRFHqVKbWFyE9EHbP9s3+hIhzbTGeW0EZUhvZUZBQsZJ/6X1OhXiUHBCmd 7thZEQsyaK6ROrEAEQEAAcJJBBgRAgAJBQJTtnEqAhsMAAoJEGpXbY3PygxKM0MAn2koBjUN AWdbknKpXf2zAWC7+EthAKD0g7enhxSa+gfN5CWm9OMBHf4bog== Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4dBx2H1xpKzCrqc X-Rspamd-Server: rspamd X-Spamd-Result: default: False [-0.20 / 16.00]; R_DKIM_ALLOW(-0.20)[uni-koblenz.de:s=dkim0001]; OUTGOING_NO_ACTION(0.00)[]; OUTGOING(0.00)[]; DKIM_TRACE(0.00)[uni-koblenz.de:+] X-Rspamd-Action: no action X-SPAM-LEVEL: Spam detection results: 0 ARC_SIGNED 0.001 Message has a ARC signature ARC_VALID 0.001 Message has a valid ARC signature AWL -0.568 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [uni-koblenz.de] Hello, sorry, I need some more lines to describe the problem :-; I have a "PveCeph" cluster and there is a second PVE cluster "PVEnoceph" which is using external rbds located on "PveCeph" for vm storage. So all VMs (~80) are running on "PVEnoceph" cluster because it has more RAM and CPU Power but storage is located and provided by the cluster PveCeph. These two clusters are running fine with pve8.4, no problems. However running the pve9 check script /usr/bin/pve8to9 it shows an error: FAIL: The Ceph configuration of the following externally managed RBD storages needs to be updated: rbd_1 and rbd_2 It also offers a script to fix this. But I am reluctant, since I would like to know first what is wrong now and how it can be fixed without breaking the running clusters? The current external rbd-configuration on "PVEnoceph" is as follows: In PVEnoceph:/etc/pve/storage.cfg there are two rbd declarations: rbd: rbd_1 content images data-pool PveCephPool-data krbd 0 monhost pool PveCephPool-metadata username admin rbd: rbd_2 ... And in PVEnoceph:/etc/pve/priv/ceph/ there are two keyring files named rbd_1.keyring and rbd_2.keyring which currently contain the client.admin Keyring of the PveCeph cluster: [client.admin] key = 4711abcdef12345ABCEFxy As said this config is running since pve 6.x without any trouble, but pve9 does not like the external rbds rbd_1 and rbd_2 and I have no idea why? Any ideas? Thanks Rainer --===============3598925561395355474== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-user mailing list pve-user@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user --===============3598925561395355474==--