From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id A9EBE1FF0BC for ; Mon, 16 Nov 2026 12:15:39 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 619D7216FE; Mon, 16 Nov 2026 12:15:34 +0100 (CET) From: Kefu Chai To: m.carrara@proxmox.com Subject: Re: [PATCH ceph master/ceph squid/manager v4 0/7] Fix #6816: Prevent ceph-exporter Daemon from Crashing on Startup Date: Mon, 16 Nov 2026 19:15:26 +0800 Message-ID: <20261116111526.1877842-1-k.chai@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261116091839.1804896-1-k.chai@proxmox.com> References: <20260623134454.710441-1-m.carrara@proxmox.com> <20261116091839.1804896-1-k.chai@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794827729709 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.204 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: JU5MRORH3NER2RVERTSS3FEAEJFSASZR X-Message-ID-Hash: JU5MRORH3NER2RVERTSS3FEAEJFSASZR X-MailFrom: k.chai@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: pve-devel@lists.proxmox.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hi Max, first a correction to my review, sorry: I claimed the mon 'allow r' cap is mostly a formality because the exporter's own code has no MonClient, and that was wrong. I hadn't tested it. global_init() runs the mon config bootstrap for the exporter anyway: as a CODE_ENVIRONMENT_DAEMON without --no-mon-config it fetches the monmap and config from the mons at startup, and that mini session does authenticate with the keyring. So the cap is real, and your bug report was spot on. Started by systemd as root, the exporter reads ceph.conf before dropping its privileges, so the keyring really is the first thing it trips over. Chasing my own mistake, I ran into the escape hatch that's printed right in the error message: --no-mon-config. It's a generic CLI flag, every ceph binary parses it, and with it the exporter needs nothing from the cluster at startup, no mon connection, no CephX, and no keyring. Its own options (sock dir, port, and so on) all have compiled defaults and can be passed as arguments, and I can't think of anything we'd want to tune via the mon config for it. Quick test on our lab, as a service: the stock unit crash loops as described. A drop-in that only appends --no-mon-config to ExecStart, nothing else, and the service comes up as plain ceph:ceph and happily serves the local daemon metrics on port 9926. If you agree this is the better route, that one word could replace a good chunk of the series: no client.exporter, no ceph.conf section for it, no pve-ceph-keyring call in postinst, and even the group change becomes optional (it just keeps a config warning out of the journal). Not trying to talk you out of the keyring approach though, that part of the series is well built either way. One more thing the flag buys us: I also started the exporter with a readable config and a valid keyring but a dead mon address, and it just sat in the bootstrap for the four minutes I gave it, printing nothing, serving nothing. With the flag it starts serving right away. A mon outage at boot time would otherwise leave you without node metrics, and those are exactly the moments I'd want them for. The one downside I can see: options set with 'ceph config set' no longer reach the exporter. I don't think we use that, and cephadm style deployments could keep the old behavior by not passing the flag. Maybe upstream could make the bootstrap best effort for this daemon some day, but that feels like a separate discussion. The pve-init-ceph-crash cleanup is unrelated to all this and worth keeping either way. Happy to test a v5 either way!