From: Kefu Chai <k.chai@proxmox.com>
To: m.carrara@proxmox.com
Cc: pve-devel@lists.proxmox.com
Subject: Re: [PATCH ceph master/ceph squid/manager v4 0/7] Fix #6816: Prevent ceph-exporter Daemon from Crashing on Startup
Date: Mon, 16 Nov 2026 19:15:26 +0800 [thread overview]
Message-ID: <20261116111526.1877842-1-k.chai@proxmox.com> (raw)
In-Reply-To: <20261116091839.1804896-1-k.chai@proxmox.com>
Hi Max,
first a correction to my review, sorry: I claimed the mon 'allow r'
cap is mostly a formality because the exporter's own code has no
MonClient, and that was wrong. I hadn't tested it. global_init()
runs the mon config bootstrap for the exporter anyway: as a
CODE_ENVIRONMENT_DAEMON without --no-mon-config it fetches the
monmap and config from the mons at startup, and that mini session
does authenticate with the keyring. So the cap is real, and your
bug report was spot on. Started by systemd as root, the exporter
reads ceph.conf before dropping its privileges, so the keyring
really is the first thing it trips over.
Chasing my own mistake, I ran into the escape hatch that's printed
right in the error message: --no-mon-config. It's a generic CLI
flag, every ceph binary parses it, and with it the exporter needs
nothing from the cluster at startup, no mon connection, no CephX,
and no keyring. Its own options (sock dir, port, and so on) all
have compiled defaults and can be passed as arguments, and I can't
think of anything we'd want to tune via the mon config for it.
Quick test on our lab, as a service: the stock unit crash loops as
described. A drop-in that only appends --no-mon-config to ExecStart,
nothing else, and the service comes up as plain ceph:ceph and
happily serves the local daemon metrics on port 9926.
If you agree this is the better route, that one word could replace
a good chunk of the series: no client.exporter, no ceph.conf
section for it, no pve-ceph-keyring call in postinst, and even the
group change becomes optional (it just keeps a config warning out
of the journal). Not trying to talk you out of the keyring approach
though, that part of the series is well built either way.
One more thing the flag buys us: I also started the exporter with a
readable config and a valid keyring but a dead mon address, and it
just sat in the bootstrap for the four minutes I gave it, printing
nothing, serving nothing. With the flag it starts serving right
away. A mon outage at boot time would otherwise leave you without
node metrics, and those are exactly the moments I'd want them for.
The one downside I can see: options set with 'ceph config set' no
longer reach the exporter. I don't think we use that, and cephadm
style deployments could keep the old behavior by not passing the
flag. Maybe upstream could make the bootstrap best effort for this
daemon some day, but that feels like a separate discussion.
The pve-init-ceph-crash cleanup is unrelated to all this and worth
keeping either way.
Happy to test a v5 either way!
prev parent reply other threads:[~2026-11-16 11:15 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-23 13:44 [PATCH ceph master/ceph squid/manager v4 0/7] Fix #6816: Prevent ceph-exporter Daemon from Crashing on Startup Max R. Carrara
2026-06-23 13:44 ` [PATCH pve-manager v4 1/7] ceph: tools: add helper sub for creating or updating keyring files Max R. Carrara
2026-06-23 13:44 ` [PATCH pve-manager v4 2/7] fix #6816: api: ceph: create 'client.exporter' w/ keyring Max R. Carrara
2026-06-23 13:44 ` [PATCH pve-manager v4 3/7] fix #6816: bin: add pve-ceph-keyring helper and call it in postinst Max R. Carrara
2026-06-23 13:44 ` [PATCH pve-manager v4 4/7] ceph: tools: simplify helper sub for crash keyring file Max R. Carrara
2026-06-23 13:44 ` [PATCH pve-manager v4 5/7] bin: make pve-init-ceph-crash call pve-ceph-keyring Max R. Carrara
2026-06-23 13:44 ` [PATCH ceph master v4 6/7] fix #6816: patches: make ceph-exporter use custom keyring Max R. Carrara
2026-06-23 13:44 ` [PATCH ceph squid v4 7/7] " Max R. Carrara
2026-11-16 9:18 ` [PATCH ceph master/ceph squid/manager v4 0/7] Fix #6816: Prevent ceph-exporter Daemon from Crashing on Startup Kefu Chai
2026-11-16 11:15 ` Kefu Chai [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116111526.1877842-1-k.chai@proxmox.com \
--to=k.chai@proxmox.com \
--cc=m.carrara@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox