Re: [RFC cluster/manager 00/10] pmxcfs: add a change notification socket

[Wolfgang Bumiller <w.bumiller@proxmox.com>]

On Fri, Sep 18, 2026 at 04:41:42PM +0200, Hannes Laimer wrote:
> Every daemon that cares about /etc/pve learns about changes by calling
> cfs_update on each loop iteration and comparing the version vector it
> gets over the libqb IPC. inotify cannot replace that, since remote
> changes arrive through corosync and never touch the local VFS, so
> pvestatd, pve-firewall, the HA daemons and pvescheduler all wake up on
> timers and re-read what usually has not changed.
> 
> This series adds a push path. pmxcfs gets a Unix stream socket at
> /run/pve-cluster/pmxcfs.sock, served by a Rust thread linked into the C
> daemon as a static library with a small C ABI. A client subscribes with
> named path templates such as nodes/{node}/qemu-server/{vmid}.conf and
> receives one JSON line per matching mutation, carrying the memdb version
> as sequence number, the event type, the path and the values the
> placeholders captured. Connections authorized by group membership never
> see private paths, as on the IPC and FUSE side.
> 
> The daemon keeps the last mutations in a fixed size ring and each
> connection a cursor into it, so the mutating thread only appends and
> never waits for a client. A slow client is caught up from the ring, one
> that fell off it gets a single resync event, and a reconnecting client
> resumes at the last sequence number it saw, also across a restart of
> pmxcfs when nothing changed meanwhile. A node that takes the whole
> state from the cluster after a membership change hands its clients a
> resync event, since no sequence of mutations describes that. A panic in

So a high level question:

Do we actually have any use case where this particular way processing
changes on the *client* side would give us any *real* benefit?

Given the complexity and custom non-blocking handling if socket states,
I'd like to see a real need for this.

Why isn't a simple async-based server with no sequence numbers where a
time-out just disconnects the client, and the disconnect itself being
the equivalent of a resync event, sufficient?

The server would be just the accept loop which `spawn()`s a
client-handler with its socket and event channel, the send-side of the
channel pushed to the list of client channels. The client handling task
just loops through a channel.recv() followed by a socket.send() with a
timeout. On timeout it just returns, the channel gets closed, the next
event-emit sees the channel.send() fail and removes the channel.