From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 3A86F1FF0A3 for ; Thu, 01 Oct 2026 09:31:27 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 4E4552163B; Thu, 01 Oct 2026 09:31:24 +0200 (CEST) Date: Thu, 1 Oct 2026 09:31:18 +0200 From: Wolfgang Bumiller To: Hannes Laimer Subject: Re: [RFC cluster/manager 00/10] pmxcfs: add a change notification socket Message-ID: References: <20260918144152.575163-1-h.laimer@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260918144152.575163-1-h.laimer@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1790839878497 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.526 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 5QHKNPUMFCR2GWLYUMZYH54WENQLBRR5 X-Message-ID-Hash: 5QHKNPUMFCR2GWLYUMZYH54WENQLBRR5 X-MailFrom: w.bumiller@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: pve-devel@lists.proxmox.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Fri, Sep 18, 2026 at 04:41:42PM +0200, Hannes Laimer wrote: > Every daemon that cares about /etc/pve learns about changes by calling > cfs_update on each loop iteration and comparing the version vector it > gets over the libqb IPC. inotify cannot replace that, since remote > changes arrive through corosync and never touch the local VFS, so > pvestatd, pve-firewall, the HA daemons and pvescheduler all wake up on > timers and re-read what usually has not changed. > > This series adds a push path. pmxcfs gets a Unix stream socket at > /run/pve-cluster/pmxcfs.sock, served by a Rust thread linked into the C > daemon as a static library with a small C ABI. A client subscribes with > named path templates such as nodes/{node}/qemu-server/{vmid}.conf and > receives one JSON line per matching mutation, carrying the memdb version > as sequence number, the event type, the path and the values the > placeholders captured. Connections authorized by group membership never > see private paths, as on the IPC and FUSE side. > > The daemon keeps the last mutations in a fixed size ring and each > connection a cursor into it, so the mutating thread only appends and > never waits for a client. A slow client is caught up from the ring, one > that fell off it gets a single resync event, and a reconnecting client > resumes at the last sequence number it saw, also across a restart of > pmxcfs when nothing changed meanwhile. A node that takes the whole > state from the cluster after a membership change hands its clients a > resync event, since no sequence of mutations describes that. A panic in So a high level question: Do we actually have any use case where this particular way processing changes on the *client* side would give us any *real* benefit? Given the complexity and custom non-blocking handling if socket states, I'd like to see a real need for this. Why isn't a simple async-based server with no sequence numbers where a time-out just disconnects the client, and the disconnect itself being the equivalent of a resync event, sufficient? The server would be just the accept loop which `spawn()`s a client-handler with its socket and event channel, the send-side of the channel pushed to the list of client channels. The client handling task just loops through a channel.recv() followed by a socket.send() with a timeout. On timeout it just returns, the channel gets closed, the next event-emit sees the channel.send() fail and removes the channel.