From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 7DDFD1FF0C0 for ; Wed, 18 Nov 2026 05:38:30 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 65CCF21661; Wed, 18 Nov 2026 05:38:27 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Subject: [PATCH cluster v3 04/38] pmxcfs: quorum: pass the corosync membership to an optional listener Date: Wed, 18 Nov 2026 05:37:16 +0100 Message-ID: <20261118043817.1659212-5-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261118043817.1659212-1-t.lamprecht@proxmox.com> References: <20261118043817.1659212-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794976701783 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.191 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: ZSUPGK3CVCBW6ZHG7RJEBOABBIGFZ45Q X-Message-ID-Hash: ZSUPGK3CVCBW6ZHG7RJEBOABBIGFZ45Q X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Quorum alone does not tell a service whether every node contributing to it has participated in the service's synchronization. Pass the node IDs from each corosync quorum notification to an optional listener, and an empty list when the connection to corosync is lost. The listener runs on the loop thread, like the dfsm callbacks, so it can compare corosync membership with dfsm group membership without locking. Existing behavior is unchanged when no listener is registered. Signed-off-by: Thomas Lamprecht --- changes v2 -> v3: * comment: the array is only valid during the call no changes from v1 to v2 src/pmxcfs/pmxcfs.c | 2 +- src/pmxcfs/quorum.c | 26 +++++++++++++++++++++----- src/pmxcfs/quorum.h | 9 ++++++++- 3 files changed, 30 insertions(+), 7 deletions(-) diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c index 2bf9145..284bae1 100644 --- a/src/pmxcfs/pmxcfs.c +++ b/src/pmxcfs/pmxcfs.c @@ -1019,7 +1019,7 @@ int main(int argc, char *argv[]) { if (dcdb) { - service_quorum = service_quorum_new(); + service_quorum = service_quorum_new(NULL, NULL); cfs_loop_add_service(corosync_loop, service_quorum, QB_LOOP_HIGH); diff --git a/src/pmxcfs/quorum.c b/src/pmxcfs/quorum.c index 0ac3cfe..3b9e012 100644 --- a/src/pmxcfs/quorum.c +++ b/src/pmxcfs/quorum.c @@ -35,12 +35,25 @@ #include "cfs-utils.h" #include "loop.h" +#include "quorum.h" #include "status.h" typedef struct { quorum_handle_t handle; + cfs_quorum_view_fn_t view_fn; + gpointer view_data; } qs_private_t; +static void quorum_set( + qs_private_t *private, uint32_t quorate, gboolean quiet, const uint32_t *nodeids, size_t count +) { + cfs_set_quorate(quorate, quiet); + + if (private->view_fn) { + private->view_fn(private->view_data, quorate, nodeids, count); + } +} + static void quorum_notification_fn( quorum_handle_t handle, uint32_t quorate, @@ -64,7 +77,7 @@ static void quorum_notification_fn( return; } - cfs_set_quorate(quorate, FALSE); + quorum_set(private, quorate, FALSE, view_list, view_list_entries); } static quorum_callbacks_t quorum_callbacks = { @@ -80,7 +93,7 @@ static gboolean service_quorum_finalize(cfs_service_t *service, gpointer context cs_error_t result; - cfs_set_quorate(0, TRUE); + quorum_set(private, 0, TRUE, NULL, 0); result = quorum_finalize(handle); private->handle = 0; @@ -150,7 +163,7 @@ static int service_quorum_initialize(cfs_service_t *service, gpointer context) { return quorum_fd; err_finalize: - cfs_set_quorate(0, FALSE); + quorum_set(private, 0, FALSE, NULL, 0); quorum_finalize(handle); err_reset_handle: private @@ -185,7 +198,7 @@ loop: cfs_critical("quorum_dispatch failed: %s", cs_strerror(result)); - cfs_set_quorate(0, FALSE); + quorum_set(private, 0, FALSE, NULL, 0); quorum_finalize(handle); private->handle = 0; return FALSE; @@ -197,7 +210,7 @@ static cfs_service_callbacks_t cfs_quorum_callbacks = { .cfs_service_dispatch_fn = service_quorum_dispatch, }; -cfs_service_t *service_quorum_new(void) { +cfs_service_t *service_quorum_new(cfs_quorum_view_fn_t view_fn, gpointer view_data) { cfs_service_t *service; qs_private_t *private = g_new0(qs_private_t, 1); @@ -205,6 +218,9 @@ cfs_service_t *service_quorum_new(void) { return NULL; } + private->view_fn = view_fn; + private->view_data = view_data; + service = cfs_service_new(&cfs_quorum_callbacks, G_LOG_DOMAIN, private); return service; diff --git a/src/pmxcfs/quorum.h b/src/pmxcfs/quorum.h index 3dec79c..06a3246 100644 --- a/src/pmxcfs/quorum.h +++ b/src/pmxcfs/quorum.h @@ -25,7 +25,14 @@ #include "loop.h" -cfs_service_t *service_quorum_new(void); +/* Called on the loop thread with the node IDs of the current corosync membership each time corosync + * reports a change, and with none when the connection to corosync is lost. The array is only valid + * during the call. */ +typedef void (*cfs_quorum_view_fn_t)( + gpointer data, uint32_t quorate, const uint32_t *nodeids, size_t count +); + +cfs_service_t *service_quorum_new(cfs_quorum_view_fn_t view_fn, gpointer view_data); void service_quorum_destroy(cfs_service_t *service); -- 2.47.3