From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 8AD481FF0DA for ; Thu, 19 Nov 2026 20:19:33 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id E2899216FE; Thu, 19 Nov 2026 20:19:23 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Subject: [PATCH cluster v5 04/41] pmxcfs: quorum: record the corosync membership Date: Thu, 19 Nov 2026 20:18:36 +0100 Message-ID: <20261119191914.2336934-5-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261119191914.2336934-1-t.lamprecht@proxmox.com> References: <20261119191914.2336934-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1795115959291 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.158 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: IPNQ2CU6VKGE2ABO3X4BJ2FWTS7TDGQD X-Message-ID-Hash: IPNQ2CU6VKGE2ABO3X4BJ2FWTS7TDGQD X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: That the cluster is quorate does not tell a service whether all nodes of the corosync membership have also taken part in the synchronization of that service. To check this, the service needs to know which nodes are in the membership. Record the node IDs from each quorum notification of corosync next to the quorate flag, and clear them when the connection to corosync is lost. A reader gets a copy of the list. The loop thread can then compare the corosync membership with the members of a dfsm group without holding the status lock. Nothing reads the list yet. Signed-off-by: Thomas Lamprecht Reviewed-by: Dominik Csapak Tested-by: Dominik Csapak --- no changes since v4 changes v3 -> v4: * record the membership in the status, next to the quorate flag, instead of passing it to a listener: the cluster ID check reads it each time it runs and needs no notification; subject changed accordingly * commit message: plainer wording, same content changes v2 -> v3: * comment: the array is only valid during the call src/pmxcfs/quorum.c | 13 +++++++++---- src/pmxcfs/status.c | 23 +++++++++++++++++++++++ src/pmxcfs/status.h | 7 +++++++ 3 files changed, 39 insertions(+), 4 deletions(-) diff --git a/src/pmxcfs/quorum.c b/src/pmxcfs/quorum.c index 0ac3cfe..7d87414 100644 --- a/src/pmxcfs/quorum.c +++ b/src/pmxcfs/quorum.c @@ -41,6 +41,11 @@ typedef struct { quorum_handle_t handle; } qs_private_t; +static void quorum_set(uint32_t quorate, gboolean quiet, const uint32_t *nodeids, size_t count) { + cfs_set_quorate(quorate, quiet); + cfs_set_quorum_members(nodeids, count); +} + static void quorum_notification_fn( quorum_handle_t handle, uint32_t quorate, @@ -64,7 +69,7 @@ static void quorum_notification_fn( return; } - cfs_set_quorate(quorate, FALSE); + quorum_set(quorate, FALSE, view_list, view_list_entries); } static quorum_callbacks_t quorum_callbacks = { @@ -80,7 +85,7 @@ static gboolean service_quorum_finalize(cfs_service_t *service, gpointer context cs_error_t result; - cfs_set_quorate(0, TRUE); + quorum_set(0, TRUE, NULL, 0); result = quorum_finalize(handle); private->handle = 0; @@ -150,7 +155,7 @@ static int service_quorum_initialize(cfs_service_t *service, gpointer context) { return quorum_fd; err_finalize: - cfs_set_quorate(0, FALSE); + quorum_set(0, FALSE, NULL, 0); quorum_finalize(handle); err_reset_handle: private @@ -185,7 +190,7 @@ loop: cfs_critical("quorum_dispatch failed: %s", cs_strerror(result)); - cfs_set_quorate(0, FALSE); + quorum_set(0, FALSE, NULL, 0); quorum_finalize(handle); private->handle = 0; return FALSE; diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c index 8538d8b..bbfa88e 100644 --- a/src/pmxcfs/status.c +++ b/src/pmxcfs/status.c @@ -132,6 +132,8 @@ typedef struct { time_t start_time; uint32_t quorate; + uint32_t *quorum_members; + size_t quorum_members_len; cfs_clinfo_t *clinfo; uint32_t clinfo_version; @@ -2000,6 +2002,27 @@ gboolean cfs_is_quorate(void) { return res; } +void cfs_set_quorum_members(const uint32_t *nodeids, size_t count) { + g_return_if_fail(!count || nodeids != NULL); + + g_mutex_lock(&mutex); + g_free(cfs_status.quorum_members); + cfs_status.quorum_members = count ? g_memdup2(nodeids, count * sizeof(*nodeids)) : NULL; + cfs_status.quorum_members_len = count; + g_mutex_unlock(&mutex); +} + +uint32_t *cfs_get_quorum_members(size_t *count) { + g_return_val_if_fail(count != NULL, NULL); + + g_mutex_lock(&mutex); + *count = cfs_status.quorum_members_len; + uint32_t *res = g_memdup2(cfs_status.quorum_members, *count * sizeof(*res)); + g_mutex_unlock(&mutex); + + return res; +} + void cfs_set_quorate(uint32_t quorate, gboolean quiet) { g_mutex_lock(&mutex); diff --git a/src/pmxcfs/status.h b/src/pmxcfs/status.h index 709ef64..a757326 100644 --- a/src/pmxcfs/status.h +++ b/src/pmxcfs/status.h @@ -69,6 +69,13 @@ void cfs_set_quorate(uint32_t quorate, gboolean quiet); gboolean cfs_is_quorate(void); +/* the node IDs of the corosync membership as of the last quorum notification, none while not + * connected to corosync */ +void cfs_set_quorum_members(const uint32_t *nodeids, size_t count); + +/* a copy of them, to free with g_free() */ +uint32_t *cfs_get_quorum_members(size_t *count); + GHashTable *vmlist_hash_new(void); gboolean vmlist_hash_insert_vm( -- 2.47.3