From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id A634B1FF0AA for ; Tue, 17 Nov 2026 23:40:32 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id C52FB216FD; Tue, 17 Nov 2026 23:39:57 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Subject: [PATCH cluster v2 07/36] pmxcfs: allow removing the cluster ID file in forced local mode Date: Tue, 17 Nov 2026 23:34:17 +0100 Message-ID: <20261117223945.862914-8-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261117223945.862914-1-t.lamprecht@proxmox.com> References: <20261117223945.862914-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794955190824 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.201 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 4VQGXSIJIVAYOTLFCJF7WNV7FO5VKI5A X-Message-ID-Hash: 4VQGXSIJIVAYOTLFCJF7WNV7FO5VKI5A X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: A node that gets separated from its cluster without reinstalling, or a cloned installation, keeps the cluster ID it had. With the file protected against changes from user space, only editing the database by hand would get it a new one. Allow unlinking the file while pmxcfs runs in forced local mode (-l), which the documented procedure for separating a node already uses to remove the corosync.conf. The next start without a corosync.conf then generates a new ID, like on any standalone node. Moving the file away is allowed then too, as libfuse unlinks a file that is still open by renaming it to a hidden name. Outside of forced local mode both stay rejected, and nothing can be moved onto the path in any mode. An unlink done in local mode on a node that later rejoins its cluster replicates like any other change made in local mode: if the database of that node wins the sync, the file is gone on all nodes, and the cluster generates a new ID. Signed-off-by: Thomas Lamprecht --- changes v1 -> v2: * rebase only src/pmxcfs/cfs-plug-memdb.c | 8 ++++++-- src/pmxcfs/cfs-utils.h | 1 + src/pmxcfs/check_memdb.c | 32 ++++++++++++++++++++++++++++++++ src/pmxcfs/pmxcfs.c | 2 ++ 4 files changed, 41 insertions(+), 2 deletions(-) diff --git a/src/pmxcfs/cfs-plug-memdb.c b/src/pmxcfs/cfs-plug-memdb.c index 0d0fa64..217585e 100644 --- a/src/pmxcfs/cfs-plug-memdb.c +++ b/src/pmxcfs/cfs-plug-memdb.c @@ -387,7 +387,9 @@ static int cfs_plug_memdb_rename(cfs_plug_t *plug, const char *from, const char g_return_val_if_fail(from != NULL, PARAM_CHECK_ERRNO); g_return_val_if_fail(to != NULL, PARAM_CHECK_ERRNO); - if (path_is_cluster_id(from) || path_is_cluster_id(to)) { + /* moving it away is fine in forced local mode, like unlinking it, which libfuse does that way + * while the file is still open */ + if ((path_is_cluster_id(from) && !cfs.force_local_mode) || path_is_cluster_id(to)) { return -EACCES; } @@ -410,7 +412,9 @@ static int cfs_plug_memdb_unlink(cfs_plug_t *plug, const char *path) { g_return_val_if_fail(plug->ops == &cfs_ops, PARAM_CHECK_ERRNO); g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO); - if (path_is_cluster_id(path)) { + /* the way to give a node that got separated from its cluster an ID of its own, generated by its + * next start without a corosync.conf */ + if (path_is_cluster_id(path) && !cfs.force_local_mode) { return -EACCES; } diff --git a/src/pmxcfs/cfs-utils.h b/src/pmxcfs/cfs-utils.h index f66e944..5016720 100644 --- a/src/pmxcfs/cfs-utils.h +++ b/src/pmxcfs/cfs-utils.h @@ -44,6 +44,7 @@ typedef struct { char *ip; gid_t gid; int debug; + gboolean force_local_mode; } cfs_t; extern cfs_t cfs; diff --git a/src/pmxcfs/check_memdb.c b/src/pmxcfs/check_memdb.c index db85cbd..86b78a4 100644 --- a/src/pmxcfs/check_memdb.c +++ b/src/pmxcfs/check_memdb.c @@ -457,6 +457,36 @@ START_TEST(test_cluster_id_standalone_write_failure) { } END_TEST +START_TEST(test_cluster_id_unlink_forced_local_mode) { + cfs_plug_t *plug = (cfs_plug_t *)cfs_plug_memdb_new("memdb", memdb, NULL); + ck_assert(plug != NULL); + + ck_assert(memdb_create(memdb, "cluster-id", 0, 1234) == 0); + ck_assert(memdb_write(memdb, "cluster-id", 0, 1234, "abcdefghijklmnopqrst\n", 21, 0, 1) == 21); + + ck_assert(memdb_create(memdb, "other", 0, 1234) == 0); + + ck_assert(plug->ops->unlink(plug, "cluster-id") == -EACCES); + ck_assert(plug->ops->rename(plug, "cluster-id", ".fuse_hidden01") == -EACCES); + + cfs.force_local_mode = TRUE; + ck_assert(plug->ops->truncate(plug, "cluster-id", 0) == -EACCES); + ck_assert(plug->ops->rename(plug, "other", "cluster-id") == -EACCES); + + /* what libfuse does on unlink while the file is open */ + ck_assert(plug->ops->rename(plug, "cluster-id", ".fuse_hidden01") == 0); + ck_assert(memdb_getattr(memdb, "cluster-id") == NULL); + + ck_assert(memdb_create(memdb, "cluster-id", 0, 1234) == 0); + ck_assert(plug->ops->unlink(plug, "cluster-id") == 0); + ck_assert(memdb_getattr(memdb, "cluster-id") == NULL); + + cfs.force_local_mode = FALSE; + + plug->destroy_plug(plug); +} +END_TEST + START_TEST(test_truncate_grow) { const char *fn = "/f1"; time_t ctime = 1234; @@ -745,6 +775,8 @@ static Suite *memdb_suite(void) { add_test(s, test_cluster_id_fuse_read_only, "cluster_id_fuse_read_only"); + add_test(s, test_cluster_id_unlink_forced_local_mode, "cluster_id_unlink_forced_local_mode"); + add_test(s, test_cluster_id_is_valid, "cluster_id_is_valid"); add_test(s, test_cluster_id_generate, "cluster_id_generate"); diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c index 09c1b28..4c9667b 100644 --- a/src/pmxcfs/pmxcfs.c +++ b/src/pmxcfs/pmxcfs.c @@ -812,6 +812,8 @@ int main(int argc, char *argv[]) { update_qb_log_settings(); } + cfs.force_local_mode = force_local_mode; + struct utsname utsname; if (uname(&utsname) != 0) { cfs_critical("Unable to read local node name"); -- 2.47.3