public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH cluster v5 07/41] pmxcfs: allow removing the cluster ID file in forced local mode
Date: Thu, 19 Nov 2026 20:18:39 +0100	[thread overview]
Message-ID: <20261119191914.2336934-8-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261119191914.2336934-1-t.lamprecht@proxmox.com>

A node that gets separated from its cluster without reinstalling, or a
cloned installation, keeps the cluster ID it had. With the file
protected against changes from user space, only editing the database by
hand would get it a new one.

Allow unlinking the file while pmxcfs runs in forced local mode (-l),
which the documented procedure for separating a node already uses to
remove the corosync.conf. The next start without a corosync.conf then
generates a new ID, like on any standalone node. Moving the file away
is allowed then too, as libfuse unlinks a file that is still open by
renaming it to a hidden name. Outside of forced local mode both stay
rejected, and nothing can be moved onto the path in any mode.

An unlink done in local mode on a node that later rejoins its cluster
replicates like any other change made in local mode: if the database
of that node wins the sync, the file is gone on all nodes, and the
cluster generates a new ID.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
---

no changes since v4

changes v3 -> v4:
* two comments: plainer wording, no other change

changes v1 -> v2:
* rebase only

 src/pmxcfs/cfs-plug-memdb.c |  8 ++++++--
 src/pmxcfs/cfs-utils.h      |  1 +
 src/pmxcfs/check_memdb.c    | 32 ++++++++++++++++++++++++++++++++
 src/pmxcfs/pmxcfs.c         |  2 ++
 4 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/src/pmxcfs/cfs-plug-memdb.c b/src/pmxcfs/cfs-plug-memdb.c
index 0d0fa64..eb79cbe 100644
--- a/src/pmxcfs/cfs-plug-memdb.c
+++ b/src/pmxcfs/cfs-plug-memdb.c
@@ -387,7 +387,9 @@ static int cfs_plug_memdb_rename(cfs_plug_t *plug, const char *from, const char
     g_return_val_if_fail(from != NULL, PARAM_CHECK_ERRNO);
     g_return_val_if_fail(to != NULL, PARAM_CHECK_ERRNO);
 
-    if (path_is_cluster_id(from) || path_is_cluster_id(to)) {
+    /* Moving it away is allowed in forced local mode, like unlinking it: libfuse unlinks a file
+     * that is still open by renaming it. */
+    if ((path_is_cluster_id(from) && !cfs.force_local_mode) || path_is_cluster_id(to)) {
         return -EACCES;
     }
 
@@ -410,7 +412,9 @@ static int cfs_plug_memdb_unlink(cfs_plug_t *plug, const char *path) {
     g_return_val_if_fail(plug->ops == &cfs_ops, PARAM_CHECK_ERRNO);
     g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
 
-    if (path_is_cluster_id(path)) {
+    /* Unlinking it in forced local mode is how a node that got separated from its cluster gets an
+     * ID of its own: its next start without a corosync.conf generates a new one. */
+    if (path_is_cluster_id(path) && !cfs.force_local_mode) {
         return -EACCES;
     }
 
diff --git a/src/pmxcfs/cfs-utils.h b/src/pmxcfs/cfs-utils.h
index f66e944..5016720 100644
--- a/src/pmxcfs/cfs-utils.h
+++ b/src/pmxcfs/cfs-utils.h
@@ -44,6 +44,7 @@ typedef struct {
     char *ip;
     gid_t gid;
     int debug;
+    gboolean force_local_mode;
 } cfs_t;
 
 extern cfs_t cfs;
diff --git a/src/pmxcfs/check_memdb.c b/src/pmxcfs/check_memdb.c
index a51433a..91cd207 100644
--- a/src/pmxcfs/check_memdb.c
+++ b/src/pmxcfs/check_memdb.c
@@ -480,6 +480,36 @@ START_TEST(test_cluster_id_standalone_replace_failure) {
 }
 END_TEST
 
+START_TEST(test_cluster_id_unlink_forced_local_mode) {
+    cfs_plug_t *plug = (cfs_plug_t *)cfs_plug_memdb_new("memdb", memdb, NULL);
+    ck_assert(plug != NULL);
+
+    ck_assert(memdb_create(memdb, "cluster-id", 0, 1234) == 0);
+    ck_assert(memdb_write(memdb, "cluster-id", 0, 1234, "abcdefghijklmnopqrst\n", 21, 0, 1) == 21);
+
+    ck_assert(memdb_create(memdb, "other", 0, 1234) == 0);
+
+    ck_assert(plug->ops->unlink(plug, "cluster-id") == -EACCES);
+    ck_assert(plug->ops->rename(plug, "cluster-id", ".fuse_hidden01") == -EACCES);
+
+    cfs.force_local_mode = TRUE;
+    ck_assert(plug->ops->truncate(plug, "cluster-id", 0) == -EACCES);
+    ck_assert(plug->ops->rename(plug, "other", "cluster-id") == -EACCES);
+
+    /* what libfuse does on unlink while the file is open */
+    ck_assert(plug->ops->rename(plug, "cluster-id", ".fuse_hidden01") == 0);
+    ck_assert(memdb_getattr(memdb, "cluster-id") == NULL);
+
+    ck_assert(memdb_create(memdb, "cluster-id", 0, 1234) == 0);
+    ck_assert(plug->ops->unlink(plug, "cluster-id") == 0);
+    ck_assert(memdb_getattr(memdb, "cluster-id") == NULL);
+
+    cfs.force_local_mode = FALSE;
+
+    plug->destroy_plug(plug);
+}
+END_TEST
+
 START_TEST(test_truncate_grow) {
     const char *fn = "/f1";
     time_t ctime = 1234;
@@ -768,6 +798,8 @@ static Suite *memdb_suite(void) {
 
     add_test(s, test_cluster_id_fuse_read_only, "cluster_id_fuse_read_only");
 
+    add_test(s, test_cluster_id_unlink_forced_local_mode, "cluster_id_unlink_forced_local_mode");
+
     add_test(s, test_cluster_id_is_valid, "cluster_id_is_valid");
 
     add_test(s, test_cluster_id_generate, "cluster_id_generate");
diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c
index 21a77a0..388c7ad 100644
--- a/src/pmxcfs/pmxcfs.c
+++ b/src/pmxcfs/pmxcfs.c
@@ -812,6 +812,8 @@ int main(int argc, char *argv[]) {
         update_qb_log_settings();
     }
 
+    cfs.force_local_mode = force_local_mode;
+
     struct utsname utsname;
     if (uname(&utsname) != 0) {
         cfs_critical("Unable to read local node name");
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 19:20 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 19:18 [PATCH cluster/guest-common/qemu-server/container/manager/docs v5 00/41] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 01/41] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 02/41] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 03/41] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 04/41] pmxcfs: quorum: record the corosync membership Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 05/41] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 06/41] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-19 19:18 ` Thomas Lamprecht [this message]
2026-11-19 19:18 ` [PATCH guest-common v5 08/41] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 09/41] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 10/41] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 11/41] guest identity: allow adopting a uid that is unused in the cluster Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 12/41] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 13/41] vzdump: notes template: list the uid variable Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 14/41] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 15/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 16/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 17/41] restore: adopt the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 18/41] api: remote migration: adopt the guest uid from the source Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 19/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 20/41] restore: decide by the reservation whether the container is new Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 21/41] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 22/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 23/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 24/41] restore: adopt the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 25/41] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 26/41] api: remote migration: adopt the guest uid from the source Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 27/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 28/41] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 29/41] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 30/41] ui: datacenter summary: show the " Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 31/41] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 32/41] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 33/41] ui: guest summary: show " Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 34/41] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 35/41] vzdump: notes template: add the guest uid as variable Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 36/41] vzdump: restore: describe what a failed restore can leave behind Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 37/41] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 38/41] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 39/41] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 40/41] pct: " Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 41/41] vzdump: notes: document the uid template variable Thomas Lamprecht
2026-11-20  3:07 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v5 00/41] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119191914.2336934-8-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal