From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id B399E1FF0AA for ; Tue, 17 Nov 2026 12:29:15 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id A9FC7215E9; Tue, 17 Nov 2026 12:29:11 +0100 (CET) Message-ID: <3f6c702a-ba0f-4e1f-ae4a-132d9b6d928a@proxmox.com> Date: Tue, 17 Nov 2026 12:29:04 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: [PATCH cluster/guest-common/qemu-server/container/manager/docs 00/34] fix #7846: guest identity (uid) and cluster ID To: Dominik Csapak , pve-devel@lists.proxmox.com References: <20261116004703.2908294-1-t.lamprecht@proxmox.com> <5aaeb776-c738-4e0a-bc6b-814b56f24b06@proxmox.com> Content-Language: en-US From: Thomas Lamprecht Autocrypt: addr=t.lamprecht@proxmox.com; keydata= xsFNBFsLjcYBEACsaQP6uTtw/xHTUCKF4VD4/Wfg7gGn47+OfCKJQAD+Oyb3HSBkjclopC5J uXsB1vVOfqVYE6PO8FlD2L5nxgT3SWkc6Ka634G/yGDU3ZC3C/7NcDVKhSBI5E0ww4Qj8s9w OQRloemb5LOBkJNEUshkWRTHHOmk6QqFB/qBPW2COpAx6oyxVUvBCgm/1S0dAZ9gfkvpqFSD 90B5j3bL6i9FIv3YGUCgz6Ue3f7u+HsEAew6TMtlt90XV3vT4M2IOuECG/pXwTy7NtmHaBQ7 UJBcwSOpDEweNob50+9B4KbnVn1ydx+K6UnEcGDvUWBkREccvuExvupYYYQ5dIhRFf3fkS4+ wMlyAFh8PQUgauod+vqs45FJaSgTqIALSBsEHKEs6IoTXtnnpbhu3p6XBin4hunwoBFiyYt6 YHLAM1yLfCyX510DFzX/Ze2hLqatqzY5Wa7NIXqYYelz7tXiuCLHP84+sV6JtEkeSUCuOiUY virj6nT/nJK8m0BzdR6FgGtNxp7RVXFRz/+mwijJVLpFsyG1i0Hmv2zTn3h2nyGK/I6yhFNt dX69y5hbo6LAsRjLUvZeHXpTU4TrpN/WiCjJblbj5um5eEr4yhcwhVmG102puTtuCECsDucZ jpKpUqzXlpLbzG/dp9dXFH3MivvfuaHrg3MtjXY1i+/Oxyp5iwARAQABzTNUaG9tYXMgTGFt cHJlY2h0IChBdXRoLTQpIDx0LmxhbXByZWNodEBwcm94bW94LmNvbT7CwY4EEwEIADgWIQQO R4qbEl/pah9K6VrTZCM6gDZWBgUCWwuNxgIbAwULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAK CRDTZCM6gDZWBm/jD/4+6JB2s67eaqoP6x9VGaXNGJPCscwzLuxDTCG90G9FYu29VcXtubH/ bPwsyBbNUQpqTm/s4XboU2qpS5ykCuTjqavrcP33tdkYfGcItj2xMipJ1i3TWvpikQVsX42R G64wovLs/dvpTYphRZkg5DwhgTmy3mRkmofFCTa+//MOcNOORltemp984tWjpR3bUJETNWpF sKGZHa3N4kCNxb7A+VMsJZ/1gN3jbQbQG7GkJtnHlWkw9rKCYqBtWrnrHa4UAvSa9M/XCIAB FThFGqZI1ojdVlv5gd6b/nWxfOPrLlSxbUo5FZ1i/ycj7/24nznW1V4ykG9iUld4uYUY86bB UGSjew1KYp9FmvKiwEoB+zxNnuEQfS7/Bj1X9nxizgweiHIyFsRqgogTvLh403QMSGNSoArk tqkorf1U+VhEncIn4H3KksJF0njZKfilrieOO7Vuot1xKr9QnYrZzJ7m7ZxJ/JfKGaRHXkE1 feMmrvZD1AtdUATZkoeQtTOpMu4r6IQRfSdwm/CkppZXfDe50DJxAMDWwfK2rr2bVkNg/yZI tKLBS0YgRTIynkvv0h8d9dIjiicw3RMeYXyqOnSWVva2r+tl+JBaenr8YTQw0zARrhC0mttu cIZGnVEvQuDwib57QLqMjQaC1gazKHvhA15H5MNxUhwm229UmdH3KM7BTQRbC43GARAAyTkR D6KRJ9Xa2fVMh+6f186q0M3ni+5tsaVhUiykxjsPgkuWXWW9MbLpYXkzX6h/RIEKlo2BGA95 QwG5+Ya2Bo3g7FGJHAkXY6loq7DgMp5/TVQ8phsSv3WxPTJLCBq6vNBamp5hda4cfXFUymsy HsJy4dtgkrPQ/bnsdFDCRUuhJHopnAzKHN8APXpKU6xV5e3GE4LwFsDhNHfH/m9+2yO/trcD txSFpyftbK2gaMERHgA8SKkzRhiwRTt9w5idOfpJVkYRsgvuSGZ0pcD4kLCOIFrer5xXudk6 NgJc36XkFRMnwqrL/bB4k6Pi2u5leyqcXSLyBgeHsZJxg6Lcr2LZ35+8RQGPOw9C0ItmRjtY ZpGKPlSxjxA1WHT2YlF9CEt3nx7c4C3thHHtqBra6BGPyW8rvtq4zRqZRLPmZ0kt/kiMPhTM 8wZAlObbATVrUMcZ/uNjRv2vU9O5aTAD9E5r1B0dlqKgxyoImUWB0JgpILADaT3VybDd3C8X s6Jt8MytUP+1cEWt9VKo4vY4Jh5vwrJUDLJvzpN+TsYCZPNVj18+jf9uGRaoK6W++DdMAr5l gQiwsNgf9372dbMI7pt2gnT5/YdG+ZHnIIlXC6OUonA1Ro/Itg90Q7iQySnKKkqqnWVc+qO9 GJbzcGykxD6EQtCSlurt3/5IXTA7t6sAEQEAAcLBdgQYAQgAIBYhBA5HipsSX+lqH0rpWtNk IzqANlYGBQJbC43GAhsMAAoJENNkIzqANlYGD1sP/ikKgHgcspEKqDED9gQrTBvipH85si0j /Jwu/tBtnYjLgKLh2cjv1JkgYYjb3DyZa1pLsIv6rGnPX9bH9IN03nqirC/Q1Y1lnbNTynPk IflgvsJjoTNZjgu1wUdQlBgL/JhUp1sIYID11jZphgzfDgp/E6ve/8xE2HMAnf4zAfJaKgD0 F+fL1DlcdYUditAiYEuN40Ns/abKs8I1MYx7Yglu3RzJfBzV4t86DAR+OvuF9v188WrFwXCS RSf4DmJ8tntyNej+DVGUnmKHupLQJO7uqCKB/1HLlMKc5G3GLoGqJliHjUHUAXNzinlpE2Vj C78pxpwxRNg2ilE3AhPoAXrY5qED5PLE9sLnmQ9AzRcMMJUXjTNEDxEYbF55SdGBHHOAcZtA kEQKub86e+GHA+Z8oXQSGeSGOkqHi7zfgW1UexddTvaRwE6AyZ6FxTApm8wq8NT2cryWPWTF BDSGB3ujWHMM8ERRYJPcBSjTvt0GcEqnd+OSGgxTkGOdufn51oz82zfpVo1t+J/FNz6MRMcg 8nEC+uKvgzH1nujxJ5pRCBOquFZaGn/p71Yr0oVitkttLKblFsqwa+10Lt6HBxm+2+VLp4Ja 0WZNncZciz3V3cuArpan/ZhhyiWYV5FD0pOXPCJIx7WS9PTtxiv0AOS4ScWEUmBxyhFeOpYa DrEx In-Reply-To: <5aaeb776-c738-4e0a-bc6b-814b56f24b06@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794914945587 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.053 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: YZLQR4PZLCM274XGFYNWWWEPQUSK6TWJ X-Message-ID-Hash: YZLQR4PZLCM274XGFYNWWWEPQUSK6TWJ X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Am 17.11.26 um 11:47 schrieb Dominik Csapak: > I started reviewing this series and wanted to give some early feedback on the > overall approach. More in depth code review see the individual patches (might > take a bit going through every patch) thank you for taking a look, appreciated! > Note that i might have overlooked something. If so, please > do tell me. > > My high level feedback: > > Cluster IDs: > > Is there a particular reason why 100bit + base32? Yes, they are much shorter serialized (20 chars) and have enough entropy to make them more than good enough with head room while not getting larger than needed. > I couldn't find a rationale for the choice (did i overlook it?) No, v1 only states the format (module comment in PVE::GuestIdentity, commit messages, docs), not really the reasoning/rational behind it. I'll expand on that for a v2 (or on applying) and will add a few sentences to the cover letter too, as it certainly is something one can wonder about. > and I think using something 'industry standard' like a > UUIDv7 would be a better choice? I'ts just 28 bit more ( > with a bit less randomness) but with a time stamp included and this > should make each cluster uniquely identifiable too. While I thought earlier about using snowflake or some other timestamp influenced uid, they are not ideal here as they do not relate to anything over the lifetime of a guest w.r.t. restores and such. > If we want to stay on base32, maybe 80 or 120 bit is a better choice, I had it at 80 bits, but that is borderline w.r.t. hundreds of billions of entities and the birthday problem, as we're getting near actual digit percentages then (1 % chance of a clash at ~150 billion IDs). 100 bits takes that boundary between practically impossible and a real chance of clash in the hundreds of trillion range (1 % at ~160 trillion), which seemed good enough for me. > since we then don't have to use half a byte (100bit => 12.5 byte) > and still provide enough randomness. Base32 has 5 bit per character, so 100 bits are always 20 chars without padding, that I found nice. > VM UIDs: > > Same question about the rationale behind the format, but with one > change: the 'registering' time is already recorded in > the meta object (as ctime), no? so providing an additional separate ctime is creation time, which is the first creation ever. Registration time is something else, it's the time the guest got its identity in this cluster. E.g. ctime will always be kept as is for a restore to a new VMID. The uid changes if unique is set or if another guest in the cluster already has it (e.g. the original still exists), but otherwise stays the same. The registration time is regenerated in any case – making the combination of rtime and uid (and cluster ID) unique in practice. This comes from the model that backup providers use and even if a bit odd at first it works well for all cases (if this series here implements it right that is...). That then allows one to tell apart e.g.: - Same uid, changed rtime: the guest was registered again, for example destroyed and restored, or moved in from another cluster. A tool can then tell this from an "untouched" guest. - Same uid in two clusters, which can happen after a remote migration that keeps the source, or a restore elsewhere. The uid together with the cluster ID and rtime still tells the instances apart. I tried hard to do all that's needed to cover all bases but only really that what's needed here. Ideally we do not touch this ever again from a principles POV (knocks wood). > timestamp seems weird to me. Also using UUIDv7 here > would include a timestamp too and also be more 'industry standard' > while only using 1 byte extra (36 vs 20 + 15). Though no v7 for these things for the same reason, it's rather confusing then. The timestamp of a v7 is fixed when the ID gets generated, while here the uid must stay the same over restores and remote migrations and the registration time must change then, so the two cannot be one value. uid *must* be random, that's a core property and ensures it's never taken for anything that it really isn't. FWIW, my format allows to still map all uids into standard uuidv8, so we're not locked in, but I do not want to use them and their various versions here, simple random nonce that is formatted such to be easy to handle (just alphanumerical) and still reasonably short. That's my core requirement here. I.e., yes uuid might be wide spread, but there to many tried to hard to add additional meaning to what should be just a simple random nonce like this use case here. > we probably would save some space by not having two properties in > the meta object. see above, required. > UID adoption/checks: > > The cluster wide check is necessary, i get it. Handling It's only needed when a guest takes over an existing uid, so once at the end of a restore or of an incoming remote migration. Without it, restoring the same backup to two VMIDs, or to a new VMID while the original still exists, would give two guests with the same uid in one cluster. Newly generated uids (create, clone, first write of an existing guest after the upgrade) take no cluster lock and do no lookup, the randomness is enough there. > it with a cluster lock + reading the cofnig > could make that lookup a bottleneck in big and dynamic clusters? > Or did i misunderstood how we get the meta config out? The lookup does not read config files one by one, it's a single IPC call to pmxcfs that collects the 'meta' property from its in-memory configs, i.e. the same call /cluster/resources already does on every request for lock and tags. The lock is held for that call plus one config write. So I don't expect this to become a bottleneck, and once we have pmxcfs in rust with types for such guest configs we can do this there on a nice abstraction level while still being really fast. > Alternatively we could have that lookup in pmxcfs itself and > replicate it like rrd, etc.? having a uuid saved as > a 128bit number (16byte) would mean no, this must IMO not be a sidecar or saved out of band, coupling it directly to the guest config, the core entity that makes out a guest, is IMO crucial to get a safe design. The lookup already happens in pmxcfs as described above, just without a second copy of the data that could get out of sync with the configs. > 160k for 10000 guests > 16M for 1000000 guests > > which looks reasonable to me? > (we already have to have much more data in pmxcfs > with that many guests...) Probably not needed, but we can optimize the basic approach here if needed (special IPC call for check+lock), but that's basically an implementation detail that won't change the overall design IMO.