From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 0ECA61FF0AA for ; Tue, 17 Nov 2026 13:47:16 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 97DEC215BA; Tue, 17 Nov 2026 13:47:13 +0100 (CET) Message-ID: Date: Tue, 17 Nov 2026 13:47:07 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: [PATCH cluster/guest-common/qemu-server/container/manager/docs 00/34] fix #7846: guest identity (uid) and cluster ID To: Dominik Csapak , pve-devel@lists.proxmox.com References: <20261116004703.2908294-1-t.lamprecht@proxmox.com> <5aaeb776-c738-4e0a-bc6b-814b56f24b06@proxmox.com> <3f6c702a-ba0f-4e1f-ae4a-132d9b6d928a@proxmox.com> Content-Language: en-US From: Thomas Lamprecht Autocrypt: addr=t.lamprecht@proxmox.com; keydata= xsFNBFsLjcYBEACsaQP6uTtw/xHTUCKF4VD4/Wfg7gGn47+OfCKJQAD+Oyb3HSBkjclopC5J uXsB1vVOfqVYE6PO8FlD2L5nxgT3SWkc6Ka634G/yGDU3ZC3C/7NcDVKhSBI5E0ww4Qj8s9w OQRloemb5LOBkJNEUshkWRTHHOmk6QqFB/qBPW2COpAx6oyxVUvBCgm/1S0dAZ9gfkvpqFSD 90B5j3bL6i9FIv3YGUCgz6Ue3f7u+HsEAew6TMtlt90XV3vT4M2IOuECG/pXwTy7NtmHaBQ7 UJBcwSOpDEweNob50+9B4KbnVn1ydx+K6UnEcGDvUWBkREccvuExvupYYYQ5dIhRFf3fkS4+ wMlyAFh8PQUgauod+vqs45FJaSgTqIALSBsEHKEs6IoTXtnnpbhu3p6XBin4hunwoBFiyYt6 YHLAM1yLfCyX510DFzX/Ze2hLqatqzY5Wa7NIXqYYelz7tXiuCLHP84+sV6JtEkeSUCuOiUY virj6nT/nJK8m0BzdR6FgGtNxp7RVXFRz/+mwijJVLpFsyG1i0Hmv2zTn3h2nyGK/I6yhFNt dX69y5hbo6LAsRjLUvZeHXpTU4TrpN/WiCjJblbj5um5eEr4yhcwhVmG102puTtuCECsDucZ jpKpUqzXlpLbzG/dp9dXFH3MivvfuaHrg3MtjXY1i+/Oxyp5iwARAQABzTNUaG9tYXMgTGFt cHJlY2h0IChBdXRoLTQpIDx0LmxhbXByZWNodEBwcm94bW94LmNvbT7CwY4EEwEIADgWIQQO R4qbEl/pah9K6VrTZCM6gDZWBgUCWwuNxgIbAwULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAK CRDTZCM6gDZWBm/jD/4+6JB2s67eaqoP6x9VGaXNGJPCscwzLuxDTCG90G9FYu29VcXtubH/ bPwsyBbNUQpqTm/s4XboU2qpS5ykCuTjqavrcP33tdkYfGcItj2xMipJ1i3TWvpikQVsX42R G64wovLs/dvpTYphRZkg5DwhgTmy3mRkmofFCTa+//MOcNOORltemp984tWjpR3bUJETNWpF sKGZHa3N4kCNxb7A+VMsJZ/1gN3jbQbQG7GkJtnHlWkw9rKCYqBtWrnrHa4UAvSa9M/XCIAB FThFGqZI1ojdVlv5gd6b/nWxfOPrLlSxbUo5FZ1i/ycj7/24nznW1V4ykG9iUld4uYUY86bB UGSjew1KYp9FmvKiwEoB+zxNnuEQfS7/Bj1X9nxizgweiHIyFsRqgogTvLh403QMSGNSoArk tqkorf1U+VhEncIn4H3KksJF0njZKfilrieOO7Vuot1xKr9QnYrZzJ7m7ZxJ/JfKGaRHXkE1 feMmrvZD1AtdUATZkoeQtTOpMu4r6IQRfSdwm/CkppZXfDe50DJxAMDWwfK2rr2bVkNg/yZI tKLBS0YgRTIynkvv0h8d9dIjiicw3RMeYXyqOnSWVva2r+tl+JBaenr8YTQw0zARrhC0mttu cIZGnVEvQuDwib57QLqMjQaC1gazKHvhA15H5MNxUhwm229UmdH3KM7BTQRbC43GARAAyTkR D6KRJ9Xa2fVMh+6f186q0M3ni+5tsaVhUiykxjsPgkuWXWW9MbLpYXkzX6h/RIEKlo2BGA95 QwG5+Ya2Bo3g7FGJHAkXY6loq7DgMp5/TVQ8phsSv3WxPTJLCBq6vNBamp5hda4cfXFUymsy HsJy4dtgkrPQ/bnsdFDCRUuhJHopnAzKHN8APXpKU6xV5e3GE4LwFsDhNHfH/m9+2yO/trcD txSFpyftbK2gaMERHgA8SKkzRhiwRTt9w5idOfpJVkYRsgvuSGZ0pcD4kLCOIFrer5xXudk6 NgJc36XkFRMnwqrL/bB4k6Pi2u5leyqcXSLyBgeHsZJxg6Lcr2LZ35+8RQGPOw9C0ItmRjtY ZpGKPlSxjxA1WHT2YlF9CEt3nx7c4C3thHHtqBra6BGPyW8rvtq4zRqZRLPmZ0kt/kiMPhTM 8wZAlObbATVrUMcZ/uNjRv2vU9O5aTAD9E5r1B0dlqKgxyoImUWB0JgpILADaT3VybDd3C8X s6Jt8MytUP+1cEWt9VKo4vY4Jh5vwrJUDLJvzpN+TsYCZPNVj18+jf9uGRaoK6W++DdMAr5l gQiwsNgf9372dbMI7pt2gnT5/YdG+ZHnIIlXC6OUonA1Ro/Itg90Q7iQySnKKkqqnWVc+qO9 GJbzcGykxD6EQtCSlurt3/5IXTA7t6sAEQEAAcLBdgQYAQgAIBYhBA5HipsSX+lqH0rpWtNk IzqANlYGBQJbC43GAhsMAAoJENNkIzqANlYGD1sP/ikKgHgcspEKqDED9gQrTBvipH85si0j /Jwu/tBtnYjLgKLh2cjv1JkgYYjb3DyZa1pLsIv6rGnPX9bH9IN03nqirC/Q1Y1lnbNTynPk IflgvsJjoTNZjgu1wUdQlBgL/JhUp1sIYID11jZphgzfDgp/E6ve/8xE2HMAnf4zAfJaKgD0 F+fL1DlcdYUditAiYEuN40Ns/abKs8I1MYx7Yglu3RzJfBzV4t86DAR+OvuF9v188WrFwXCS RSf4DmJ8tntyNej+DVGUnmKHupLQJO7uqCKB/1HLlMKc5G3GLoGqJliHjUHUAXNzinlpE2Vj C78pxpwxRNg2ilE3AhPoAXrY5qED5PLE9sLnmQ9AzRcMMJUXjTNEDxEYbF55SdGBHHOAcZtA kEQKub86e+GHA+Z8oXQSGeSGOkqHi7zfgW1UexddTvaRwE6AyZ6FxTApm8wq8NT2cryWPWTF BDSGB3ujWHMM8ERRYJPcBSjTvt0GcEqnd+OSGgxTkGOdufn51oz82zfpVo1t+J/FNz6MRMcg 8nEC+uKvgzH1nujxJ5pRCBOquFZaGn/p71Yr0oVitkttLKblFsqwa+10Lt6HBxm+2+VLp4Ja 0WZNncZciz3V3cuArpan/ZhhyiWYV5FD0pOXPCJIx7WS9PTtxiv0AOS4ScWEUmBxyhFeOpYa DrEx In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794919628056 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.203 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 5KHFVIEZCGDI4WHOL7DHFUNSVHPKBY5W X-Message-ID-Hash: 5KHFVIEZCGDI4WHOL7DHFUNSVHPKBY5W X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Am 17.11.26 um 12:55 schrieb Dominik Csapak: > On 11/17/26 12:29 PM, Thomas Lamprecht wrote: >> Am 17.11.26 um 11:47 schrieb Dominik Csapak: >> I had it at 80 bits, but that is borderline w.r.t. hundreds of billions >> of entities and the birthday problem, as we're getting near actual digit >> percentages then (1 % chance of a clash at ~150 billion IDs). 100 bits >> takes that boundary between practically impossible and a real chance of >> clash in the hundreds of trillion range (1 % at ~160 trillion), which >> seemed good enough for me. > > but this is about the cluster id, surely there aren't billions of > clusters/nodes ? I want to use the same for clusters and guests, what benefit do I have in doing that differently? > They also don't have to be globally unique in practice, just for one > provider/customer/etc. They do, else reality catches up with you fast, companies get bought and their infra merged, cluster can move from providers (e.g., ceph moved their whole CI infra from a RH datacenter to an IBM one with some big trucks about a year ago), PDM can connect all of them. Really not sure what you're trying to recommend here? Why make these edge cases harder for anyone that has the bad luck handling them, what's the benefit. >>> since we then don't have to use half a byte (100bit => 12.5 byte) >>> and still provide enough randomness. >> >> Base32 has 5 bit per character, so 100 bits are always 20 chars without >> padding, that I found nice. > > yes i meant the 'raw size' and basically always discarding half a byte > of randomness, but sure. What does that matter? 80 bit also doesn't align in word size of 64-bit, could be also relevant. How many bits are relevant for entropy, that needs a lower boundary and some upper bounder for practical purposes and leveraging the chosen serialization of it. Sure, we can discuss all day about some in a framework of core rules somewhat arbitrary chosen specifics of our bikeshed color but if there is not actually a relevant problem (e.g., to continue the methaphor sun gets reflected and blinds one, or absorbed and heats the shed up) I rather keep them, I played around with a few different formats and bit sizes and this here felt best while still fulfilling the core properties needed for us. base32 is IMO a great tradeof here between ease of handling the character set and packing bits in single chars, the non-uuid format ensures this is not ever used (or its generation transformed to) something it shouldn't be. I value your skepticism but here I'd rather avoid us derailing into this to much, as while yes some properties are chosen by me in a way that might seem arbitrary and might bee done slightly different too while still fulfilling the core design-framework needs here, just switching to another flavor for the sake of it doesn't really help us much either – and IMO a random nonce being really just that and not looking like something else has its merits. Adding some more rationale to docs and commit message is certainly warranted, this will be questioned by others, so from that POV I'm happy it happens upfront from you. >> That then allows one to tell apart e.g.: >> - Same uid, changed rtime: the guest was registered again, for example >> destroyed and restored, or moved in from another cluster. A tool can >> then tell this from an "untouched" guest. > > but this could also be detectable via ctime (except if we don't want > to touch that on restore/migration ofc) With "ctime will always be kept as is for a restore to a new VMID" I meant that a ctime won't be touched and can't be used for this. Changing this would also divert from a fixed behavior, i.e. technically a breaking change. And yes, I do not believe in just technicalities guiding design and allowing some technical breakage if it's not gonna cause issues in practice and has some good justification (ROI), here the ROI of such a change is IMO questionable, and as said, rather than optimizing to much here I rather have this as complete as it might make practical sense > >> - Same uid in two clusters, which can happen after a remote migration >> that keeps the source, or a restore elsewhere. The uid together with >> the cluster ID and rtime still tells the instances apart. > > for this the rtime is not needed though? uid + clusterid should > be enough? (since same uid in one cluster can't happen) > I was a bit imprecise here, for a single point in time yes that's true, but backups normally have a history over a longer period of time and there this ensures one can tell them apart (also potentially in logs or so, but how/if we use them there is not all too relevant for now IMO, that can be checked out later, we're not getting worse there in that regard as the status quo now is after all). I.e., this is to differentiate between the same guest and cluster uid over time, which is not that far out for us due to remote migrations. >> FWIW, my format allows to still map all uids into standard uuidv8, >> so we're not locked in, but I do not want to use them and their >> various versions here, simple random nonce that is formatted such >> to be easy to handle (just alphanumerical) and still reasonably short. >> That's my core requirement here. I.e., yes uuid might be wide spread, >> but there to many tried to hard to add additional meaning to what should >> be just a simple random nonce like this use case here. > > makes sense, though i think having 20 vs 36 bytes in a config > that is already ~500 bytes by default wouldn't make much difference IMO. > (also we already have at least 2 uuids in the default config, so > there would be precedent to using that format) The uuid were not chosen by us and their (change) meaning is completely different, so I do not see how we should go down that road just because they got brought uppon us. Again, uid's are random nonces, the format reflects that and is roughly optimized for its use case. no uuid. We might look into deriving the smbios one from the uid by default in the future though, still allowing manual overriding it as that should be easily doable just needs a closer check if any change there when uid changes causes actual issues, that would allow to avoid ~45 chars by default. Similar we could also default the *initial* vmgenid to be derived from that and only persist it once actually needed, dropping like 92 bytes from our default config again, would make more than up for the ones added here (and no uuid by default in the config anymore). > we could also already opt to use a custom uuid format with the > rtime encoded instead of having a separate property, but > that is probably more pain that having it separate... That would always mean uid changes if rtime changes, which breaks the whole design. >> It's only needed when a guest takes over an existing uid, so once at >> the end of a restore or of an incoming remote migration. Without it, >> restoring the same backup to two VMIDs, or to a new VMID while the >> original still exists, would give two guests with the same uid in one >> cluster. Newly generated uids (create, clone, first write of an >> existing guest after the upgrade) take no cluster lock and do no >> lookup, the randomness is enough there. > > it's also necessary on clone and creation though? otherwise we can't > be sure the random value is not in use? (unlikely but still) > > if you argue about billions of clusters/nodes for 100bit randomness, > then i think arguing with a much higher number of vms is fair? No, I argue that this is safe for guests and the uses cases and reuse the same for cluster as they get used as tripple for determining the whole underlying reason of doing this all in the first place. Mixing those then has no benefit here, on the contrary IMO.