all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: Dominik Csapak <d.csapak@proxmox.com>, pve-devel@lists.proxmox.com
Subject: Re: [PATCH cluster/guest-common/qemu-server/container/manager/docs 00/34] fix #7846: guest identity (uid) and cluster ID
Date: Tue, 17 Nov 2026 13:47:07 +0100	[thread overview]
Message-ID: <a10b546f-42d1-43cc-9ef7-15a715aa8aab@proxmox.com> (raw)
In-Reply-To: <deaa229d-4d7b-420f-ba8a-c72cdcfb055e@proxmox.com>

Am 17.11.26 um 12:55 schrieb Dominik Csapak:
> On 11/17/26 12:29 PM, Thomas Lamprecht wrote:
>> Am 17.11.26 um 11:47 schrieb Dominik Csapak:
>> I had it at 80 bits, but that is borderline w.r.t. hundreds of billions
>> of entities and the birthday problem, as we're getting near actual digit
>> percentages then (1 % chance of a clash at ~150 billion IDs). 100 bits
>> takes that boundary between practically impossible and a real chance of
>> clash in the hundreds of trillion range (1 % at ~160 trillion), which
>> seemed good enough for me.
> 
> but this is about the cluster id, surely there aren't billions of
> clusters/nodes ?

I want to use the same for clusters and guests, what benefit do I
have in doing that differently? 

> They also don't have to be globally unique in practice, just for one
> provider/customer/etc.

They do, else reality catches up with you fast, companies get bought
and their infra merged, cluster can move from providers (e.g., ceph
moved their whole CI infra from a RH datacenter to an IBM one with
some big trucks about a year ago), PDM can connect all of them. Really
not sure what you're trying to recommend here? Why make these edge
cases harder for anyone that has the bad luck handling them, what's
the benefit.

>>> since we then don't have to use half a byte (100bit => 12.5 byte)
>>> and still provide enough randomness.
>>
>> Base32 has 5 bit per character, so 100 bits are always 20 chars without
>> padding, that I found nice.
> 
> yes i meant the 'raw size' and basically always discarding half a byte
> of randomness, but sure.

What does that matter? 80 bit also doesn't align in word size of 64-bit,
could be also relevant. How many bits are relevant for entropy, that
needs a lower boundary and some upper bounder for practical purposes and
leveraging the chosen serialization of it.

Sure, we can discuss all day about some in a framework of core rules
somewhat arbitrary chosen specifics of our bikeshed color but if there
is not actually a relevant problem (e.g., to continue the methaphor
sun gets reflected and blinds one, or absorbed and heats the shed up)
I rather keep them, I played around with a few different formats and
bit sizes and this here felt best while still fulfilling the core
properties needed for us. base32 is IMO a great tradeof here between
ease of handling the character set and packing bits in single chars,
the non-uuid format ensures this is not ever used (or its generation
transformed to) something it shouldn't be.
I value your skepticism but here I'd rather avoid us derailing into
this to much, as while yes some properties are chosen by me in a way
that might seem arbitrary and might bee done slightly different too
while still fulfilling the core design-framework needs here, just
switching to another flavor for the sake of it doesn't really help us
much either – and IMO a random nonce being really just that and not
looking like something else has its merits.

Adding some more rationale to docs and commit message is certainly
warranted, this will be questioned by others, so from that POV I'm
happy it happens upfront from you.

>> That then allows one to tell apart e.g.:
>> - Same uid, changed rtime: the guest was registered again, for example
>>    destroyed and restored, or moved in from another cluster. A tool can
>>    then tell this from an "untouched" guest.
> 
> but this could also be detectable via ctime (except if we don't want
> to touch that on restore/migration ofc)

With "ctime will always be kept as is for a restore to a new VMID" I
meant that a ctime won't be touched and can't be used for this. Changing
this would also divert from a fixed behavior, i.e. technically a
breaking change. And yes, I do not believe in just technicalities
guiding design and allowing some technical breakage if it's not gonna
cause issues in practice and has some good justification (ROI), here the
ROI of such a change is IMO questionable, and as said, rather than
optimizing to much here I rather have this as complete as it might make
practical sense

> 
>> - Same uid in two clusters, which can happen after a remote migration
>>    that keeps the source, or a restore elsewhere. The uid together with
>>    the cluster ID and rtime still tells the instances apart.
> 
> for this the rtime is not needed though? uid + clusterid should
> be enough? (since same uid in one cluster can't happen)
> 

I was a bit imprecise here, for a single point in time yes that's true, but
backups normally have a history over a longer period of time and there
this ensures one can tell them apart (also potentially in logs or so,
but how/if we use them there is not all too relevant for now IMO, that
can be checked out later, we're not getting worse there in that regard as
the status quo now is after all). I.e., this is to differentiate between
the same guest and cluster uid over time, which is not that far out for us
due to remote migrations.


>> FWIW, my format allows to still map all uids into standard uuidv8,
>> so we're not locked in, but I do not want to use them and their
>> various versions here, simple random nonce that is formatted such
>> to be easy to handle (just alphanumerical) and still reasonably short.
>> That's my core requirement here. I.e., yes uuid might be wide spread,
>> but there to many tried to hard to add additional meaning to what should
>> be just a simple random nonce like this use case here.
> 
> makes sense, though i think having 20 vs 36 bytes in a config
> that is already ~500 bytes by default wouldn't make much difference IMO.
> (also we already have at least 2 uuids in the default config, so
> there would be precedent to using that format)

The uuid were not chosen by us and their (change) meaning is completely
different, so I do not see how we should go down that road just because
they got brought uppon us. Again, uid's are random nonces, the format
reflects that and is roughly optimized for its use case. no uuid.
We might look into deriving the smbios one from the uid by default
in the future though, still allowing manual overriding it as that should
be easily doable just needs a closer check if any change there when
uid changes causes actual issues, that would allow to avoid ~45 chars
by default. Similar we could also default the *initial* vmgenid to be
derived from that and only persist it once actually needed, dropping
like 92 bytes from our default config again, would make more than up
for the ones added here (and no uuid by default in the config anymore).

> we could also already opt to use a custom uuid format with the
> rtime encoded instead of having a separate property, but
> that is probably more pain that having it separate...

That would always mean uid changes if rtime changes, which breaks the
whole design.

>> It's only needed when a guest takes over an existing uid, so once at
>> the end of a restore or of an incoming remote migration. Without it,
>> restoring the same backup to two VMIDs, or to a new VMID while the
>> original still exists, would give two guests with the same uid in one
>> cluster. Newly generated uids (create, clone, first write of an
>> existing guest after the upgrade) take no cluster lock and do no
>> lookup, the randomness is enough there.
> 
> it's also necessary on clone and creation though? otherwise we can't
> be sure the random value is not in use? (unlikely but still)
> 
> if you argue about billions of clusters/nodes for 100bit randomness,
> then i think arguing with a much higher number of vms is fair?

No, I argue that this is safe for guests and the uses cases and reuse
the same for cluster as they get used as tripple for determining the
whole underlying reason of doing this all in the first place. Mixing
those then has no benefit here, on the contrary IMO.




  reply	other threads:[~2026-11-17 12:47 UTC|newest]

Thread overview: 49+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  0:46 [PATCH cluster/guest-common/qemu-server/container/manager/docs 00/34] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 01/34] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 02/34] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 03/34] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 04/34] pmxcfs: quorum: pass the corosync membership to an optional listener Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 05/34] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 06/34] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 07/34] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-16  0:46 ` [PATCH guest-common 08/34] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-17 10:47   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH guest-common 09/34] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-17 10:47   ` Dominik Csapak
2026-11-17 12:47     ` Dominik Csapak
2026-11-16  0:46 ` [PATCH guest-common 10/34] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-16  0:46 ` [PATCH guest-common 11/34] guest identity: allow taking over a uid that is unused in the cluster Thomas Lamprecht
2026-11-17 13:14   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH qemu-server 12/34] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
2026-11-16  0:46 ` [PATCH qemu-server 13/34] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-16  0:46 ` [PATCH qemu-server 14/34] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-16  0:46 ` [PATCH qemu-server 15/34] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-17 15:17   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH qemu-server 16/34] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-17 15:26   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH qemu-server 17/34] api: status: return the guest uid Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 18/34] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 19/34] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 20/34] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 21/34] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 22/34] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 23/34] migrate: remote: clean up the target on a tunnel version mismatch Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 24/34] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 25/34] api: status: return the guest uid Thomas Lamprecht
2026-11-16  0:46 ` [PATCH manager 26/34] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-17 15:30   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH manager 27/34] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-16  0:46 ` [PATCH manager 28/34] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-16  0:46 ` [PATCH manager 29/34] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-16  0:46 ` [PATCH manager 30/34] ui: guest summary: show " Thomas Lamprecht
2026-11-17 15:44   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH docs 31/34] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-16  0:46 ` [PATCH docs 32/34] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-16  0:46 ` [PATCH docs 33/34] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-16  0:46 ` [PATCH docs 34/34] pct: " Thomas Lamprecht
2026-11-17 10:47 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs 00/34] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
2026-11-17 11:29   ` Thomas Lamprecht
2026-11-17 11:55     ` Dominik Csapak
2026-11-17 12:47       ` Thomas Lamprecht [this message]
2026-11-17 13:54         ` Dominik Csapak
2026-11-17 15:55 ` Dominik Csapak

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a10b546f-42d1-43cc-9ef7-15a715aa8aab@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=d.csapak@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal