* [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs
@ 2026-10-12 15:45 Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 01/22] cluster files: add virtual-guest/used-guest-ids Michael Köppl
` (21 more replies)
0 siblings, 22 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
This is based on the original series [0] by Severen Redwood and Daniel
Krambrock. It is rebased on the latest master branches and incorporates
the feedback from v7 [3].
The last patch for cluster and the last two for guest-common and manager
add the optional enforcement of the next-id settings. The rest of the
series works without them. Since IDs are recorded before a guest exists
and stay recorded if its creation fails, retrying a failed creation
with the same ID is rejected when both 'enforce' and 'unique' are set.
I kept the trailers from the original series where I made only minimal
changes, but added markers (MK: ...) in those cases. For bigger
changes, I changed the `Co-authored-by` and `Signed-off-by` trailers to
`Originally-by` (except for the UI patch for the 'unique' checkbox,
which I rewrote). I removed Aaron's T-b and R-b trailers since it's
been a long time and the trailers might be misleading considering the
series has changed in some of its implementation details overall.
Dependencies:
- guest-common needs a versioned build and runtime dependency on
libpve-cluster-perl with virtual-guest/used-guest-ids registered,
since loading PVE::AbstractConfig now registers the file.
- qemu-server and container need a versioned dependency on
libpve-guest-common-perl with register_used_id().
- manager needs versioned dependencies on libpve-cluster-perl (next-id
'unique' and 'enforce') and libpve-guest-common-perl
(get_next_unused_id() and assert_id_satisfies_next_id_settings()).
Changes since v7 [3] (thanks @Fiona for the review and testing):
- guest-common:
- get_next_unused_id() no longer restarts the traversal of the used ID
ranges for every existing guest. next_unused() is inlined and
assert_id_satisfies_next_id_settings() uses get_next_unused_id()
instead.
- register_used_id() detects existing guests itself through
check_vmid_unused() instead of taking an 'existing' option. This
drops the qemu-server and pve-container patches that passed the
param in.
- manager: keep the translated message for IDs of guests known to the UI
in the guest ID field. Only show the nextid error otherwise.
- Added Fiona's R-b and T-b to all patches except 5/22, since I
considered the rework of get_next_unused_id() substantial enough to
not want to suggest that this has actually been reviewed and tested by
her.
Changes since v6 [2] (thanks @Fiona for the feedback):
- guest-common:
- Add preparatory commit, sorting the install list in src/Makefile
alphabetically.
- Renamed check_range() to assert_id_in_next_id_range() and
check_enforced_id() to assert_id_satisfies_next_id_settings(), and
moved the latter into its own commit.
- Improved naming of variables for better clarity
- Expanded the example comment for close_smallest_gaps() (thanks,
@Fiona)
- Tests: added cases for a missing trailing newline and for merging
ranges with adjacent IDs, use only valid guest IDs, and dropped
checks of the order in which equal gaps are closed.
- manager:
- New patches to show the error returned by the nextid endpoint in
the guest ID field, instead of always reporting the ID as in use.
- nextid: the error for IDs of existing guests now refers to guest
IDs instead of VMs.
- docs: new patches to list the used-guest-ids file in pmxcfs.adoc and
to describe the 'unique' and 'enforce' options in pvecm.adoc.
Changes since v5 [1] (thanks @Fiona for the extensive feedback):
- Moved the file to virtual-guest/used-guest-ids.
- Keep used IDs as ranges in memory as well. get_next_unused_id()
returns the lowest ID that is neither in use nor was used before.
- Renamed the module to PVE::GuestID, use signatures, and renamed
add_id() to register_used_id().
- Record IDs in create_and_lock_config(), which also covers qm
importovf, and when remote migration with --delete removes a guest.
- Failing to record an ID only warns while 'unique' is off.
- Close the smallest gaps once the file would exceed 768 KiB, and warn
above 512 KiB. Are these thresholds fine?
- 'unique' is now a sub-property of next-id.
- New optional 'enforce' sub-property of next-id.
- Added tests for for the GuestID module.
Changes since v4:
- File in pmxcfs has been renamed to used-guest-ids to make it a bit
more general (both VMs and CTs) while still making clear which IDs
are meant.
- Moved the `UsedVmidList` module from pve-manager to pve-guest-common
and renamed it `UsedGuestIDs`. Also renamed some of the functions
there for more consistent naming.
- Implemented `write_id_list` more closely aligned with Fabian's
feedback.
- Also record used VMIDs when creating, cloning, destroying guests as
well as incoming remote migrations. (Thanks, @Fiona, for the
off-list input and feedback!)
- Moved the `add_id` calls for recording used IDs to before anything
is persisted during any of the above actions.
- Rephrased some commit messages.
[0] https://lore.proxmox.com/pve-devel/mailman.62.1731030290.372.pve-devel@lists.proxmox.com/
[1] https://lore.proxmox.com/pve-devel/20260921155410.938337-1-m.koeppl@proxmox.com/
[2] https://lore.proxmox.com/pve-devel/20260924161510.847362-1-m.koeppl@proxmox.com/
[3] https://lore.proxmox.com/pve-devel/20261005144805.825538-1-m.koeppl@proxmox.com/
cluster:
Michael Köppl (2):
datacenter config: add unique subproperty to next-id
datacenter config: next-id: add enforce subproperty
Severen Redwood (1):
cluster files: add virtual-guest/used-guest-ids
src/PVE/Cluster.pm | 1 +
src/PVE/DataCenterConfig.pm | 15 +++++++++++++++
src/pmxcfs/status.c | 1 +
3 files changed, 17 insertions(+)
guest-common:
Michael Köppl (8):
src/makefile: order files alphabetically
add module to track previously used guest IDs
tests: add tests for used guest ID tracking
abstract config: register used guest ID when creating config
guest id: keep used ID list below the pmxcfs file size limit
tests: add tests for used-guest-ids max file size handling
guest id: optionally enforce the next-id range and uniqueness
guest id: add helper to check guest IDs against next-id enforcement
debian/control | 1 +
src/Makefile | 7 +-
src/PVE/AbstractConfig.pm | 3 +
src/PVE/GuestID.pm | 253 ++++++++++++++++++++++++++++++
src/tests/Makefile | 7 +-
src/tests/guest-id-tests.pl | 304 ++++++++++++++++++++++++++++++++++++
6 files changed, 570 insertions(+), 5 deletions(-)
create mode 100644 src/PVE/GuestID.pm
create mode 100755 src/tests/guest-id-tests.pl
qemu-server:
Michael Köppl (1):
api: record VM ID as used on destruction and remote migration
src/PVE/API2/Qemu.pm | 9 +++++++++
src/PVE/QemuMigrate.pm | 4 ++++
2 files changed, 13 insertions(+)
container:
Michael Köppl (1):
api: record CT ID as used on destruction and remote migration
src/PVE/API2/LXC.pm | 5 +++++
src/PVE/LXC/Migrate.pm | 4 ++++
2 files changed, 9 insertions(+)
manager:
Michael Köppl (7):
ui: guest ID selector: rename exists flag to rejected
ui: guest ID selector: show the error returned by nextid
fix #4369: api: optionally only suggest unique IDs
ui: dc options: rename VMID to guest ID
fix #4369: ui: dc options: add option for unique VM/CT IDs
api: nextid: reject IDs forbidden by next-id enforcement
ui: dc options: add option to enforce next free guest ID settings
PVE/API2/Cluster.pm | 19 +++++++++++++++----
www/manager6/dc/OptionView.js | 18 +++++++++++++++++-
www/manager6/form/GuestIDSelector.js | 19 ++++++++++++++-----
3 files changed, 46 insertions(+), 10 deletions(-)
docs:
Michael Köppl (2):
pmxcfs: files: add virtual-guest/used-guest-ids
pvecm: next-id: document unique and enforce options
pmxcfs.adoc | 1 +
pvecm.adoc | 33 +++++++++++++++++++++++++++++++--
2 files changed, 32 insertions(+), 2 deletions(-)
Summary over all repositories:
18 files changed, 687 insertions(+), 17 deletions(-)
--
Generated by murpp 0.12.1
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH cluster v8 01/22] cluster files: add virtual-guest/used-guest-ids
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 02/22] datacenter config: add unique subproperty to next-id Michael Köppl
` (20 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
From: Severen Redwood <severen.redwood@sitehost.co.nz>
Add `/etc/pve/virtual-guest/used-guest-ids` to the list of cluster
files, which will be used for recording previously used VM/CT IDs.
This is required so that it can optionally be ensured that such IDs
are not suggested.
Co-authored-by: Daniel Krambrock <krambrock@hrz.uni-marburg.de>
Signed-off-by: Severen Redwood <severen.redwood@sitehost.co.nz>
[MK: change name of file to used-guest-ids, move it to virtual-guest/
adapt commit message]
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/Cluster.pm | 1 +
src/pmxcfs/status.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index 034b78c..9479baa 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -89,6 +89,7 @@ my $observed = {
'sdn/.running-config' => 1,
'virtual-guest/cpu-models.conf' => 1,
'virtual-guest/profiles.cfg' => 1,
+ 'virtual-guest/used-guest-ids' => 1,
'mapping/directory.cfg' => 1,
'mapping/pci.cfg' => 1,
'mapping/usb.cfg' => 1,
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index 12a6c46..d04ad22 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -116,6 +116,7 @@ static memdb_change_t memdb_change_array[] = {
{.path = "sdn/.running-config"},
{.path = "virtual-guest/cpu-models.conf"},
{.path = "virtual-guest/profiles.cfg"},
+ {.path = "virtual-guest/used-guest-ids"},
{.path = "firewall/cluster.fw"},
{.path = "mapping/directory.cfg"},
{.path = "mapping/pci.cfg"},
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH cluster v8 02/22] datacenter config: add unique subproperty to next-id
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 01/22] cluster files: add virtual-guest/used-guest-ids Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 03/22] datacenter config: next-id: add enforce subproperty Michael Köppl
` (19 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
Add 'unique' as a subproperty to next-id config schema. It controls
whether only unique (i.e. neither currently nor previously in use)
guest IDs should be suggested by the `/cluster/nextid` API endpoint.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/DataCenterConfig.pm | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/src/PVE/DataCenterConfig.pm b/src/PVE/DataCenterConfig.pm
index 004122e..4a58efe 100644
--- a/src/PVE/DataCenterConfig.pm
+++ b/src/PVE/DataCenterConfig.pm
@@ -217,6 +217,13 @@ my $next_id_format = {
default => 1000 * 1000, # lower than the maximum on purpose
optional => 1,
},
+ unique => {
+ type => 'boolean',
+ description => "Only suggest guest IDs that are neither currently in use"
+ . " nor have previously been used.",
+ default => 0,
+ optional => 1,
+ },
};
my $u2f_format = {
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH cluster v8 03/22] datacenter config: next-id: add enforce subproperty
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 01/22] cluster files: add virtual-guest/used-guest-ids Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 02/22] datacenter config: add unique subproperty to next-id Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 04/22] src/makefile: order files alphabetically Michael Köppl
` (18 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
Add 'enforce' as a subproperty to the next-id config schema. If enabled,
guest IDs outside of the configured range are rejected and, if 'unique'
is enabled as well, previously used IDs are also rejected, even if they
are not currently in use. The boundaries of the range count only when
explicitly configured, not for defaults.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/DataCenterConfig.pm | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/src/PVE/DataCenterConfig.pm b/src/PVE/DataCenterConfig.pm
index 4a58efe..5f4960c 100644
--- a/src/PVE/DataCenterConfig.pm
+++ b/src/PVE/DataCenterConfig.pm
@@ -224,6 +224,14 @@ my $next_id_format = {
default => 0,
optional => 1,
},
+ enforce => {
+ type => 'boolean',
+ description => "Reject IDs for new guests outside of the configured"
+ . " 'lower' and 'upper' boundaries and, with 'unique', IDs that"
+ . " were used before.",
+ default => 0,
+ optional => 1,
+ },
};
my $u2f_format = {
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH guest-common v8 04/22] src/makefile: order files alphabetically
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (2 preceding siblings ...)
2026-10-12 15:45 ` [PATCH cluster v8 03/22] datacenter config: next-id: add enforce subproperty Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 05/22] add module to track previously used guest IDs Michael Köppl
` (17 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/Makefile | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/Makefile b/src/Makefile
index 030e7f7..01ee0ae 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -6,12 +6,12 @@ all:
install: PVE
install -d ${PERL5DIR}/PVE
- install -m 0644 PVE/GuestHelpers.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/AbstractConfig.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/AbstractMigrate.pm ${PERL5DIR}/PVE/
+ install -m 0644 PVE/GuestHelpers.pm ${PERL5DIR}/PVE/
+ install -m 0644 PVE/Replication.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/ReplicationConfig.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/ReplicationState.pm ${PERL5DIR}/PVE/
- install -m 0644 PVE/Replication.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/StorageTunnel.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/Tunnel.pm ${PERL5DIR}/PVE/
install -d ${PERL5DIR}/PVE/Mapping
@@ -19,9 +19,9 @@ install: PVE
install -m 0644 PVE/Mapping/PCI.pm ${PERL5DIR}/PVE/Mapping/
install -m 0644 PVE/Mapping/USB.pm ${PERL5DIR}/PVE/Mapping/
install -d ${PERL5DIR}/PVE/VZDump
- install -m 0644 PVE/VZDump/Plugin.pm ${PERL5DIR}/PVE/VZDump/
install -m 0644 PVE/VZDump/Common.pm ${PERL5DIR}/PVE/VZDump/
install -m 0644 PVE/VZDump/JobBase.pm ${PERL5DIR}/PVE/VZDump/
+ install -m 0644 PVE/VZDump/Plugin.pm ${PERL5DIR}/PVE/VZDump/
.PHONY: check
check:
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH guest-common v8 05/22] add module to track previously used guest IDs
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (3 preceding siblings ...)
2026-10-12 15:45 ` [PATCH guest-common v8 04/22] src/makefile: order files alphabetically Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 06/22] tests: add tests for used guest ID tracking Michael Köppl
` (16 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
The /cluster/nextid API endpoint always suggests the lowest free
guest ID, so IDs of destroyed guests get handed out again. Offering
an opt-out requires recording every ID that has ever been in use
somewhere both the endpoint and the guest create and destroy paths
can reach.
Store them in /etc/pve/virtual-guest/used-guest-ids and register it as
a cfs file, so the list is kept in sync cluster-wide. IDs are stored as
"<start>-<end>" ranges and are not expanded per ID since cfs_read_file()
hands out a deep copy of the parsed data on every call, which gets
expensive on clusters that churn through many guests. Adjacent ranges
are merged on write to keep the file small. Entries that do not parse,
ranges with reversed bounds included, are skipped with a warning instead
of failing the whole read.
get_next_unused_id() returns the lowest ID at or above a given one that
is neither in use nor recorded as having been used previously.
Recording failures are non-critical and will only warn while 'unique' is
off.
Originally-by: Daniel Krambrock <krambrock@hrz.uni-marburg.de>
Originally-by: Severen Redwood <severen.redwood@sitehost.co.nz>
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
---
src/Makefile | 1 +
src/PVE/GuestID.pm | 128 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 129 insertions(+)
create mode 100644 src/PVE/GuestID.pm
diff --git a/src/Makefile b/src/Makefile
index 01ee0ae..161dbe5 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -9,6 +9,7 @@ install: PVE
install -m 0644 PVE/AbstractConfig.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/AbstractMigrate.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/GuestHelpers.pm ${PERL5DIR}/PVE/
+ install -m 0644 PVE/GuestID.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/Replication.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/ReplicationConfig.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/ReplicationState.pm ${PERL5DIR}/PVE/
diff --git a/src/PVE/GuestID.pm b/src/PVE/GuestID.pm
new file mode 100644
index 0000000..89219c8
--- /dev/null
+++ b/src/PVE/GuestID.pm
@@ -0,0 +1,128 @@
+package PVE::GuestID;
+
+use v5.36;
+
+use PVE::Cluster qw(
+ cfs_lock_file
+ cfs_read_file
+ cfs_register_file
+ cfs_write_file
+);
+
+my $FILENAME = 'virtual-guest/used-guest-ids';
+
+my sub parse_id_list($filename, $raw) {
+ my $ranges = [];
+
+ return $ranges if !defined($raw);
+
+ for my $line (split(/\n/, $raw)) {
+ next if $line =~ m/^\s*$/;
+
+ if ($line =~ m/^(\d+)$/) {
+ push $ranges->@*, [$1, $1];
+ } elsif ($line =~ m/^(\d+)-(\d+)$/) {
+ my ($start, $end) = ($1, $2);
+ if ($start > $end) {
+ warn "skipping reversed range in $filename: $line\n";
+ next;
+ }
+ push $ranges->@*, [$start, $end];
+ } else {
+ warn "skipping invalid entry in $filename: $line\n";
+ }
+ }
+
+ # lookup and insertion rely on the ranges being ordered by start,
+ # merging adjacent and overlapping ones is done when writing the
+ # id list.
+ return [sort { $a->[0] <=> $b->[0] } $ranges->@*];
+}
+
+my sub format_entry($start, $last) {
+ return $start == $last ? "$start\n" : "$start-$last\n";
+}
+
+my sub write_id_list($filename, $ranges) {
+ my $output = '';
+ my ($start, $last);
+
+ for my $range ($ranges->@*) {
+ my ($curr_start, $curr_end) = $range->@*;
+
+ if (!defined($start)) {
+ ($start, $last) = ($curr_start, $curr_end);
+ } elsif ($curr_start <= $last + 1) {
+ $last = $curr_end if $curr_end > $last;
+ } else {
+ $output .= format_entry($start, $last);
+ ($start, $last) = ($curr_start, $curr_end);
+ }
+ }
+
+ $output .= format_entry($start, $last) if defined($start);
+
+ return $output;
+}
+
+# Returns the lowest ID at or above $id that is neither taken by an
+# existing guest nor recorded as used before.
+sub get_next_unused_id($id) {
+ my $ranges = cfs_read_file($FILENAME);
+ my $vmlist = PVE::Cluster::get_vmlist() // {};
+ my $existing = $vmlist->{ids} // {};
+
+ my $i = 0;
+ while (1) {
+ while ($i < scalar($ranges->@*) && $ranges->[$i]->[1] < $id) {
+ $i++;
+ }
+
+ if ($i < scalar($ranges->@*) && $ranges->[$i]->[0] <= $id) {
+ $id = $ranges->[$i]->[1] + 1;
+ } elsif (defined($existing->{$id})) {
+ $id++;
+ } else {
+ return $id;
+ }
+ }
+}
+
+my sub insert_id($ranges, $id) {
+ my $i = 0;
+
+ while ($i < scalar(@$ranges) && $ranges->[$i]->[1] < $id) {
+ $i++;
+ }
+
+ return 0 if $i < scalar(@$ranges) && $ranges->[$i]->[0] <= $id;
+
+ splice(@$ranges, $i, 0, [$id, $id]);
+
+ return 1;
+}
+
+sub register_used_id($id) {
+ cfs_lock_file(
+ $FILENAME,
+ 10,
+ sub {
+ my $ranges = cfs_read_file($FILENAME);
+
+ return if !insert_id($ranges, $id);
+
+ cfs_write_file($FILENAME, $ranges);
+ },
+ );
+ if (my $err = $@) {
+ my $dc_conf = cfs_read_file('datacenter.cfg');
+
+ my $emsg = "unable to record guest ID $id as used";
+ die "$emsg - $err" if $dc_conf->{'next-id'}->{unique};
+ warn "$emsg - $err";
+ }
+}
+
+cfs_register_file($FILENAME, \&parse_id_list, \&write_id_list);
+
+1;
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH guest-common v8 06/22] tests: add tests for used guest ID tracking
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (4 preceding siblings ...)
2026-10-12 15:45 ` [PATCH guest-common v8 05/22] add module to track previously used guest IDs Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 07/22] abstract config: register used guest ID when creating config Michael Köppl
` (15 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
The used guest ID range handling has a few cases that are easy to get
subtly wrong. Adjacent and contained ranges have to be merged on write,
the next unused ID has to be found across adjacent ranges and past
existing guests that were possibly never recorded in the file, etc.
Add tests to cover the parser, writer, get_next_unused_id() and
register_used_id().
Test::MockModule is used here and added as a build dependency because
that is what most other PVE repositories use.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
debian/control | 1 +
src/tests/Makefile | 7 +-
src/tests/guest-id-tests.pl | 208 ++++++++++++++++++++++++++++++++++++
3 files changed, 214 insertions(+), 2 deletions(-)
create mode 100755 src/tests/guest-id-tests.pl
diff --git a/debian/control b/debian/control
index 21f0db2..539e3f0 100644
--- a/debian/control
+++ b/debian/control
@@ -6,6 +6,7 @@ Build-Depends: debhelper-compat (= 13),
libpve-cluster-perl,
libpve-common-perl,
libpve-storage-perl (>= 8.3.4),
+ libtest-mockmodule-perl,
pve-cluster,
Standards-Version: 4.6.2
Homepage: https://www.proxmox.com
diff --git a/src/tests/Makefile b/src/tests/Makefile
index 2fec185..f10564b 100644
--- a/src/tests/Makefile
+++ b/src/tests/Makefile
@@ -1,8 +1,11 @@
all:
-.PHONY: check abstract-config
-check: abstract-config
+.PHONY: check abstract-config guest-id
+check: abstract-config guest-id
abstract-config: abstract-config-tests.pl
perl -I.. ./abstract-config-tests.pl
+
+guest-id: guest-id-tests.pl
+ perl -I.. ./guest-id-tests.pl
diff --git a/src/tests/guest-id-tests.pl b/src/tests/guest-id-tests.pl
new file mode 100755
index 0000000..5cf796b
--- /dev/null
+++ b/src/tests/guest-id-tests.pl
@@ -0,0 +1,208 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib qw(..);
+
+use Test::MockModule;
+use Test::More;
+
+my $FILE = '/etc/pve/virtual-guest/used-guest-ids';
+
+my ($parse, $write);
+my $raw; # contents of the mocked used-guest-ids file
+my $writes = 0;
+my $injected_write_error;
+my $vmlist;
+my $dc_conf = {};
+
+my $pve_cluster_module = Test::MockModule->new('PVE::Cluster');
+$pve_cluster_module->mock(
+ cfs_register_file => sub($filename, $parser, $writer) {
+ ($parse, $write) = ($parser, $writer);
+ },
+ cfs_read_file => sub($filename) {
+ return $dc_conf if $filename eq 'datacenter.cfg';
+ return $parse->("/etc/pve/$filename", $raw);
+ },
+ cfs_write_file => sub($filename, $data) {
+ die $injected_write_error if defined($injected_write_error);
+ $writes++;
+ $raw = $write->("/etc/pve/$filename", $data);
+ },
+ cfs_lock_file => sub($filename, $timeout, $code) {
+ return eval { $code->() };
+ },
+ get_vmlist => sub() { return $vmlist },
+);
+
+# not 'use', since PVE::GuestID registers the parser and writer via cfs_register_file() when
+# loaded, so the mock above must be in place first to capture them
+require PVE::GuestID;
+
+sub range_to_vmlist(@ids) {
+ return { ids => { map { $_ => { node => 'localhost', type => 'qemu' } } @ids } };
+}
+
+subtest 'parse' => sub {
+ my $tests = [
+ ['canonical file', "100-250\n300\n900-1200\n", [[100, 250], [300, 300], [900, 1200]]],
+ [
+ 'unordered file is sorted but not merged',
+ "300-400\n100-200\n201-250\n150-160\n",
+ [[100, 200], [150, 160], [201, 250], [300, 400]],
+ ],
+ ['missing newline', "100-250\n300\n900-1000", [[100, 250], [300, 300], [900, 1000]]],
+ ['missing file', undef, []],
+ ['empty file', '', []],
+ ['blank lines only', "\n \n", []],
+ ];
+
+ for my $test ($tests->@*) {
+ my ($name, $input, $expected) = $test->@*;
+ is_deeply($parse->($FILE, $input), $expected, $name);
+ }
+
+ my @warnings;
+ local $SIG{__WARN__} = sub { push @warnings, $_[0] };
+
+ is_deeply(
+ $parse->($FILE, "100\nfoo\n50-10\n200\n"),
+ [[100, 100], [200, 200]],
+ 'invalid entry and reversed range are skipped',
+ );
+ is_deeply(
+ \@warnings,
+ [
+ "skipping invalid entry in $FILE: foo\n",
+ "skipping reversed range in $FILE: 50-10\n",
+ ],
+ 'skipped entries are warned about',
+ );
+};
+
+subtest 'write' => sub {
+ my $tests = [
+ [
+ 'canonical file is written unchanged',
+ "100-250\n300\n900-1200\n",
+ "100-250\n300\n900-1200\n",
+ ],
+ [
+ 'overlapping and adjacent ranges are merged',
+ "300-400\n100-200\n201-250\n150-160\n",
+ "100-250\n300-400\n",
+ ],
+ ['adjacent single IDs are merged', "100\n101\n102\n", "100-102\n"],
+ ['contained range keeps its container', "100-500\n200-300\n", "100-500\n"],
+ [
+ 'ranges with adjacent IDs are merged',
+ "1000-1004\n999\n100-200\n1005\n",
+ "100-200\n999-1005\n",
+ ],
+ ['nothing recorded', '', ''],
+ ];
+
+ for my $test ($tests->@*) {
+ my ($name, $input, $expected) = $test->@*;
+ is($write->($FILE, $parse->($FILE, $input)), $expected, $name);
+ }
+};
+
+subtest 'get_next_unused_id' => sub {
+ my $tests = [
+ # name, file contents, existing guests, input ID, expected ID
+ ['below all ranges', "101-200\n300-400\n", [], 100, 100],
+ ['start of a range', "100-200\n300-400\n", [], 100, 201],
+ ['inside a range', "100-200\n300-400\n", [], 150, 201],
+ ['in a gap', "100-200\n300-400\n", [], 250, 250],
+ ['above all ranges', "100-200\n300-400\n", [], 500, 500],
+ ['across adjacent ranges', "100-200\n201-300\n", [], 150, 301],
+ ['existing guests are skipped', "100-105\n", [106, 107], 100, 108],
+ ['used range after an existing guest', "100-105\n107-110\n", [106], 100, 111],
+ ['existing guests not recorded in the file', '', [100, 101], 100, 102],
+ ['no vmlist', "100-105\n", undef, 100, 106],
+ ['nothing recorded, no guests', undef, [], 100, 100],
+ ];
+
+ for my $test ($tests->@*) {
+ my ($name, $file, $existing, $id, $expected) = $test->@*;
+
+ $raw = $file;
+ $vmlist = defined($existing) ? range_to_vmlist($existing->@*) : undef;
+
+ is(PVE::GuestID::get_next_unused_id($id), $expected, $name);
+ }
+};
+
+subtest 'register_used_id' => sub {
+ $raw = undef;
+
+ my $steps = [
+ ['first ID', 101, "101\n"],
+ ['extends a range at its end', 102, "101-102\n"],
+ ['starts a new range after a gap', 104, "101-102\n104\n"],
+ ['bridges two ranges', 103, "101-104\n"],
+ ['extends a range at its start', 100, "100-104\n"],
+ ];
+
+ for my $step ($steps->@*) {
+ my ($name, $id, $expected) = $step->@*;
+ PVE::GuestID::register_used_id($id);
+ is($raw, $expected, $name);
+ }
+
+ $writes = 0;
+ PVE::GuestID::register_used_id(101);
+ PVE::GuestID::register_used_id(104);
+ is($writes, 0, 'already recorded IDs are not written again');
+ is($raw, "100-104\n", 'file unchanged by already recorded IDs');
+
+ $raw = "300-400\n100-200\n201-250\n150-160\n";
+ PVE::GuestID::register_used_id(275);
+ is($raw, "100-250\n275\n300-400\n", 'unordered file is normalized on write');
+};
+
+subtest 'register_used_id errors' => sub {
+ my $tests = [
+ # name, datacenter.cfg, whether the error reaches the caller
+ ['unique IDs requested', { 'next-id' => { unique => 1 } }, 1],
+ ['unique IDs not requested', { 'next-id' => { lower => 100 } }, 0],
+ ['no next-id settings', {}, 0],
+ ];
+
+ for my $test ($tests->@*) {
+ my ($name, $conf, $expect_fatal) = $test->@*;
+
+ $dc_conf = $conf;
+ $raw = "100\n";
+ $injected_write_error = "write failed\n";
+
+ my @warnings;
+ local $SIG{__WARN__} = sub { push @warnings, $_[0] };
+
+ eval { PVE::GuestID::register_used_id(500) };
+
+ if ($expect_fatal) {
+ is(
+ $@,
+ "unable to record guest ID 500 as used - write failed\n",
+ "$name: error is passed on to the caller",
+ );
+ is_deeply(\@warnings, [], "$name: no warning");
+ } else {
+ is($@, '', "$name: error is not passed on");
+ is_deeply(
+ \@warnings,
+ ["unable to record guest ID 500 as used - write failed\n"],
+ "$name: error is warned about",
+ );
+ }
+ is($raw, "100\n", "$name: file unchanged");
+ }
+
+ $injected_write_error = undef;
+ $dc_conf = {};
+};
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH guest-common v8 07/22] abstract config: register used guest ID when creating config
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (5 preceding siblings ...)
2026-10-12 15:45 ` [PATCH guest-common v8 06/22] tests: add tests for used guest ID tracking Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 08/22] guest id: keep used ID list below the pmxcfs file size limit Michael Köppl
` (14 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
This ensures that upon creation of a guest, the chosen guest ID is
always recorded such that optionally enabling unique IDs later on can
ensure that no ID that has previously been used can be chosen again.
Moving it into create_and_lock_config avoids repeating the
register_used_id call at every site where a guest config is created,
including importovf and incoming remote migrations.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/AbstractConfig.pm | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/PVE/AbstractConfig.pm b/src/PVE/AbstractConfig.pm
index 7bcae19..229e4bd 100644
--- a/src/PVE/AbstractConfig.pm
+++ b/src/PVE/AbstractConfig.pm
@@ -10,6 +10,7 @@ use PVE::Storage;
use PVE::Tools qw(lock_file lock_file_full);
use PVE::GuestHelpers qw(typesafe_ne);
+use PVE::GuestID;
use PVE::ReplicationConfig;
use PVE::Replication;
@@ -261,6 +262,8 @@ sub create_and_lock_config {
sub {
PVE::Cluster::check_vmid_unused($vmid, $allow_existing);
+ PVE::GuestID::register_used_id($vmid);
+
my $conf = eval { $class->load_config($vmid) } || {};
$class->check_lock($conf);
$conf->{lock} = $lock // 'create';
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH guest-common v8 08/22] guest id: keep used ID list below the pmxcfs file size limit
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (6 preceding siblings ...)
2026-10-12 15:45 ` [PATCH guest-common v8 07/22] abstract config: register used guest ID when creating config Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 09/22] tests: add tests for used-guest-ids max file size handling Michael Köppl
` (13 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
The used-guest-ids file can grow to more than 1 MiB if many
non-adjacent ID ranges or individual guest IDs are stored. While
unlikely to happen with regular operation, exceeding this limit on a
file in pmxcfs will cause every operation to fail, regardless of
whether the feature is turned on or not. To avoid this, add 2
thresholds: the first at 512 KiB, at which users are warned that their
ranges might be aggressively merged in the future, and the second at
768 KiB, at which this aggressive merging happens.
Once the second threshold is exceeded, the smallest gaps between ranges
will be merged until the file size is below the threshold again. This
will cause previously unused IDs to be recorded as used, resulting in
fewer available IDs overall.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/GuestID.pm | 112 ++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 101 insertions(+), 11 deletions(-)
diff --git a/src/PVE/GuestID.pm b/src/PVE/GuestID.pm
index 89219c8..0494913 100644
--- a/src/PVE/GuestID.pm
+++ b/src/PVE/GuestID.pm
@@ -11,6 +11,10 @@ use PVE::Cluster qw(
my $FILENAME = 'virtual-guest/used-guest-ids';
+# keep well below the 1 MiB file size limit of pmxcfs, writes fail beyond it
+my $SIZE_WARN = 512 * 1024;
+my $SIZE_MAX = 768 * 1024;
+
my sub parse_id_list($filename, $raw) {
my $ranges = [];
@@ -43,24 +47,110 @@ my sub format_entry($start, $last) {
return $start == $last ? "$start\n" : "$start-$last\n";
}
-my sub write_id_list($filename, $ranges) {
- my $output = '';
- my ($start, $last);
+my sub merge_ranges($ranges) {
+ my $merged = [];
for my $range ($ranges->@*) {
- my ($curr_start, $curr_end) = $range->@*;
+ my ($start, $end) = $range->@*;
+ my $current_merge = $merged->[-1];
- if (!defined($start)) {
- ($start, $last) = ($curr_start, $curr_end);
- } elsif ($curr_start <= $last + 1) {
- $last = $curr_end if $curr_end > $last;
+ if ($current_merge && $start <= $current_merge->[1] + 1) {
+ $current_merge->[1] = $end if $end > $current_merge->[1];
} else {
- $output .= format_entry($start, $last);
- ($start, $last) = ($curr_start, $curr_end);
+ push $merged->@*, [$start, $end];
}
}
- $output .= format_entry($start, $last) if defined($start);
+ return $merged;
+}
+
+my sub format_ranges($ranges) {
+ return join('', map { format_entry($_->@*) } $ranges->@*);
+}
+
+# Closes the smallest gaps between ranges in $merged until the
+# formatted list of $output_size bytes fits into $SIZE_MAX. The unused
+# IDs in a closed gap count as used from then on. Returns the resulting
+# ranges and the number of unused IDs that got marked as used.
+my sub close_smallest_gaps($merged, $output_size) {
+ my @gap_sizes;
+ for my $i (1 .. $#$merged) {
+ my $prev_end = $merged->[$i - 1]->[1];
+ my $curr_start = $merged->[$i]->[0];
+
+ $gap_sizes[$i] = $curr_start - $prev_end - 1;
+ }
+
+ my @gap_order = sort { $gap_sizes[$a] <=> $gap_sizes[$b] || $a <=> $b } 1 .. $#$merged;
+
+ # Record start and end of each group. Initially, with e.g. 6 ranges,
+ # this would be:
+ # start: (0, 1, 2, 3, 4, 5)
+ # end: (0, 1, 2, 3, 4, 5)
+ # After closing the gap between 1 and 2:
+ # start: (0, 1, 1, 3, 4, 5)
+ # end: (0, 2, 2, 3, 4, 5)
+ # After closing the gap between 2 and 3 (note that range 2 is now fully
+ # merged into adjacent ranges, so its group indices won't be accessed
+ # anymore):
+ # start: (0, 1, 1, 1, 4, 5)
+ # end: (0, 3, 2, 3, 4, 5)
+ # After closing the gap between 4 and 5:
+ # start: (0, 1, 1, 1, 4, 4)
+ # end: (0, 3, 2, 3, 5, 5)
+ my @group_start = (0 .. $#$merged);
+ my @group_end = (0 .. $#$merged);
+
+ my $closed_id_count = 0;
+
+ for my $i (@gap_order) {
+ last if $output_size <= $SIZE_MAX;
+
+ my ($first_index, $last_index) = ($group_start[$i - 1], $group_end[$i]);
+ my ($new_range_start, $new_range_end) =
+ ($merged->[$first_index]->[0], $merged->[$last_index]->[1]);
+ # used IDs directly before and after gap $i
+ my ($before_gap, $after_gap) = ($merged->[$i - 1]->[1], $merged->[$i]->[0]);
+
+ # the lines of both groups are replaced by a single line
+ # spanning them
+ $output_size +=
+ length(format_entry($new_range_start, $new_range_end)) -
+ length(format_entry($new_range_start, $before_gap)) -
+ length(format_entry($after_gap, $new_range_end));
+
+ $group_end[$first_index] = $last_index;
+ $group_start[$last_index] = $first_index;
+ $closed_id_count += $gap_sizes[$i];
+ }
+
+ my $closed = [];
+ for (my $i = 0; $i <= $#$merged; $i = $group_end[$i] + 1) {
+ push $closed->@*, [$merged->[$i]->[0], $merged->[$group_end[$i]]->[1]];
+ }
+
+ return ($closed, $closed_id_count);
+}
+
+my sub write_id_list($filename, $ranges) {
+ my $merged = merge_ranges($ranges);
+ my $output = format_ranges($merged);
+
+ if (length($output) > $SIZE_MAX) {
+ my ($closed, $closed_ids) = close_smallest_gaps($merged, length($output));
+ $output = format_ranges($closed);
+ warn "$filename would exceed "
+ . ($SIZE_MAX / 1024) . " KiB"
+ . ", recorded $closed_ids unused guest IDs in the smallest"
+ . " gaps as used\n";
+ } elsif (length($output) > $SIZE_WARN) {
+ warn "$filename is "
+ . int(length($output) / 1024) . " KiB"
+ . ", the smallest gaps between used guest IDs will be"
+ . " closed beyond "
+ . ($SIZE_MAX / 1024)
+ . " KiB\n";
+ }
return $output;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH guest-common v8 09/22] tests: add tests for used-guest-ids max file size handling
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (7 preceding siblings ...)
2026-10-12 15:45 ` [PATCH guest-common v8 08/22] guest id: keep used ID list below the pmxcfs file size limit Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 10/22] guest id: optionally enforce the next-id range and uniqueness Michael Köppl
` (12 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
The test cases cover whether a file exceeding the thresholds is
correctly detected and also cover multiple scenarios for merging ranges
with gaps between them until file size is below the threshold again.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/tests/guest-id-tests.pl | 96 +++++++++++++++++++++++++++++++++++++
1 file changed, 96 insertions(+)
diff --git a/src/tests/guest-id-tests.pl b/src/tests/guest-id-tests.pl
index 5cf796b..f5091af 100755
--- a/src/tests/guest-id-tests.pl
+++ b/src/tests/guest-id-tests.pl
@@ -4,11 +4,16 @@ use v5.36;
use lib qw(..);
+use List::Util qw(min sum0);
use Test::MockModule;
use Test::More;
my $FILE = '/etc/pve/virtual-guest/used-guest-ids';
+# must match the thresholds in PVE::GuestID
+my $SIZE_WARN = 512 * 1024;
+my $SIZE_MAX = 768 * 1024;
+
my ($parse, $write);
my $raw; # contents of the mocked used-guest-ids file
my $writes = 0;
@@ -44,6 +49,38 @@ sub range_to_vmlist(@ids) {
return { ids => { map { $_ => { node => 'localhost', type => 'qemu' } } @ids } };
}
+# single IDs from 100 upwards, with the gaps between them cycling through @gaps
+sub single_ids($count, @gaps) {
+ my @lines;
+ my $id = 100;
+ for my $i (0 .. $count - 1) {
+ push @lines, "$id\n";
+ $id += 1 + $gaps[$i % scalar(@gaps)];
+ }
+ return join('', @lines);
+}
+
+sub gap_sizes($ranges) {
+ my @gap_sizes;
+ for my $i (1 .. $#$ranges) {
+ push @gap_sizes, $ranges->[$i]->[0] - $ranges->[$i - 1]->[1] - 1;
+ }
+ return @gap_sizes;
+}
+
+# whether every range in $inner lies within a range in $outer, both sorted
+sub covers($outer, $inner) {
+ my $i = 0;
+ for my $range ($inner->@*) {
+ while ($i < scalar(@$outer) && $outer->[$i]->[1] < $range->[0]) {
+ $i++;
+ }
+ return 0 if $i >= scalar(@$outer);
+ return 0 if $outer->[$i]->[0] > $range->[0] || $outer->[$i]->[1] < $range->[1];
+ }
+ return 1;
+}
+
subtest 'parse' => sub {
my $tests = [
['canonical file', "100-250\n300\n900-1200\n", [[100, 250], [300, 300], [900, 1200]]],
@@ -109,6 +146,65 @@ subtest 'write' => sub {
}
};
+subtest 'write size limit' => sub {
+ my sub write_with_warnings($input) {
+ my @warnings;
+ local $SIG{__WARN__} = sub { push @warnings, $_[0] };
+ my $output = $write->($FILE, $parse->($FILE, $input));
+ return ($output, \@warnings);
+ }
+
+ my $small = single_ids(50_000, 1);
+ ok(length($small) < $SIZE_WARN, 'small list is below the warning threshold');
+ my ($output, $warnings) = write_with_warnings($small);
+ is($output, $small, 'below the warning threshold: written unchanged');
+ is_deeply($warnings, [], 'below the warning threshold: no warning');
+
+ my $medium = single_ids(90_000, 1);
+ ok(length($medium) > $SIZE_WARN && length($medium) < $SIZE_MAX,
+ 'medium list is in between');
+ ($output, $warnings) = write_with_warnings($medium);
+ is($output, $medium, 'above the warning threshold: written unchanged');
+ is(scalar($warnings->@*), 1, 'above the warning threshold: one warning');
+ like(
+ $warnings->[0],
+ qr/^\Q$FILE\E is \d+ KiB, .* closed beyond 768 KiB\n\z/,
+ 'above the warning threshold: size is warned about',
+ );
+
+ # above the hard limit, equal gaps
+ my $large = single_ids(150_000, 1);
+ ok(length($large) > $SIZE_MAX, 'large list is above the limit');
+ my $input = $parse->($FILE, $large);
+ ($output, $warnings) = write_with_warnings($large);
+ my $closed = $parse->($FILE, $output);
+ ok(length($output) <= $SIZE_MAX, 'above the limit: output fits');
+ ok(covers($closed, $input), 'above the limit: every used ID is still recorded');
+ is(scalar($warnings->@*), 1, 'above the limit: one warning');
+ my ($count) =
+ ($warnings->[0] // '') =~ m/would exceed 768 KiB, recorded (\d+) unused guest IDs? /;
+ is(
+ $count,
+ sum0(gap_sizes($input)) - sum0(gap_sizes($closed)),
+ 'above the limit: number of IDs marked as used is reported',
+ );
+
+ # above the hard limit, mixed gaps
+ my $mixed = single_ids(200_000, 1, 2, 5, 50);
+ $input = $parse->($FILE, $mixed);
+ ($output, $warnings) = write_with_warnings($mixed);
+ $closed = $parse->($FILE, $output);
+ my @remaining = gap_sizes($closed);
+ ok(length($output) <= $SIZE_MAX, 'mixed gaps: output fits');
+ ok(covers($closed, $input), 'mixed gaps: every used ID is still recorded');
+ is(min(@remaining), 5, 'mixed gaps: all gaps of 1 and 2 are closed first');
+ is(
+ scalar(grep { $_ == 50 } @remaining),
+ scalar(grep { $_ == 50 } gap_sizes($input)),
+ 'mixed gaps: the largest gaps are kept',
+ );
+};
+
subtest 'get_next_unused_id' => sub {
my $tests = [
# name, file contents, existing guests, input ID, expected ID
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH guest-common v8 10/22] guest id: optionally enforce the next-id range and uniqueness
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (8 preceding siblings ...)
2026-10-12 15:45 ` [PATCH guest-common v8 09/22] tests: add tests for used-guest-ids max file size handling Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 11/22] guest id: add helper to check guest IDs against next-id enforcement Michael Köppl
` (11 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
If the 'enforce' subproperty of next-id is enabled, ensure that only
guest IDs from the configured range and, if unique is enabled as well,
only previously unused IDs can be used. Actions on existing guests, such
as destroy, remote migration with --delete, and restore over an existing
guest, are always exempt from enforcement. A guest counts as existing if
its config is present in the cluster.
IDs are recorded before a guest exists and stay recorded if its
creation fails later. With both 'enforce' and 'unique' set, retrying a
failed creation with the same ID is therefore rejected.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/GuestID.pm | 32 +++++++++++++++++++++++++++-----
1 file changed, 27 insertions(+), 5 deletions(-)
diff --git a/src/PVE/GuestID.pm b/src/PVE/GuestID.pm
index 0494913..83876db 100644
--- a/src/PVE/GuestID.pm
+++ b/src/PVE/GuestID.pm
@@ -192,25 +192,47 @@ my sub insert_id($ranges, $id) {
return 1;
}
+my sub assert_id_in_next_id_range($next_id_opts, $id) {
+ my $lower = $next_id_opts->{lower};
+ my $upper = $next_id_opts->{upper};
+
+ die "guest ID $id is below the lower boundary $lower of the next-id range\n"
+ if defined($lower) && $id < $lower;
+ die "guest ID $id is not below the upper boundary $upper of the next-id range\n"
+ if defined($upper) && $id >= $upper;
+}
+
+# Records $id as used. If the next-id datacenter option enforces its
+# range or uniqueness, IDs a new guest must not use are rejected.
+# IDs of currently existing guests are never rejected.
sub register_used_id($id) {
- cfs_lock_file(
+ my $next_id_opts = cfs_read_file('datacenter.cfg')->{'next-id'} // {};
+ my $enforce = $next_id_opts->{enforce} && PVE::Cluster::check_vmid_unused($id, 1);
+
+ assert_id_in_next_id_range($next_id_opts, $id) if $enforce;
+
+ my $recorded = cfs_lock_file(
$FILENAME,
10,
sub {
my $ranges = cfs_read_file($FILENAME);
- return if !insert_id($ranges, $id);
+ return 0 if !insert_id($ranges, $id);
cfs_write_file($FILENAME, $ranges);
+ return 1;
},
);
if (my $err = $@) {
- my $dc_conf = cfs_read_file('datacenter.cfg');
-
my $emsg = "unable to record guest ID $id as used";
- die "$emsg - $err" if $dc_conf->{'next-id'}->{unique};
+ die "$emsg - $err" if $next_id_opts->{unique};
warn "$emsg - $err";
+
+ return;
}
+
+ die "guest ID $id was used before\n"
+ if $enforce && $next_id_opts->{unique} && !$recorded;
}
cfs_register_file($FILENAME, \&parse_id_list, \&write_id_list);
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH guest-common v8 11/22] guest id: add helper to check guest IDs against next-id enforcement
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (9 preceding siblings ...)
2026-10-12 15:45 ` [PATCH guest-common v8 10/22] guest id: optionally enforce the next-id range and uniqueness Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH qemu-server v8 12/22] api: record VM ID as used on destruction and remote migration Michael Köppl
` (10 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
With 'enforce' set in the next-id datacenter option, creating a guest
with an ID outside the configured range or, with 'unique', an ID that
was used before fails when the ID is registered. That only happens once
the creation task is already running.
Add a helper that runs the same checks without recording the ID, so that
callers can reject such IDs early. The helper reads the list of used
IDs without taking the lock, so another node may record the ID in
between. register_used_id() still checks it under the lock and remains
the authoritative check.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/GuestID.pm | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/src/PVE/GuestID.pm b/src/PVE/GuestID.pm
index 83876db..a7e93a5 100644
--- a/src/PVE/GuestID.pm
+++ b/src/PVE/GuestID.pm
@@ -202,6 +202,19 @@ my sub assert_id_in_next_id_range($next_id_opts, $id) {
if defined($upper) && $id >= $upper;
}
+# Dies if the next-id datacenter option enforces its range or uniqueness
+# and a new guest must not use $id.
+sub assert_id_satisfies_next_id_settings($id) {
+ my $next_id_opts = cfs_read_file('datacenter.cfg')->{'next-id'} // {};
+ return if !$next_id_opts->{enforce};
+
+ assert_id_in_next_id_range($next_id_opts, $id);
+
+ if ($next_id_opts->{unique}) {
+ die "guest ID $id was used before\n" if get_next_unused_id($id) != $id;
+ }
+}
+
# Records $id as used. If the next-id datacenter option enforces its
# range or uniqueness, IDs a new guest must not use are rejected.
# IDs of currently existing guests are never rejected.
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH qemu-server v8 12/22] api: record VM ID as used on destruction and remote migration
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (10 preceding siblings ...)
2026-10-12 15:45 ` [PATCH guest-common v8 11/22] guest id: add helper to check guest IDs against next-id enforcement Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH container v8 13/22] api: record CT " Michael Köppl
` (9 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
Record the ID of a VM as used when it is destroyed, cloned or removed
after a remote migration with --delete, so that the `/cluster/nextid`
endpoint can later optionally avoid suggesting previously used IDs.
For creation, restore and incoming remote migrations the ID is
recorded by create_and_lock_config(). VM clones write the new config
directly instead, so they record it explicitly. Recording on
destruction as well covers guests that were created before IDs were
tracked. The same holds for a remote migration with --delete, which
removes the source guest without going through the destroy API. This
call explicitly only warns on failure, since the guest has already
moved.
With 'unique' set in the next-id datacenter option, a failure to
record aborts create, clone and destroy before anything is changed.
Otherwise it only warns.
Originally-by: Daniel Krambrock <krambrock@hrz.uni-marburg.de>
Originally-by: Severen Redwood <severen.redwood@sitehost.co.nz>
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/API2/Qemu.pm | 9 +++++++++
src/PVE/QemuMigrate.pm | 4 ++++
2 files changed, 13 insertions(+)
diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index cfbc1f51..5c64dcc6 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -25,6 +25,7 @@ use PVE::JSONSchema qw(get_standard_option);
use PVE::RESTHandler;
use PVE::ReplicationConfig;
use PVE::GuestHelpers qw(assert_tag_permissions);
+use PVE::GuestID;
use PVE::GuestImport;
use PVE::QemuConfig;
use PVE::QemuServer;
@@ -2878,6 +2879,10 @@ __PACKAGE__->register_method({
$early_checks->();
+ # record before destroying anything, so a failure here leaves the VM intact
+ eval { PVE::GuestID::register_used_id($vmid) };
+ die "unable to destroy VM $vmid - $@" if $@;
+
my $realcmd = sub {
my $upid = shift;
@@ -4641,6 +4646,10 @@ __PACKAGE__->register_method({
$newconf->{description} = $param->{description};
}
+ # record before the config exists, so a failure here leaves nothing behind
+ eval { PVE::GuestID::register_used_id($newid) };
+ die "unable to clone VM $vmid - $@" if $@;
+
# create empty/temp config - this fails if VM already exists on other node
# FIXME use PVE::QemuConfig->create_and_lock_config and adapt code
PVE::Tools::file_set_contents($conffile, "# qmclone temporary file\nlock: clone\n");
diff --git a/src/PVE/QemuMigrate.pm b/src/PVE/QemuMigrate.pm
index 6605f5d8..c083a20a 100644
--- a/src/PVE/QemuMigrate.pm
+++ b/src/PVE/QemuMigrate.pm
@@ -13,6 +13,7 @@ use PVE::Cluster;
use PVE::Format qw(render_bytes);
use PVE::Firewall::Helpers;
use PVE::GuestHelpers qw(safe_boolean_ne safe_string_ne);
+use PVE::GuestID;
use PVE::INotify;
use PVE::JSONSchema;
use PVE::RPCEnvironment;
@@ -1905,6 +1906,9 @@ sub phase3_cleanup {
}
if ($self->{opts}->{remote} && $self->{opts}->{delete}) {
+ eval { PVE::GuestID::register_used_id($vmid) };
+ warn $@ if $@;
+
eval { PVE::QemuServer::destroy_vm($self->{storecfg}, $vmid, 1, undef, 0) };
warn "Failed to remove source VM - $@\n" if $@;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH container v8 13/22] api: record CT ID as used on destruction and remote migration
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (11 preceding siblings ...)
2026-10-12 15:45 ` [PATCH qemu-server v8 12/22] api: record VM ID as used on destruction and remote migration Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 14/22] ui: guest ID selector: rename exists flag to rejected Michael Köppl
` (8 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
Record the ID of a CT as used when it is destroyed or removed after a
remote migration with --delete, so that the `/cluster/nextid` endpoint
can later optionally avoid suggesting previously used IDs.
For creation, restore and incoming remote migrations the ID is
recorded by create_and_lock_config(). Recording on destruction as well
covers guests that were created before IDs were tracked. The same holds
for a remote migration with --delete, which removes the source guest
without going through the destroy API; that call only warns on failure,
since the guest has already moved.
With 'unique' set in the next-id datacenter option, a failure to
record aborts create, clone and destroy before anything is changed.
Otherwise it only warns.
Originally-by: Daniel Krambrock <krambrock@hrz.uni-marburg.de>
Originally-by: Severen Redwood <severen.redwood@sitehost.co.nz>
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
src/PVE/API2/LXC.pm | 5 +++++
src/PVE/LXC/Migrate.pm | 4 ++++
2 files changed, 9 insertions(+)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 5f94d5a1..fb4262c9 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -14,6 +14,7 @@ use PVE::DataCenterConfig;
use PVE::Exception qw(raise raise_param_exc raise_perm_exc);
use PVE::Firewall;
use PVE::GuestHelpers;
+use PVE::GuestID;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
use PVE::RESTHandler;
@@ -882,6 +883,10 @@ __PACKAGE__->register_method({
$early_checks->($conf);
+ # record before destroying anything, so a failure here leaves the CT intact
+ eval { PVE::GuestID::register_used_id($vmid) };
+ die "unable to destroy CT $vmid - $@" if $@;
+
my $running_error_msg = "unable to destroy CT $vmid - container is running\n";
die $running_error_msg if !$param->{force} && PVE::LXC::check_running($vmid); # check early
diff --git a/src/PVE/LXC/Migrate.pm b/src/PVE/LXC/Migrate.pm
index d243d901..a215dede 100644
--- a/src/PVE/LXC/Migrate.pm
+++ b/src/PVE/LXC/Migrate.pm
@@ -7,6 +7,7 @@ use File::Basename;
use File::Copy; # fixme: remove
use PVE::Cluster;
+use PVE::GuestID;
use PVE::INotify;
use PVE::JSONSchema;
use PVE::Replication;
@@ -541,6 +542,9 @@ sub final_cleanup {
PVE::Tunnel::write_tunnel($self->{tunnel}, 60, 'start');
}
if ($self->{opts}->{delete}) {
+ eval { PVE::GuestID::register_used_id($vmid) };
+ warn $@ if $@;
+
PVE::LXC::destroy_lxc_container(
PVE::Storage::config(),
$vmid,
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH manager v8 14/22] ui: guest ID selector: rename exists flag to rejected
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (12 preceding siblings ...)
2026-10-12 15:45 ` [PATCH container v8 13/22] api: record CT " Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 15/22] ui: guest ID selector: show the error returned by nextid Michael Köppl
` (7 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
The flag is set whenever the /cluster/nextid endpoint rejects an ID,
which includes failed requests, so it does not reliably tell whether a
guest with that ID exists. Name it after what it actually records.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
www/manager6/form/GuestIDSelector.js | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/www/manager6/form/GuestIDSelector.js b/www/manager6/form/GuestIDSelector.js
index e440cf7ae..f3160a744 100644
--- a/www/manager6/form/GuestIDSelector.js
+++ b/www/manager6/form/GuestIDSelector.js
@@ -22,11 +22,11 @@ Ext.define('PVE.form.GuestIDSelector', {
return true;
}
- if (me.validateExists === true && !me.exists) {
+ if (me.validateExists === true && !me.rejected) {
return me.unknownID;
}
- if (me.validateExists === false && me.exists) {
+ if (me.validateExists === false && me.rejected) {
return me.inUseID;
}
@@ -62,11 +62,11 @@ Ext.define('PVE.form.GuestIDSelector', {
url: '/cluster/nextid',
method: 'GET',
success: function (response, opts) {
- me.exists = false;
+ me.rejected = false;
me.validate();
},
failure: function (response, opts) {
- me.exists = true;
+ me.rejected = true;
me.validate();
},
});
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH manager v8 15/22] ui: guest ID selector: show the error returned by nextid
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (13 preceding siblings ...)
2026-10-12 15:45 ` [PATCH manager v8 14/22] ui: guest ID selector: rename exists flag to rejected Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 16/22] fix #4369: api: optionally only suggest unique IDs Michael Köppl
` (6 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
The guest ID field validates IDs through the /cluster/nextid endpoint
and reported any failure, including failed requests, as the ID being
in use. Show the error the endpoint returns instead, so that the
actual reason for rejecting an ID is visible. This is also needed for
the endpoint to reject IDs that are not in use, but forbidden by the
next-id datacenter option.
The endpoint's errors are not translated, so keep showing the translated
message that the ID is already in use if the guest is known to the UI.
This leaves the endpoint's error for IDs rejected by next-id enforcement
and for guests the user cannot see.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
www/manager6/form/GuestIDSelector.js | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/www/manager6/form/GuestIDSelector.js b/www/manager6/form/GuestIDSelector.js
index f3160a744..a7cf421ac 100644
--- a/www/manager6/form/GuestIDSelector.js
+++ b/www/manager6/form/GuestIDSelector.js
@@ -27,7 +27,7 @@ Ext.define('PVE.form.GuestIDSelector', {
}
if (me.validateExists === false && me.rejected) {
- return me.inUseID;
+ return me.nextIDError;
}
return true;
@@ -67,6 +67,15 @@ Ext.define('PVE.form.GuestIDSelector', {
},
failure: function (response, opts) {
me.rejected = true;
+ // prefer translated message if guest is known to the UI
+ if (PVE.data.ResourceStore.findVMID(newValue)) {
+ me.nextIDError = me.inUseID;
+ } else {
+ let error = response.result?.errors?.vmid;
+ me.nextIDError = error
+ ? Ext.htmlEncode(error.trim())
+ : response.htmlStatus;
+ }
me.validate();
},
});
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH manager v8 16/22] fix #4369: api: optionally only suggest unique IDs
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (14 preceding siblings ...)
2026-10-12 15:45 ` [PATCH manager v8 15/22] ui: guest ID selector: show the error returned by nextid Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 17/22] ui: dc options: rename VMID to guest ID Michael Köppl
` (5 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
At the moment, the `/cluster/nextid` API endpoint will return the lowest
available VM/CT ID, which means that it will suggest re-using VM IDs.
This can be undesirable, so add an optional check to ensure that it
chooses an ID which is not and has never been in use.
This optional behaviour is enabled with unique=1 in the next-id
datacenter option and the previously used IDs are tracked as a list in
the file `/etc/pve/virtual-guest/used-guest-ids`.
Originally-by: Daniel Krambrock <krambrock@hrz.uni-marburg.de>
Originally-by: Severen Redwood <severen.redwood@sitehost.co.nz>
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
PVE/API2/Cluster.pm | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index 4e5efbfd9..e5017f6c4 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -11,6 +11,7 @@ use PVE::DataCenterConfig;
use PVE::Exception qw(raise_param_exc);
use PVE::Firewall;
use PVE::GuestHelpers;
+use PVE::GuestID;
use PVE::HA::Config;
use PVE::HA::Env::PVE2;
use PVE::INotify;
@@ -1045,8 +1046,13 @@ __PACKAGE__->register_method({
my $lower = $next_id->{lower} // 100;
my $upper = $next_id->{upper} // (1000 * 1000); # note, lower than the schema-maximum
- for (my $i = $lower; $i < $upper; $i++) {
- return $i if !defined($idlist->{$i});
+ if ($next_id->{unique}) {
+ my $id = PVE::GuestID::get_next_unused_id($lower);
+ return $id if $id < $upper;
+ } else {
+ for (my $i = $lower; $i < $upper; $i++) {
+ return $i if !defined($idlist->{$i});
+ }
}
die "unable to get any free VMID in range [$lower, $upper]\n";
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH manager v8 17/22] ui: dc options: rename VMID to guest ID
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (15 preceding siblings ...)
2026-10-12 15:45 ` [PATCH manager v8 16/22] fix #4369: api: optionally only suggest unique IDs Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 18/22] fix #4369: ui: dc options: add option for unique VM/CT IDs Michael Köppl
` (4 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
To make clear that this is for both VMs and CTs, use the more general
term "guest ID".
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
www/manager6/dc/OptionView.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/www/manager6/dc/OptionView.js b/www/manager6/dc/OptionView.js
index dc12aa7e1..b4291afd3 100644
--- a/www/manager6/dc/OptionView.js
+++ b/www/manager6/dc/OptionView.js
@@ -360,7 +360,7 @@ Ext.define('PVE.dc.OptionView', {
minValue: 1,
maxValue: 64, // arbitrary but generous limit as limits are good
});
- me.add_inputpanel_row('next-id', gettext('Next Free VMID Range'), {
+ me.add_inputpanel_row('next-id', gettext('Next Free Guest ID Range'), {
renderer: PVE.Utils.render_as_property_string,
url: '/api2/extjs/cluster/options',
items: [
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH manager v8 18/22] fix #4369: ui: dc options: add option for unique VM/CT IDs
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (16 preceding siblings ...)
2026-10-12 15:45 ` [PATCH manager v8 17/22] ui: dc options: rename VMID to guest ID Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 19/22] api: nextid: reject IDs forbidden by next-id enforcement Michael Köppl
` (3 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
Add an additional checkbox to the "Next Free Guest ID" option that
allows choosing whether the `/cluster/nextid` API endpoint (and thereby
any UI elements that suggest IDs) should avoid suggesting previously
used IDs. This option defaults to off to ensure that this change in
behaviour is opt in.
Also rename the top-level option to "Next Free Guest ID" since it's no
longer just the range.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
www/manager6/dc/OptionView.js | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/www/manager6/dc/OptionView.js b/www/manager6/dc/OptionView.js
index b4291afd3..97f54cda1 100644
--- a/www/manager6/dc/OptionView.js
+++ b/www/manager6/dc/OptionView.js
@@ -360,7 +360,7 @@ Ext.define('PVE.dc.OptionView', {
minValue: 1,
maxValue: 64, // arbitrary but generous limit as limits are good
});
- me.add_inputpanel_row('next-id', gettext('Next Free Guest ID Range'), {
+ me.add_inputpanel_row('next-id', gettext('Next Free Guest ID'), {
renderer: PVE.Utils.render_as_property_string,
url: '/api2/extjs/cluster/options',
items: [
@@ -382,6 +382,13 @@ Ext.define('PVE.dc.OptionView', {
maxValue: 1000 * 1000 * 1000 - 1,
submitValue: true,
},
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'unique',
+ fieldLabel: gettext('Unique'),
+ boxLabel: gettext('Only suggest guest IDs that were never used'),
+ value: 0,
+ },
],
});
me.rows['tag-style'] = {
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH manager v8 19/22] api: nextid: reject IDs forbidden by next-id enforcement
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (17 preceding siblings ...)
2026-10-12 15:45 ` [PATCH manager v8 18/22] fix #4369: ui: dc options: add option for unique VM/CT IDs Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 20/22] ui: dc options: add option to enforce next free guest ID settings Michael Köppl
` (2 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
With 'enforce' set in the next-id datacenter option, new guests may
only use IDs within the configured range and, with 'unique', IDs that
were not used before. Creating a guest with any other ID fails anyway,
but the GUI validates its guest ID field through this endpoint, so
reject such IDs here as well to report the problem before a task is
started.
Since the endpoint serves both VMs and CTs and its error is now shown
in the GUI, also change the message for IDs of existing guests to
refer to guest IDs.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
PVE/API2/Cluster.pm | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index e5017f6c4..c487980a4 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -1036,8 +1036,13 @@ __PACKAGE__->register_method({
my $idlist = $vmlist->{ids} || {};
if (my $vmid = $param->{vmid}) {
- return $vmid if !defined($idlist->{$vmid});
- raise_param_exc({ vmid => "VM $vmid already exists" });
+ raise_param_exc({ vmid => "guest ID $vmid is already in use" })
+ if defined($idlist->{$vmid});
+
+ eval { PVE::GuestID::assert_id_satisfies_next_id_settings($vmid) };
+ raise_param_exc({ vmid => $@ }) if $@;
+
+ return $vmid;
}
my $dc_conf = PVE::Cluster::cfs_read_file('datacenter.cfg');
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH manager v8 20/22] ui: dc options: add option to enforce next free guest ID settings
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (18 preceding siblings ...)
2026-10-12 15:45 ` [PATCH manager v8 19/22] api: nextid: reject IDs forbidden by next-id enforcement Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:45 ` [PATCH docs v8 21/22] pmxcfs: files: add virtual-guest/used-guest-ids Michael Köppl
2026-10-12 15:46 ` [PATCH docs v8 22/22] pvecm: next-id: document unique and enforce options Michael Köppl
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
Add an 'Enforce' checkbox to the "Next Free Guest ID" option. With it
set, the range and, if 'unique' is set as well, never having been used
are required for the IDs of new guests, instead of only being used to
suggest IDs.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
www/manager6/dc/OptionView.js | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/www/manager6/dc/OptionView.js b/www/manager6/dc/OptionView.js
index 97f54cda1..72680b2d6 100644
--- a/www/manager6/dc/OptionView.js
+++ b/www/manager6/dc/OptionView.js
@@ -389,6 +389,15 @@ Ext.define('PVE.dc.OptionView', {
boxLabel: gettext('Only suggest guest IDs that were never used'),
value: 0,
},
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'enforce',
+ fieldLabel: gettext('Enforce'),
+ boxLabel: gettext(
+ 'Reject guest IDs outside the range or, with unique, guest IDs that were used before',
+ ),
+ value: 0,
+ },
],
});
me.rows['tag-style'] = {
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH docs v8 21/22] pmxcfs: files: add virtual-guest/used-guest-ids
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (19 preceding siblings ...)
2026-10-12 15:45 ` [PATCH manager v8 20/22] ui: dc options: add option to enforce next free guest ID settings Michael Köppl
@ 2026-10-12 15:45 ` Michael Köppl
2026-10-12 15:46 ` [PATCH docs v8 22/22] pvecm: next-id: document unique and enforce options Michael Köppl
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:45 UTC (permalink / raw)
To: pve-devel
The file records the IDs of all guests that have been in use, so that
the next-id datacenter option can optionally avoid their reuse.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
pmxcfs.adoc | 1 +
1 file changed, 1 insertion(+)
diff --git a/pmxcfs.adoc b/pmxcfs.adoc
index 4a334fde..11284252 100644
--- a/pmxcfs.adoc
+++ b/pmxcfs.adoc
@@ -130,6 +130,7 @@ Files
|`storage.cfg` | {pve} storage configuration
|`user.cfg` | {pve} access control configuration (users/groups/...)
|`virtual-guest/cpu-models.conf` | For storing custom CPU models
+|`virtual-guest/used-guest-ids` | Guest IDs that have been in use, for optionally avoiding their reuse
|`vzdump.cron` | Cluster-wide vzdump backup-job schedule
|=======
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH docs v8 22/22] pvecm: next-id: document unique and enforce options
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
` (20 preceding siblings ...)
2026-10-12 15:45 ` [PATCH docs v8 21/22] pmxcfs: files: add virtual-guest/used-guest-ids Michael Köppl
@ 2026-10-12 15:46 ` Michael Köppl
21 siblings, 0 replies; 23+ messages in thread
From: Michael Köppl @ 2026-10-12 15:46 UTC (permalink / raw)
To: pve-devel
The next-id datacenter option can now optionally only suggest guest IDs
that have never been in use (through 'unique') and turn its range and
uniqueness into a hard limit for new guests (through 'enforce').
Describe both options, why avoiding the reuse of guest IDs can be
desirable, and how the list of used IDs is kept small.
Drop the note that the range is not a hard limit since that no longer
holds with 'enforce' set.
Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
---
pvecm.adoc | 33 +++++++++++++++++++++++++++++++--
1 file changed, 31 insertions(+), 2 deletions(-)
diff --git a/pvecm.adoc b/pvecm.adoc
index 8309647a..cd5a0fb8 100644
--- a/pvecm.adoc
+++ b/pvecm.adoc
@@ -1566,8 +1566,37 @@ To accommodate this use case one can set either lower, upper or both boundaries
via the `datacenter.cfg` configuration file, which can be edited in the web
interface under 'Datacenter' -> 'Options'.
-NOTE: The range is only used for the next-id API call, so it isn't a hard
-limit.
+By default, the lowest free VMID in the range is suggested, so the VMIDs of
+destroyed guests get reused. This can be undesirable, for example, if external
+backup or monitoring systems identify guests by their VMID. With the `unique`
+option set, only VMIDs that have never been in use are suggested. For this,
+{pve} records the VMID of every guest on creation and destruction in
+`/etc/pve/virtual-guest/used-guest-ids`, independent of whether `unique` is
+set. VMIDs of guests that were removed before {pve} started recording them are
+not known and may still be suggested.
+
+The range and the `unique` option only affect which VMID is suggested. Any
+other free VMID can still be chosen manually. With the `enforce` option set,
+they become a hard limit instead. New guests can then only use VMIDs from the
+range and, if `unique` is set as well, only VMIDs that have never been in use.
+Operations on existing guests, like destroying them or restoring a backup over
+them, are not restricted.
+
+----
+next-id: lower=10000,upper=20000,unique=1,enforce=1
+----
+
+NOTE: Consecutive VMIDs are stored as a single range, so the list of used VMIDs
+usually stays small. It only grows large if many VMIDs with gaps between them
+are used. If the list reaches 512 KiB, a warning is logged whenever it is
+updated. The list shrinks again as gaps get filled, since the ranges on both
+sides of a gap are merged into a single entry once all VMIDs in it have been
+used. With `unique` set, auto-selection suggests the lowest VMID that has never
+been used and thereby fills the gaps from the bottom up. To stay within the file
+size limit of the cluster file system, the smallest gaps between recorded VMIDs
+are marked as used once the list would exceed 768 KiB. As a result, they are no
+longer suggested with `unique` set and cannot be used for new guests anymore
+with `enforce` set as well.
Guest Migration
---------------
--
2.47.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
end of thread, other threads:[~2026-10-12 15:54 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-12 15:45 [PATCH many v8 00/22] add option to prevent suggesting previously used VMIDs Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 01/22] cluster files: add virtual-guest/used-guest-ids Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 02/22] datacenter config: add unique subproperty to next-id Michael Köppl
2026-10-12 15:45 ` [PATCH cluster v8 03/22] datacenter config: next-id: add enforce subproperty Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 04/22] src/makefile: order files alphabetically Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 05/22] add module to track previously used guest IDs Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 06/22] tests: add tests for used guest ID tracking Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 07/22] abstract config: register used guest ID when creating config Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 08/22] guest id: keep used ID list below the pmxcfs file size limit Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 09/22] tests: add tests for used-guest-ids max file size handling Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 10/22] guest id: optionally enforce the next-id range and uniqueness Michael Köppl
2026-10-12 15:45 ` [PATCH guest-common v8 11/22] guest id: add helper to check guest IDs against next-id enforcement Michael Köppl
2026-10-12 15:45 ` [PATCH qemu-server v8 12/22] api: record VM ID as used on destruction and remote migration Michael Köppl
2026-10-12 15:45 ` [PATCH container v8 13/22] api: record CT " Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 14/22] ui: guest ID selector: rename exists flag to rejected Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 15/22] ui: guest ID selector: show the error returned by nextid Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 16/22] fix #4369: api: optionally only suggest unique IDs Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 17/22] ui: dc options: rename VMID to guest ID Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 18/22] fix #4369: ui: dc options: add option for unique VM/CT IDs Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 19/22] api: nextid: reject IDs forbidden by next-id enforcement Michael Köppl
2026-10-12 15:45 ` [PATCH manager v8 20/22] ui: dc options: add option to enforce next free guest ID settings Michael Köppl
2026-10-12 15:45 ` [PATCH docs v8 21/22] pmxcfs: files: add virtual-guest/used-guest-ids Michael Köppl
2026-10-12 15:46 ` [PATCH docs v8 22/22] pvecm: next-id: document unique and enforce options Michael Köppl
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox