From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 54B2E1FF15E for ; Fri, 4 Oct 2024 07:10:25 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 788A937FE5; Fri, 4 Oct 2024 07:10:46 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Fri, 4 Oct 2024 18:07:40 +1300 In-Reply-To: <20241004050957.441759-1-severen.redwood@sitehost.co.nz> References: <20241004050957.441759-1-severen.redwood@sitehost.co.nz> MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Severen Redwood via pve-devel Precedence: list Cc: Severen Redwood , Thomas Lamprecht X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH manager v2 1/2] close #4369: api: optionally only suggest unique IDs Content-Type: multipart/mixed; boundary="===============1460630761919870917==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============1460630761919870917== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id CC7F3C27D6 for ; Fri, 4 Oct 2024 07:10:44 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 9B18B37FC6 for ; Fri, 4 Oct 2024 07:10:44 +0200 (CEST) Received: from mx3.ext.sitehost.co.nz (mx3.ext.sitehost.co.nz [120.138.20.239]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 4 Oct 2024 07:10:43 +0200 (CEST) Received: from localhost (extmx1-new.vps.sitehost.co.nz [127.0.0.1]) by mx3.ext.sitehost.co.nz (Postfix) with ESMTP id 6CEEC180B8B; Fri, 4 Oct 2024 18:10:39 +1300 (NZDT) X-Virus-Scanned: SiteHost Virus/Spam Prevention on mx3.ext.sitehost.co.nz X-Spam-Flag: NO X-Spam-Score: -3.449 X-Spam-Status: No, score=-3.449 tagged_above=-100 required=5 tests=[ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.55, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mx3.ext.sitehost.co.nz ([127.0.0.1]) by localhost (mx3.ext.sitehost.co.nz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7e4otOjuCf9Q; Fri, 4 Oct 2024 18:10:34 +1300 (NZDT) Received: from yggdrasill.. (oep.air.sitehost.co.nz [120.138.16.30]) by mx3.ext.sitehost.co.nz (Postfix) with ESMTPSA id A9865180A2F; Fri, 4 Oct 2024 18:10:32 +1300 (NZDT) From: Severen Redwood To: pve-devel@lists.proxmox.com Cc: Thomas Lamprecht , Dietmar Maurer , Severen Redwood , Daniel Krambrock Subject: [PATCH manager v2 1/2] close #4369: api: optionally only suggest unique IDs Date: Fri, 4 Oct 2024 18:07:40 +1300 Message-ID: <20241004050957.441759-2-severen.redwood@sitehost.co.nz> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241004050957.441759-1-severen.redwood@sitehost.co.nz> References: <20241004050957.441759-1-severen.redwood@sitehost.co.nz> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.149 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_PASS -0.1 DMARC pass policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record At the moment, the `/cluster/nextid` API endpoint will return the lowest available VM/CT ID, which means that it will suggest re-using VM IDs. This can be undesirable, so add an optional check to ensure that it chooses an ID which is not and has never been in use. This optional behaviour is enabled when `unique-next-id: 1` in the data centre config, and the previously used IDs are tracked as a list in the file `/etc/pve/used_vmids.list`. Co-authored-by: Daniel Krambrock Signed-off-by: Severen Redwood --- PVE/API2/Cluster.pm | 13 +++++++-- PVE/Makefile | 1 + PVE/UsedVmidList.pm | 70 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 82 insertions(+), 2 deletions(-) create mode 100644 PVE/UsedVmidList.pm diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm index 04387ab4..d0b02d78 100644 --- a/PVE/API2/Cluster.pm +++ b/PVE/API2/Cluster.pm @@ -20,6 +20,7 @@ use PVE::RPCEnvironment; use PVE::SafeSyslog; use PVE::Storage; use PVE::Tools qw(extract_param); +use PVE::UsedVmidList; use PVE::API2::ACMEAccount; use PVE::API2::ACMEPlugin; @@ -813,12 +814,20 @@ __PACKAGE__->register_method({ my $dc_conf = PVE::Cluster::cfs_read_file('datacenter.cfg'); my $next_id = $dc_conf->{'next-id'} // {}; + my $want_unique = $dc_conf->{'unique-next-id'} // 0; my $lower = $next_id->{lower} // 100; my $upper = $next_id->{upper} // (1000 * 1000); # note, lower than the schema-maximum - for (my $i = $lower; $i < $upper; $i++) { - return $i if !defined($idlist->{$i}); + if ($want_unique) { + my $used_ids = PVE::Cluster::cfs_read_file('used_vmids.list'); + for (my $i = $lower; $i < $upper; $i++) { + return $i if !defined($idlist->{$i}) and !defined($used_ids->{$i}); + } + } else { + for (my $i = $lower; $i < $upper; $i++) { + return $i if !defined($idlist->{$i}); + } } die "unable to get any free VMID in range [$lower, $upper]\n"; diff --git a/PVE/Makefile b/PVE/Makefile index efcb250d..29775e78 100644 --- a/PVE/Makefile +++ b/PVE/Makefile @@ -15,6 +15,7 @@ PERLSOURCE = \ NodeConfig.pm \ PullMetric.pm \ Report.pm \ + UsedVmidList.pm \ VZDump.pm all: pvecfg.pm $(SUBDIRS) diff --git a/PVE/UsedVmidList.pm b/PVE/UsedVmidList.pm new file mode 100644 index 00000000..b88a8681 --- /dev/null +++ b/PVE/UsedVmidList.pm @@ -0,0 +1,70 @@ +package PVE::UsedVmidList; + +use strict; +use warnings; + +use PVE::Cluster; + +my $read_id_list = sub { + my ($filename, $raw) = @_; + + return {} if !defined($raw); + + my %used_ids; + my @lines = split(/\n/, $raw); + foreach my $line (@lines) { + if ($line =~ m/^(\d+)$/) { + $used_ids{$1} = 1; + } elsif ($line =~ m/^(\d+)-(\d+)$/) { + foreach my $id ($1..$2) { + $used_ids{$id} = 1; + } + } else { + warn "Skipping invalid entry in used_vmids.list: $line\n"; + } + } + + return \%used_ids; +}; + +my $write_id_list = sub { + my ($filename, $used_ids) = @_; + my @used_ids = sort {$a <=> $b} keys(%$used_ids); + + my @lines; + my $len = scalar(@used_ids); + for (my $i = 0; $i < $len; $i++) { + my $line = "$used_ids[$i]"; + + my $j = $i; + while ($j + 1 < $len and $used_ids[$j] + 1 == $used_ids[$j + 1]) { + $j++; + } + + # If we find a range of consecutive IDs, write $ids[$i]-$ids[$j] to + # denote the range so that we avoid storing each individual integer. + if ($i != $j) { + $line .= "-$used_ids[$j]"; + } + + $i = $j; + push(@lines, $line); + } + + return join("\n", @lines) . "\n"; +}; + +PVE::Cluster::cfs_register_file('used_vmids.list', $read_id_list, $write_id_list); + +sub add_vmid { + my ($vmid) = @_; + + PVE::Cluster::cfs_lock_file('used_vmids.list', 10, sub { + my $used_ids = PVE::Cluster::cfs_read_file('used_vmids.list'); + + $used_ids->{$vmid} = 1; + PVE::Cluster::cfs_write_file('used_vmids.list', $used_ids); + }); +} + +1; -- 2.46.2 --===============1460630761919870917== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============1460630761919870917==--