From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id B25DB1FF164
for <inbox@lore.proxmox.com>; Fri, 20 Jun 2025 16:34:25 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 3E4E8E432;
Fri, 20 Jun 2025 16:32:44 +0200 (CEST)
From: Daniel Kral <d.kral@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Fri, 20 Jun 2025 16:31:23 +0200
Message-Id: <20250620143148.218469-16-d.kral@proxmox.com>
X-Mailer: git-send-email 2.39.5
In-Reply-To: <20250620143148.218469-1-d.kral@proxmox.com>
References: <20250620143148.218469-1-d.kral@proxmox.com>
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.011 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH ha-manager v2 11/26] manager: migrate ha groups
to location rules in-memory
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Migrate the currently configured HA groups to HA Location rules
in-memory if the use-location-rules feature flag isn't set, so that they
can be applied as such in the next patches and therefore replace HA
groups internally.
Also ignore location rules written to the rules config if the
use-location-rules isn't set, because these should be mutually exclusive
so that users can only use HA groups or HA location rules.
Location rules in their initial implementation are designed to be as
restrictive as HA groups, i.e. only allow a service to be used in a
single location rule, to ease the migration between them.
HA groups map directly to location rules, except that the 'restricted'
property is renamed to 'strict', so the name is the same as for
colocation rules, and that the 'failback' property is moved to the
service config.
The 'nofailback' property is moved to the service config, because it
allows users to set it more granularly for individual services and
allows the location rules to be more extendible in the future, e.g.
multiple location rules for a single service.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
changes since v1:
- NEW!
src/PVE/HA/Config.pm | 3 ++-
src/PVE/HA/Groups.pm | 48 ++++++++++++++++++++++++++++++++++++
src/PVE/HA/Manager.pm | 24 ++++++++++++++++--
src/PVE/HA/Rules/Location.pm | 17 +++++++++++++
4 files changed, 89 insertions(+), 3 deletions(-)
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index 1b67443..2b3d726 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -131,7 +131,8 @@ sub read_and_check_resources_config {
}
}
- return $conf;
+ # TODO PVE 10: Remove digest when HA groups have been fully migrated to location rules
+ return wantarray ? ($conf, $res->{digest}) : $conf;
}
sub update_resources_config {
diff --git a/src/PVE/HA/Groups.pm b/src/PVE/HA/Groups.pm
index 821d969..d0f5721 100644
--- a/src/PVE/HA/Groups.pm
+++ b/src/PVE/HA/Groups.pm
@@ -107,4 +107,52 @@ sub parse_section_header {
__PACKAGE__->register();
__PACKAGE__->init();
+# Migrate nofailback from group config to service config
+sub migrate_groups_to_services {
+ my ($groups, $services) = @_;
+
+ for my $sid (keys %$services) {
+ my $groupid = $services->{$sid}->{group}
+ or next; # skip services without groups
+
+ # name it 'failback' to remove the double negation
+ $services->{$sid}->{failback} = !$groups->{ids}->{$groupid}->{nofailback};
+ }
+}
+
+# Migrate groups from group config to location rules in rules config
+sub migrate_groups_to_rules {
+ my ($rules, $groups, $services) = @_;
+
+ my $order = (sort { $a <=> $b } values %{ $rules->{order} })[0] || 0;
+
+ my $group_services = {};
+
+ for my $sid (keys %$services) {
+ my $groupid = $services->{$sid}->{group}
+ or next; # skip services without groups
+
+ $group_services->{$groupid}->{$sid} = 1;
+ }
+
+ while (my ($group, $services) = each %$group_services) {
+ # prefix generated rules with '_' as those are not allowed for user-provided ruleids
+ my $ruleid = "_group_$group";
+ my $nodes = {};
+ for my $entry (keys %{ $groups->{ids}->{$group}->{nodes} }) {
+ my ($node, $priority) = PVE::HA::Tools::parse_node_priority($entry);
+
+ $nodes->{$node} = { priority => $priority };
+ }
+
+ $rules->{order}->{$ruleid} = ++$order;
+ $rules->{ids}->{$ruleid} = {
+ type => 'location',
+ services => $services,
+ nodes => $nodes,
+ strict => $groups->{ids}->{$group}->{restricted},
+ };
+ }
+}
+
1;
diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index 08c2fd3..5ae8da1 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -6,6 +6,7 @@ use warnings;
use Digest::MD5 qw(md5_base64);
use PVE::Tools;
+use PVE::HA::Groups;
use PVE::HA::Tools ':exit_codes';
use PVE::HA::NodeStatus;
use PVE::HA::Rules;
@@ -48,6 +49,8 @@ sub new {
haenv => $haenv,
crs => {},
last_rules_digest => '',
+ last_groups_digest => '',
+ last_services_digest => '',
}, $class;
my $old_ms = $haenv->read_manager_status();
@@ -530,7 +533,7 @@ sub manage {
$self->update_crs_scheduler_mode();
- my $sc = $haenv->read_service_config();
+ my ($sc, $services_digest) = $haenv->read_service_config();
$self->{groups} = $haenv->read_group_config(); # update
@@ -565,7 +568,22 @@ sub manage {
my $new_rules = $haenv->read_rules_config();
- if ($new_rules->{digest} ne $self->{last_rules_digest}) {
+ my $dc_cfg = $haenv->get_datacenter_settings();
+ my $use_location_rules = $dc_cfg->{ha}->{'use-location-rules'};
+
+ # TODO PVE 10: Remove group migration when HA groups have been fully migrated to location rules
+ PVE::HA::Groups::migrate_groups_to_services($self->{groups}, $sc) if !$use_location_rules;
+
+ if (
+ $new_rules->{digest} ne $self->{last_rules_digest}
+ || $self->{groups}->{digest} ne $self->{last_groups_digest}
+ || $services_digest && $services_digest ne $self->{last_services_digest}
+ ) {
+
+ if (!$use_location_rules) {
+ PVE::HA::Rules::Location::delete_location_rules($new_rules);
+ PVE::HA::Groups::migrate_groups_to_rules($new_rules, $self->{groups}, $sc);
+ }
my $messages = PVE::HA::Rules->canonicalize($new_rules);
$haenv->log('info', $_) for @$messages;
@@ -573,6 +591,8 @@ sub manage {
$self->{rules} = $new_rules;
$self->{last_rules_digest} = $self->{rules}->{digest};
+ $self->{last_groups_digest} = $self->{groups}->{digest};
+ $self->{last_services_digest} = $services_digest;
}
$self->update_crm_commands();
diff --git a/src/PVE/HA/Rules/Location.pm b/src/PVE/HA/Rules/Location.pm
index 67f0b32..b9f76c7 100644
--- a/src/PVE/HA/Rules/Location.pm
+++ b/src/PVE/HA/Rules/Location.pm
@@ -203,4 +203,21 @@ __PACKAGE__->register_check(
},
);
+=head1 LOCATION RULE HELPERS
+
+=cut
+
+# Remove location rules from rules config
+# TODO PVE 10: Can be removed if use-location-rules feature flag is not needed anymore
+sub delete_location_rules {
+ my ($rules) = @_;
+
+ for my $ruleid (keys %{ $rules->{ids} }) {
+ next if $rules->{ids}->{$ruleid}->{type} ne 'location';
+
+ delete $rules->{ids}->{$ruleid};
+ delete $rules->{order}->{$ruleid};
+ }
+}
+
1;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel