From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id D2FD71FF164
for <inbox@lore.proxmox.com>; Fri, 20 Jun 2025 16:32:16 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 5C8B3DC21;
Fri, 20 Jun 2025 16:32:10 +0200 (CEST)
From: Daniel Kral <d.kral@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Fri, 20 Jun 2025 16:31:18 +0200
Message-Id: <20250620143148.218469-11-d.kral@proxmox.com>
X-Mailer: git-send-email 2.39.5
In-Reply-To: <20250620143148.218469-1-d.kral@proxmox.com>
References: <20250620143148.218469-1-d.kral@proxmox.com>
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.011 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH ha-manager v2 06/26] rules: add global checks
between location and colocation rules
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Add checks, which determine infeasible colocation rules, because their
services are already restricted by their location rules in such a way,
that these cannot be satisfied or are reasonable to be proven to be
satisfiable.
Positive colocation rule services need to have at least one common node
to be feasible and negative colocation rule services need to have at
least the amount of nodes available that nodes are restricted to in
total, i.e. services that are in strict location rules.
Since location rules allow nodes to be put in priority groups, but the
information which priority group is relevant depends on the online
nodes, these checks currently prohibit colocation rules with services,
which make use of these kinds of location rules.
Even though location rules are restricted to only allow a service to be
used in a single location rule, the checks here still go over all
location rules, as this restriction is bound to be changed in the
future.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
changes since v1:
- moved global checks from Colocation plugin to base plugin
- add check to only allow colocation rules for services which are in
single-priority location rules / ha groups because these are quite
stateful and cannot be easily verified to be possible
src/PVE/HA/Rules.pm | 189 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 189 insertions(+)
diff --git a/src/PVE/HA/Rules.pm b/src/PVE/HA/Rules.pm
index 4134283..588e53b 100644
--- a/src/PVE/HA/Rules.pm
+++ b/src/PVE/HA/Rules.pm
@@ -3,6 +3,7 @@ package PVE::HA::Rules;
use strict;
use warnings;
+use PVE::HashTools;
use PVE::JSONSchema qw(get_standard_option);
use PVE::Tools;
@@ -469,4 +470,192 @@ sub foreach_rule : prototype($$;$) {
}
}
+=head1 INTER-PLUGIN RULE CHECKERS
+
+=cut
+
+=head3 check_single_priority_location_for_colocated_services($location_rules, $colocation_rules)
+
+Returns a list of colocation rule ids defined in C<$colocation_rules>, where
+the services in the colocation rule are in location rules, defined in
+C<$location_rules>, which have multiple priority groups defined. That is, the
+colocation rule cannot be statically checked to be feasible as the selection of
+the priority group is dependent on the currently online nodes.
+
+If there are none, the returned list is empty.
+
+=cut
+
+sub check_single_priority_location_for_colocated_services {
+ my ($colocation_rules, $location_rules) = @_;
+
+ my @errors = ();
+
+ while (my ($colocationid, $colocation_rule) = each %$colocation_rules) {
+ my $priority;
+ my $services = $colocation_rule->{services};
+
+ for my $locationid (keys %$location_rules) {
+ my $location_rule = $location_rules->{$locationid};
+
+ next if PVE::HashTools::sets_are_disjoint($services, $location_rule->{services});
+
+ for my $node (values %{ $location_rule->{nodes} }) {
+ $priority = $node->{priority} if !defined($priority);
+
+ if ($priority != $node->{priority}) {
+ push @errors, $colocationid;
+ last; # early return to check next colocation rule
+ }
+ }
+ }
+ }
+
+ @errors = sort @errors;
+ return \@errors;
+}
+
+__PACKAGE__->register_check(
+ sub {
+ my ($args) = @_;
+
+ return check_single_priority_location_for_colocated_services(
+ $args->{colocation_rules},
+ $args->{location_rules},
+ );
+ },
+ sub {
+ my ($ruleids, $errors) = @_;
+
+ for my $ruleid (@$ruleids) {
+ push @{ $errors->{$ruleid}->{services} },
+ "services are in location rules with multiple priorities";
+ }
+ },
+);
+
+=head3 check_positive_colocation_location_consistency($positive_rules, $location_rules)
+
+Returns a list of positive colocation rule ids defined in C<$positive_rules>,
+where the services in the positive colocation rule are restricted to a disjoint
+set of nodes by their location rules, defined in C<$location_rules>. That is,
+the positive colocation rule cannot be fullfilled as the services cannot be
+placed on the same node.
+
+If there are none, the returned list is empty.
+
+=cut
+
+sub check_positive_colocation_location_consistency {
+ my ($positive_rules, $location_rules) = @_;
+
+ my @errors = ();
+
+ while (my ($positiveid, $positive_rule) = each %$positive_rules) {
+ my $allowed_nodes;
+ my $services = $positive_rule->{services};
+
+ for my $locationid (keys %$location_rules) {
+ my $location_rule = $location_rules->{$locationid};
+
+ next if !$location_rule->{strict};
+ next if PVE::HashTools::sets_are_disjoint($services, $location_rule->{services});
+
+ $allowed_nodes = { $location_rule->{nodes}->%* } if !defined($allowed_nodes);
+ $allowed_nodes = PVE::HashTools::set_intersect($allowed_nodes, $location_rule->{nodes});
+
+ if (keys %$allowed_nodes < 1) {
+ push @errors, $positiveid;
+ last; # early return to check next positive colocation rule
+ }
+ }
+ }
+
+ @errors = sort @errors;
+ return \@errors;
+}
+
+__PACKAGE__->register_check(
+ sub {
+ my ($args) = @_;
+
+ return check_positive_colocation_location_consistency(
+ $args->{positive_rules},
+ $args->{location_rules},
+ );
+ },
+ sub {
+ my ($ruleids, $errors) = @_;
+
+ for my $ruleid (@$ruleids) {
+ push @{ $errors->{$ruleid}->{services} },
+ "two or more services are restricted to different nodes";
+ }
+ },
+);
+
+=head3 check_negative_colocation_location_consistency($negative_rules, $location_rules)
+
+Returns a list of negative colocation rule ids defined in C<$negative_rules>,
+where the services in the negative colocation rule are restricted to less nodes
+than needed to keep them separate by their location rules, defined in
+C<$location_rules>. That is, the negative colocation rule cannot be fullfilled
+as there are not enough nodes to spread the services on.
+
+If there are none, the returned list is empty.
+
+=cut
+
+sub check_negative_colocation_location_consistency {
+ my ($negative_rules, $location_rules) = @_;
+
+ my @errors = ();
+
+ while (my ($negativeid, $negative_rule) = each %$negative_rules) {
+ my $allowed_nodes = {};
+ my $located_services;
+ my $services = $negative_rule->{services};
+
+ for my $locationid (keys %$location_rules) {
+ my $location_rule = $location_rules->{$locationid};
+
+ my $location_services = $location_rule->{services};
+ my $common_services = PVE::HashTools::set_intersect($services, $location_services);
+
+ next if !$location_rule->{strict};
+ next if keys %$common_services < 1;
+
+ $located_services = PVE::HashTools::set_union($located_services, $common_services);
+ $allowed_nodes = PVE::HashTools::set_union($allowed_nodes, $location_rule->{nodes});
+
+ if (keys %$allowed_nodes < keys %$located_services) {
+ push @errors, $negativeid;
+ last; # early return to check next negative colocation rule
+ }
+ }
+ }
+
+ @errors = sort @errors;
+ return \@errors;
+}
+
+__PACKAGE__->register_check(
+ sub {
+ my ($args) = @_;
+
+ return check_negative_colocation_location_consistency(
+ $args->{negative_rules},
+ $args->{location_rules},
+ );
+ },
+ sub {
+ my ($ruleids, $errors) = @_;
+
+ for my $ruleid (@$ruleids) {
+ push @{ $errors->{$ruleid}->{services} },
+ "two or more services are restricted to less nodes than available to the services";
+ }
+ },
+);
+
1;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel