Re: [pve-devel] [RFC cluster/ha-manager 00/16] HA colocation rules

[Fiona Ebner <f.ebner@proxmox.com>]

Am 01.04.25 um 11:39 schrieb Daniel Kral:
> On 4/1/25 03:50, DERUMIER, Alexandre wrote:
>> my 2cents, but everybody in the industry is calling this
>> affinity/antiafifnity (vmware, nutanix, hyperv, openstack, ...).
>> More precisely, vm affinity rules (vm<->vm)   vs  node affinity rules
>> (vm->node , the current HA group)
>>
>> Personnally I don't care, it's just a name ^_^ .
>>
>> But I have a lot  of customers asking about "does proxmox support
>> affinity/anti-affinity". and if they are doing their own research, they
>> will think that it doesnt exist.
>> (or at minimum, write  somewhere in the doc something like "aka vm
>> affinity" or in commercial presentation ^_^)
> 
> I see your point and also called it affinity/anti-affinity before, but
> if we go for the HA Rules route here, it'd be really neat to have
> "Location Rules" and "Colocation Rules" in the end to coexist and
> clearly show the distinction between them, as both are affinity rules at
> least for me.
> 
> I'd definitely make sure that it is clear from the release notes and
> documentation, that this adds the feature to assign affinity between
> services, but let's wait for some other comments on this ;).

In the UI/docs we can be always be more descriptive and say things like
"(Anti-)Affinity Between Services" and "(Anti-)Affinity With Node",
while in the section config it's of course advantageous to have a single
word.

> 
> On 4/1/25 03:50, DERUMIER, Alexandre wrote:
>> More serious question : Don't have read yet all the code, but how does
>> it play with the current topsis placement algorithm ?
> 
> I currently implemented the colocation rules to put a constraint on
> which nodes the manager can select from for the to-be-migrated service.
> 
> So if users use the static load scheduler (and the basic / service count
> scheduler for that matter too), the colocation rules just make sure that
> no recovery node is selected, which contradicts the colocation rules. So
> the TOPSIS algorithm isn't changed at all.
> 
> There are two things that should/could be changed in the future (besides
> the many future ideas that I pointed out already), which are
> 
> - (1) the schedulers will still consider all online nodes, i.e. even
> though HA groups and/or colocation rules restrict the allowed nodes in
> the end, the calculation is done for all nodes which could be
> significant for larger clusters, and
> 
> - (2) the service (generally) are currently recovered one-by-one in a
> best-fit fashion, i.e. there's no order on the service's needed
> resources, etc. There could be some edge cases (e.g. think about a
> failing node with a bunch of service to be kept together; these should
> now be migrated to the same node, if possible, or put them on the
> minimum amount of nodes), where the algorithm could find better
> solutions if it either orders the to-be-recovered services, and/or the
> utilization scheduler has knowledge about the 'keep together'
> colocations and considers these (and all subsets) as a single service.

Yes, a simple heuristic here could be to take the subsets of:
1. (strict?) 'keep together' services
2. single services that are not otherwise in a (strict?) 'keep
   together' relation, consider each by itself a subset too

Then order the above subsets by their usage (ordering inside a subset
should not be that important) and then recover the services in that
order one-by-one (i.e. one-by-one for the first subset in the ordering,
then one-by-one for the second subset in the ordering, etc.). Even if
it's one-by-one that should mean keeping the (strict) 'keep together'
together, right?

Like that you get the heavy subsets out of the way first. This prevents
the otherwise likely scenario where too many small services are
recovered in a balanced fashion to other nodes (let's say nodes all end
up at 80% usage) and then there's no single node with the necessary
resources for a heavy service that is still to be recovered (e.g. one
that would need 30% usage on a node).

Can of course be done as a follow-up.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel