[pve-devel] cluster resource scheduler question

[pve-devel] cluster resource scheduler question

[DERUMIER, Alexandre]

Hi,

I was looking at the proxmox 7.3 video
https://www.proxmox.com/en/training/video-tutorials/item/what-s-new-in-proxmox-ve-7-3

showing the new cluster resource scheduling with static-ressource.

I'm not sure to understand, but the video seem to say that vm is
migrated to the least "loaded" node.


But looking at static-info of the node,  cpu && memory are the physical
core number && full memory size

(in pvestatd,

    broadcast_static_node_info($maxcpu, $meminfo->{memtotal});
     my sub broadcast_static_node_info {
      ...
        my $info = {
            cpus => $cpus,
            memory => $memory,
     }


I'm not sure about the logic, but if we only look at number of total
cpus && memory without current usage, the biggest node will always be
choose ?  or does I miss something ?

Re: [pve-devel] cluster resource scheduler question

[Fiona Ebner]

Am 22.11.22 um 17:43 schrieb DERUMIER, Alexandre:
> Hi,
> 
> I was looking at the proxmox 7.3 video
> https://www.proxmox.com/en/training/video-tutorials/item/what-s-new-in-proxmox-ve-7-3
> 
> showing the new cluster resource scheduling with static-ressource.
> 
> I'm not sure to understand, but the video seem to say that vm is
> migrated to the least "loaded" node.

yes, the video is misleading there. The static node CPU and memory
values and the configured CPU and memory for guests will be used to
approximate the real usage.

> 
> 
> But looking at static-info of the node,  cpu && memory are the physical
> core number && full memory size
> 
> (in pvestatd,
> 
>     broadcast_static_node_info($maxcpu, $meminfo->{memtotal});
>      my sub broadcast_static_node_info {
>       ...
>         my $info = {
>             cpus => $cpus,
>             memory => $memory,
>      }
> 
> 
> I'm not sure about the logic, but if we only look at number of total
> cpus && memory without current usage, the biggest node will always be
> choose ?  or does I miss something ?

We add the static usage of all active guests (i.e. configured CPU and
memory) to the node's usage. Then, for choosing a node for a guest, we
check how the highest/average CPU/memory usage (in percent) in the
cluster would look like after starting the guest on each node and score
according to those alternatives. In effect, the alternative leading to
the least load (in percent) across the cluster, according to the model
with static guest and node information will be chosen.

Re: [pve-devel] cluster resource scheduler question

[DERUMIER, Alexandre]

After doing some test, 


if I understand, the node static infos max-mem, max-cpu are only used
to have percents to be able to compare each node.

and the nodeusage is compute with the sum of static mem/cpu of the
services running on this node.

But I really don't see where it's node in the rust code ?


I see a 

+impl StaticNodeUsage {
+    /// Add usage of `service` to the node's usage.
+    pub fn add_service_usage(&mut self, service: &StaticServiceUsage)
{
+        self.cpu = add_cpu_usage(self.cpu, self.maxcpu as f64,
service.maxcpu);
+        self.mem += service.maxmem;
+    }
+}


But I don't see where it's called ?


(BTW, I don't see the repo in proxmox git ?)


Le mardi 22 novembre 2022 à 16:43 +0000, DERUMIER, Alexandre a écrit :
> Hi,
> 
> I was looking at the proxmox 7.3 video
> https://www.proxmox.com/en/training/video-tutorials/item/what-s-new-in-proxmox-ve-7-3
> 
> showing the new cluster resource scheduling with static-ressource.
> 
> I'm not sure to understand, but the video seem to say that vm is
> migrated to the least "loaded" node.
> 
> 
> But looking at static-info of the node,  cpu && memory are the
> physical
> core number && full memory size
> 
> (in pvestatd,
> 
>     broadcast_static_node_info($maxcpu, $meminfo->{memtotal});
>      my sub broadcast_static_node_info {
>       ...
>         my $info = {
>             cpus => $cpus,
>             memory => $memory,
>      }
> 
> 
> I'm not sure about the logic, but if we only look at number of total
> cpus && memory without current usage, the biggest node will always be
> choose ?  or does I miss something ?
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 

Re: [pve-devel] cluster resource scheduler question

[Thomas Lamprecht]

Am 23/11/2022 um 10:09 schrieb Fiona Ebner:
>> I'm not sure to understand, but the video seem to say that vm is
>> migrated to the least "loaded" node.
> yes, the video is misleading there. The static node CPU and memory
> values and the configured CPU and memory for guests will be used to
> approximate the real usage.
> 

Not really misleading. It's the least loaded node, there's just more ways to
determine load and the current mode is for static-load (vs. dynamic runtime
load, planned for a next release).

Am 22/11/2022 um 17:43 schrieb DERUMIER, Alexandre:
> I'm not sure about the logic, but if we only look at number of total
> cpus && memory without current usage, the biggest node will always be
> choose ?

Well yes, until it's "filled" enough for other nodes to be more favorable,
a "bigger" node can host more after all.

Am 23/11/2022 um 11:18 schrieb DERUMIER, Alexandre:
> (BTW, I don't see the repo in proxmox git ?)

fixed now, thanks for the reminder

Re: [pve-devel] cluster resource scheduler question

[Fiona Ebner]

Am 23.11.22 um 11:18 schrieb DERUMIER, Alexandre:
> After doing some test, 
> 
> 
> if I understand, the node static infos max-mem, max-cpu are only used
> to have percents to be able to compare each node.
> 
> and the nodeusage is compute with the sum of static mem/cpu of the
> services running on this node.
> 
> But I really don't see where it's node in the rust code ?
> 
> 
> I see a 
> 
> +impl StaticNodeUsage {
> +    /// Add usage of `service` to the node's usage.
> +    pub fn add_service_usage(&mut self, service: &StaticServiceUsage)
> {
> +        self.cpu = add_cpu_usage(self.cpu, self.maxcpu as f64,
> service.maxcpu);
> +        self.mem += service.maxmem;
> +    }
> +}
> 
> 
> But I don't see where it's called ?

The Perl code calls into the PVE::RS::ResourceScheduling module. That
module acts as a FFI (generated by perlmod) for the Rust code in [0].
That Rust code then calls into the proxmox-resource-scheduling crate.
For example, at line 93, it calls the method you mentioned.

[0]
https://git.proxmox.com/?p=proxmox-perl-rs.git;a=blob;f=pve-rs/src/resource_scheduling/static.rs;h=c47dcd3477eb75d7c7f14c7c4d914179a549430e;hb=HEAD

Re: [pve-devel] cluster resource scheduler question

[DERUMIER, Alexandre]

Ah ok, got it, thanks.

I had totally missed HA/Usage/Static.pm , which is used in
recompute_online_node_usage.


I was a bit lost between perl && rust, but I begin to understand how
it's working.  (Nice code BTW ;)




Le mercredi 23 novembre 2022 à 12:28 +0100, Fiona Ebner a écrit :
> Am 23.11.22 um 11:18 schrieb DERUMIER, Alexandre:
> > After doing some test, 
> > 
> > 
> > if I understand, the node static infos max-mem, max-cpu are only
> > used
> > to have percents to be able to compare each node.
> > 
> > and the nodeusage is compute with the sum of static mem/cpu of the
> > services running on this node.
> > 
> > But I really don't see where it's node in the rust code ?
> > 
> > 
> > I see a 
> > 
> > +impl StaticNodeUsage {
> > +    /// Add usage of `service` to the node's usage.
> > +    pub fn add_service_usage(&mut self, service:
> > &StaticServiceUsage)
> > {
> > +        self.cpu = add_cpu_usage(self.cpu, self.maxcpu as f64,
> > service.maxcpu);
> > +        self.mem += service.maxmem;
> > +    }
> > +}
> > 
> > 
> > But I don't see where it's called ?
> 
> The Perl code calls into the PVE::RS::ResourceScheduling module. That
> module acts as a FFI (generated by perlmod) for the Rust code in [0].
> That Rust code then calls into the proxmox-resource-scheduling crate.
> For example, at line 93, it calls the method you mentioned.
> 
> [0]
> https://antiphishing.cetsi.fr/proxy/v3?i=d1l4NXNNaWE4SWZqU0dLWcuTfdxEd98NfWIp9dma5kY&r=MXJUa0FrUVJqc1UwYWxNZ8tuXduEO8AMVnCPQXMJjAacdo9GABnHjKxpaZ5vEHCU6zVrUZu3S1lZhwcGmwx2yA&f=c2xMdVN4Smh2R2tOZDdIRLb87S4icH52IJimWEw_GeUm5GbjoNM9wfgv6zQHlbuo&u=https%3A//git.proxmox.com/%3Fp%3Dproxmox-perl-rs.git%3Ba%3Dblob%3Bf%3Dpve-rs/src/resource_scheduling/static.rs%3Bh%3Dc47dcd3477eb75d7c7f14c7c4d914179a549430e%3Bhb%3DHEAD&k=fjzS
> 
>