[pve-devel] [PATCH qemu-server 0/3] vmstatus: add pressure + hostcpu/hostmem

[pve-devel] [PATCH qemu-server 0/3] vmstatus: add pressure + hostcpu/hostmem

[Alexandre Derumier]

Need pve-common patches serie.


Alexandre Derumier (3):
  vmstatus: add hostcpu value
  vmstatus: add hostmem value
  vmstatus: add pressure stats

 PVE/QemuServer.pm | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

-- 
2.20.1

[pve-devel] [PATCH qemu-server 1/3] vmstatus: add hostcpu value

[Alexandre Derumier]

---
 PVE/QemuServer.pm | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index af74d64..0c2b1db 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -2647,8 +2647,11 @@ sub vmstatus {
 
 	my $pstat = PVE::ProcFSTools::read_proc_pid_stat($pid);
 	next if !$pstat; # not running
+	my $cgroups = PVE::QemuServer::CGroup->new($vmid);
+	my $hostcpustat = $cgroups->get_cpu_stat();
 
 	my $used = $pstat->{utime} + $pstat->{stime};
+	my $hostused = $hostcpustat->{utime} + $hostcpustat->{stime};
 
 	$d->{uptime} = int(($uptime - $pstat->{starttime})/$cpuinfo->{user_hz});
 
@@ -2662,6 +2665,9 @@ sub vmstatus {
 		time => $ctime,
 		used => $used,
 		cpu => 0,
+		hostused => $hostused,
+		hostcpu => 0,
+
 	    };
 	    next;
 	}
@@ -2670,15 +2676,20 @@ sub vmstatus {
 
 	if ($dtime > 1000) {
 	    my $dutime = $used -  $old->{used};
+	    my $dhostutime = $hostused -  $old->{hostused};
 
 	    $d->{cpu} = (($dutime/$dtime)* $cpucount) / $d->{cpus};
+	    $d->{hostcpu} = (($dhostutime/$dtime)* $cpucount) / $d->{cpus};
 	    $last_proc_pid_stat->{$pid} = {
 		time => $ctime,
 		used => $used,
 		cpu => $d->{cpu},
+		hostused => $hostused,
+		hostcpu => $d->{hostcpu},
 	    };
 	} else {
 	    $d->{cpu} = $old->{cpu};
+	    $d->{hostcpu} = $old->{hostcpu};
 	}
     }
 
-- 
2.20.1

[pve-devel] [PATCH qemu-server 2/3] vmstatus: add hostmem value

[Alexandre Derumier]

---
 PVE/QemuServer.pm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 0c2b1db..64d3635 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -2658,6 +2658,11 @@ sub vmstatus {
 	if ($pstat->{vsize}) {
 	    $d->{mem} = int(($pstat->{rss}/$pstat->{vsize})*$d->{maxmem});
 	}
+	if (defined(my $hostmemstat = $cgroups->get_memory_stat())) {
+	    $d->{hostmem} = $hostmemstat->{mem};
+	} else {
+	    $d->{hostmem} = 0;
+	}
 
 	my $old = $last_proc_pid_stat->{$pid};
 	if (!$old) {
-- 
2.20.1

[pve-devel] [PATCH qemu-server 3/3] vmstatus: add pressure stats

[Alexandre Derumier]

---
 PVE/QemuServer.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 64d3635..f1b778e 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -2696,6 +2696,24 @@ sub vmstatus {
 	    $d->{cpu} = $old->{cpu};
 	    $d->{hostcpu} = $old->{hostcpu};
 	}
+
+	if (defined(my $pressure = $cgroups->get_pressure_stat())) {
+            $d->{pressure} = $pressure;
+	} else {
+	    $d->{pressure} = {
+		cpu => {
+		    some => { avg10 => 10, avg60 => 0, avg300 => 0 }
+		},
+		memory => {
+		    some => { avg10 => 10, avg60 => 0, avg300 => 0 },
+		    full => { avg10 => 10, avg60 => 0, avg300 => 0 }
+		},
+		io => {
+		    some => { avg10 => 10, avg60 => 0, avg300 => 0 },
+		    full => { avg10 => 10, avg60 => 0, avg300 => 0 }
+		},
+	    };
+        }
     }
 
     return $res if !$full;
-- 
2.20.1

[pve-devel] [PATCH-SERIES cluster/common/qemu-server/lxc/manager] add new metric stats in single rrd

[Alexandre Derumier]

Hi,

I'm still working on vm balancing/scheduling, and need some new metrics.

This patch-series add new metrics stats in rrd

- cpu,mem,io pressure for qemu/lxc/host
- hostcpu/hostmem cgroup for qemu
- ksm

as we discussed last year, theses new metrics are pushed in single rrd files,
like proxmox backup server.

The paths are :

pve2-metrics/vms/<vmid>/<metric>
pve2-metrics/nodes/<node>/<metric>
pve2-metrics/storages/<node>/<storage>/<metric>



pve-cluster:

Alexandre Derumier (1):
  add pve2-metrics rrd (single metrics)

 data/src/status.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)


pve-common:

Alexandre Derumier (2):
  cgroup: get_pressure_stat: use controllers in get_path
  cgroup: get_pressure_stat: add cpu full pressure

 src/PVE/CGroup.pm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

qemu-server:

Alexandre Derumier (3):
  vmstatus: add hostcpu value
  vmstatus: add hostmem value
  vmstatus: add pressure stats

 PVE/QemuServer.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

pve-container:

Alexandre Derumier (1):
  vmstatus: add pressure stats

 src/PVE/LXC.pm | 2 ++
 1 file changed, 2 insertions(+)

pve-manager:

Alexandre Derumier (3):
  pvestatd: qemu/lxc/host : broadcast rrd pressure metrics
  pvestatd: qemu: broadcast rrd hostcpu && hostmem metrics
  pvestatd: host: broadcast rrd ksm metric

 PVE/Service/pvestatd.pm | 53 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

-- 
2.30.2

[pve-devel] [PATCH qemu-server 2/3] vmstatus: add hostmem value

[Alexandre Derumier]

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 PVE/QemuServer.pm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 9441cf2..4fc183e 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -2933,6 +2933,11 @@ sub vmstatus {
 	if ($pstat->{vsize}) {
 	    $d->{mem} = int(($pstat->{rss}/$pstat->{vsize})*$d->{maxmem});
 	}
+	if (defined(my $hostmemstat = $cgroups->get_memory_stat())) {
+	    $d->{hostmem} = $hostmemstat->{mem};
+	} else {
+	    $d->{hostmem} = 0;
+	}
 
 	my $old = $last_proc_pid_stat->{$pid};
 	if (!$old) {
-- 
2.30.2

[pve-devel] [PATCH-SERIES common/qemu-server/lxc/manager] brodcast new metric stats in kvstore

[Alexandre Derumier]

Hi,

I'm still working on vm balancing/scheduling, and need some new metrics.

As suggested by Thomas, instead sending values to new rrd,
and then reparse rrd later in balancer, we broadcast:

- last minute && last 5 min average cpu/ram of qemu/lxc/node
- pressure average of cpu/mem/io for qemu/lxc/node

Theses values are broadcast each minute in 1 kvstore key "balancer-stats"

Size of broadcasted "balancer-stats" is around : 500bytes * number vms.

Size of in memory hash for average values is around : 6k (host+mem) * number vms.
(for last minute, we keep each X seconds iteration, for last 5min we keep last 5x1min average)


- last ksm value
This value is broadcast each iteration is a separate kvstore key "ksm"



pve-common:

Alexandre Derumier (2):
  cgroup: get_pressure_stat: use controllers in get_path
  cgroup: get_pressure_stat: add cpu full pressure

 src/PVE/CGroup.pm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

qemu-server:

Alexandre Derumier (3):
  vmstatus: add hostcpu value
  vmstatus: add hostmem value
  vmstatus: add pressure stats

 PVE/QemuServer.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

pve-container:

Alexandre Derumier (1):
  vmstatus: add pressure stats

 src/PVE/LXC.pm | 2 ++
 1 file changed, 2 insertions(+)

Alexandre Derumier (4):
  pvestatd: add broadcast_balancer_stats
  pvestatd: qemu/lxc/node : add pressure stats
  pvestatd: qemu/lxc/node : add hostcpu/hostmem average stats
  pvestatd: node : broadcast ksm value

 PVE/Service/pvestatd.pm | 154 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 153 insertions(+), 1 deletion(-)

-- 
2.30.2

[pve-devel] [PATCH qemu-server 2/3] vmstatus: add hostmem value

[Alexandre Derumier]

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 PVE/QemuServer.pm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 9441cf2..4fc183e 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -2933,6 +2933,11 @@ sub vmstatus {
 	if ($pstat->{vsize}) {
 	    $d->{mem} = int(($pstat->{rss}/$pstat->{vsize})*$d->{maxmem});
 	}
+	if (defined(my $hostmemstat = $cgroups->get_memory_stat())) {
+	    $d->{hostmem} = $hostmemstat->{mem};
+	} else {
+	    $d->{hostmem} = 0;
+	}
 
 	my $old = $last_proc_pid_stat->{$pid};
 	if (!$old) {
-- 
2.30.2