all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH-SERIES cluster/common/qemu-server/lxc/manager] add new metric stats in single rrd
@ 2022-05-25  6:52 Alexandre Derumier
  2022-05-25  6:52 ` [pve-devel] [PATCH pve-cluster 1/1] add pve2-metrics rrd (single metrics) Alexandre Derumier
                   ` (10 more replies)
  0 siblings, 11 replies; 15+ messages in thread
From: Alexandre Derumier @ 2022-05-25  6:52 UTC (permalink / raw)
  To: pve-devel

Hi,

I'm still working on vm balancing/scheduling, and need some new metrics.

This patch-series add new metrics stats in rrd

- cpu,mem,io pressure for qemu/lxc/host
- hostcpu/hostmem cgroup for qemu
- ksm

as we discussed last year, theses new metrics are pushed in single rrd files,
like proxmox backup server.

The paths are :

pve2-metrics/vms/<vmid>/<metric>
pve2-metrics/nodes/<node>/<metric>
pve2-metrics/storages/<node>/<storage>/<metric>



pve-cluster:

Alexandre Derumier (1):
  add pve2-metrics rrd (single metrics)

 data/src/status.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)


pve-common:

Alexandre Derumier (2):
  cgroup: get_pressure_stat: use controllers in get_path
  cgroup: get_pressure_stat: add cpu full pressure

 src/PVE/CGroup.pm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

qemu-server:

Alexandre Derumier (3):
  vmstatus: add hostcpu value
  vmstatus: add hostmem value
  vmstatus: add pressure stats

 PVE/QemuServer.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

pve-container:

Alexandre Derumier (1):
  vmstatus: add pressure stats

 src/PVE/LXC.pm | 2 ++
 1 file changed, 2 insertions(+)

pve-manager:

Alexandre Derumier (3):
  pvestatd: qemu/lxc/host : broadcast rrd pressure metrics
  pvestatd: qemu: broadcast rrd hostcpu && hostmem metrics
  pvestatd: host: broadcast rrd ksm metric

 PVE/Service/pvestatd.pm | 53 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

-- 
2.30.2




^ permalink raw reply	[flat|nested] 15+ messages in thread
* [pve-devel] [PATCH-SERIES common/qemu-server/lxc/manager] brodcast new metric stats in kvstore
@ 2022-06-01  8:12 Alexandre Derumier
  2022-06-01  8:12 ` [pve-devel] [PATCH qemu-server 3/3] vmstatus: add pressure stats Alexandre Derumier
  0 siblings, 1 reply; 15+ messages in thread
From: Alexandre Derumier @ 2022-06-01  8:12 UTC (permalink / raw)
  To: pve-devel; +Cc: t.lamprecht, Alexandre Derumier

Hi,

I'm still working on vm balancing/scheduling, and need some new metrics.

As suggested by Thomas, instead sending values to new rrd,
and then reparse rrd later in balancer, we broadcast:

- last minute && last 5 min average cpu/ram of qemu/lxc/node
- pressure average of cpu/mem/io for qemu/lxc/node

Theses values are broadcast each minute in 1 kvstore key "balancer-stats"

Size of broadcasted "balancer-stats" is around : 500bytes * number vms.

Size of in memory hash for average values is around : 6k (host+mem) * number vms.
(for last minute, we keep each X seconds iteration, for last 5min we keep last 5x1min average)


- last ksm value
This value is broadcast each iteration is a separate kvstore key "ksm"



pve-common:

Alexandre Derumier (2):
  cgroup: get_pressure_stat: use controllers in get_path
  cgroup: get_pressure_stat: add cpu full pressure

 src/PVE/CGroup.pm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

qemu-server:

Alexandre Derumier (3):
  vmstatus: add hostcpu value
  vmstatus: add hostmem value
  vmstatus: add pressure stats

 PVE/QemuServer.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

pve-container:

Alexandre Derumier (1):
  vmstatus: add pressure stats

 src/PVE/LXC.pm | 2 ++
 1 file changed, 2 insertions(+)

Alexandre Derumier (4):
  pvestatd: add broadcast_balancer_stats
  pvestatd: qemu/lxc/node : add pressure stats
  pvestatd: qemu/lxc/node : add hostcpu/hostmem average stats
  pvestatd: node : broadcast ksm value

 PVE/Service/pvestatd.pm | 154 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 153 insertions(+), 1 deletion(-)

-- 
2.30.2



^ permalink raw reply	[flat|nested] 15+ messages in thread
* [pve-devel] [PATCH qemu-server 0/3] vmstatus: add pressure + hostcpu/hostmem
@ 2020-11-04  1:41 Alexandre Derumier
  2020-11-04  1:41 ` [pve-devel] [PATCH qemu-server 3/3] vmstatus: add pressure stats Alexandre Derumier
  0 siblings, 1 reply; 15+ messages in thread
From: Alexandre Derumier @ 2020-11-04  1:41 UTC (permalink / raw)
  To: pve-devel

Need pve-common patches serie.


Alexandre Derumier (3):
  vmstatus: add hostcpu value
  vmstatus: add hostmem value
  vmstatus: add pressure stats

 PVE/QemuServer.pm | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

-- 
2.20.1




^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2022-06-01  8:13 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-25  6:52 [pve-devel] [PATCH-SERIES cluster/common/qemu-server/lxc/manager] add new metric stats in single rrd Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH pve-cluster 1/1] add pve2-metrics rrd (single metrics) Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH pve-common 1/2] cgroup: get_pressure_stat: use controllers in get_path Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH pve-manager 1/3] pvestatd: qemu/lxc/host : broadcast rrd pressure metrics Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH qemu-server 1/3] vmstatus: add hostcpu value Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH pve-container 1/1] vmstatus: add pressure stats Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH pve-common 2/2] cgroup: get_pressure_stat: add cpu full pressure Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH pve-manager 2/3] pvestatd: qemu: broadcast rrd hostcpu && hostmem metrics Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH qemu-server 2/3] vmstatus: add hostmem value Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH pve-manager 3/3] pvestatd: host: broadcast rrd ksm metric Alexandre Derumier
2022-05-25  6:52 ` [pve-devel] [PATCH qemu-server 3/3] vmstatus: add pressure stats Alexandre Derumier
2022-05-25 15:03 ` [pve-devel] [PATCH-SERIES cluster/common/qemu-server/lxc/manager] add new metric stats in single rrd Thomas Lamprecht
2022-05-28 17:11   ` DERUMIER, Alexandre
  -- strict thread matches above, loose matches on Subject: below --
2022-06-01  8:12 [pve-devel] [PATCH-SERIES common/qemu-server/lxc/manager] brodcast new metric stats in kvstore Alexandre Derumier
2022-06-01  8:12 ` [pve-devel] [PATCH qemu-server 3/3] vmstatus: add pressure stats Alexandre Derumier
2020-11-04  1:41 [pve-devel] [PATCH qemu-server 0/3] vmstatus: add pressure + hostcpu/hostmem Alexandre Derumier
2020-11-04  1:41 ` [pve-devel] [PATCH qemu-server 3/3] vmstatus: add pressure stats Alexandre Derumier

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal