[pve-devel] Telegraf added in-built Proxmox support - thoughts versus our external metric support?

[pve-devel] Telegraf added in-built Proxmox support - thoughts versus our external metric support?

[Victor Hooi]

Hi,

I know that Proxmox has it's own inbuilt InfluxDB client:

https://pve.proxmox.com/wiki/External_Metric_Server

However, Telegraf recently added first-party support for Proxmox:

https://github.com/influxdata/telegraf/tree/master/plugins/inputs/proxmox

Telegraf lets you output to InfluxDB, Graphite, Prometheus, as well as a
bunch of others (Telegraf in-built output clients
<https://github.com/influxdata/telegraf/tree/master/plugins/outputs>)

What do you think of using the above and contributing to that, instead of
maintaining our own Proxmox InfluxDB support?

Or are there advantages to maintaining our own code here?

Thanks,
Victor

Re: [pve-devel] Telegraf added in-built Proxmox support - thoughts versus our external metric support?

[Thomas Lamprecht]

Hi,

On 07.09.20 03:42, Victor Hooi wrote:
> I know that Proxmox has it's own inbuilt InfluxDB client:
> 
> https://pve.proxmox.com/wiki/External_Metric_Server
> 
> However, Telegraf recently added first-party support for Proxmox:
> 
> https://github.com/influxdata/telegraf/tree/master/plugins/inputs/proxmox

great!

> Telegraf lets you output to InfluxDB, Graphite, Prometheus, as well as a
> bunch of others (Telegraf in-built output clients
> <https://github.com/influxdata/telegraf/tree/master/plugins/outputs>)
> 
> What do you think of using the above and contributing to that, instead of
> maintaining our own Proxmox InfluxDB support?
> 
> Or are there advantages to maintaining our own code here?

The pvestatd, which queries statistics periodically, does also the sending
of said statistics without extra overhead. API request may get, at least
partially, up to date information with an extra overhead, e.g., if storage
stats are to be queried too. That could be addressed by providing a pvestatd
fed cache in /run (fast memory tmpfs) or so and provide access to that over
the API.

Anyway, I do not think that we should drop our direct plugins (yet), some
people like me, are happy feeding directly to InfluxDB without anything
in-between.

But, we definitively want to mention this in the documentation and see how
we can improve integration.

cheers,
Thomas

Re: [pve-devel] Telegraf added in-built Proxmox support - thoughts versus our external metric support?

[Alexandre DERUMIER]

Hi,

>>Anyway, I do not think that we should drop our direct plugins (yet), some
>>people like me, are happy feeding directly to InfluxDB without anything
>>in-between.

me too ;)


I'm not sure, but I think than we send more metrics to influxdb, than we stream through the cluster. 

and the plugin does seem to use /cluster/resources api, but seem to do 1 api request by vm.
(So with a lot of vm, maybe it'll flood the api)


----- Mail original -----
De: "Thomas Lamprecht" <t.lamprecht@proxmox.com>
À: "Proxmox VE development discussion" <pve-devel@lists.proxmox.com>, "Victor Hooi" <victorhooi@yahoo.com>
Envoyé: Lundi 7 Septembre 2020 08:57:31
Objet: Re: [pve-devel] Telegraf added in-built Proxmox support - thoughts versus our external metric support?

Hi, 

On 07.09.20 03:42, Victor Hooi wrote: 
> I know that Proxmox has it's own inbuilt InfluxDB client: 
> 
> https://pve.proxmox.com/wiki/External_Metric_Server 
> 
> However, Telegraf recently added first-party support for Proxmox: 
> 
> https://github.com/influxdata/telegraf/tree/master/plugins/inputs/proxmox 

great! 

> Telegraf lets you output to InfluxDB, Graphite, Prometheus, as well as a 
> bunch of others (Telegraf in-built output clients 
> <https://github.com/influxdata/telegraf/tree/master/plugins/outputs>) 
> 
> What do you think of using the above and contributing to that, instead of 
> maintaining our own Proxmox InfluxDB support? 
> 
> Or are there advantages to maintaining our own code here? 

The pvestatd, which queries statistics periodically, does also the sending 
of said statistics without extra overhead. API request may get, at least 
partially, up to date information with an extra overhead, e.g., if storage 
stats are to be queried too. That could be addressed by providing a pvestatd 
fed cache in /run (fast memory tmpfs) or so and provide access to that over 
the API. 

Anyway, I do not think that we should drop our direct plugins (yet), some 
people like me, are happy feeding directly to InfluxDB without anything 
in-between. 

But, we definitively want to mention this in the documentation and see how 
we can improve integration. 

cheers, 
Thomas 



_______________________________________________ 
pve-devel mailing list 
pve-devel@lists.proxmox.com 
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel