Re: [pve-devel] [RFC many 0/3] combine and simplify RRD handling

[Aaron Lauterer <a.lauterer@proxmox.com>]

On  2025-09-04  20:20, Thomas Lamprecht wrote:
> Am 04.09.25 um 16:09 schrieb Aaron Lauterer:
>> this series does two things:
>> * switch the RRD API backend to use the old RRD files is available
>> * always use RRD files with the new 9.0 schema
>>
>> The motivation for the combination of old and new RRD files is that during the
>> migration of old RRD files to the new ones, spikes get flattened. By combining
>> the old and new RRD files, we can keep the old coarse data with all it spikes
>> and only show the new much mode finer stepped data where available.
>>
>> This will also enable us to just write any new data into an RRD file that is
>> created with the new 9.0 schema. As a result, no migration step is needed
>> anymore and we can simplify the logic around the whole "does the only the old
>> file exist or is the new one also present" alot in quite a few places.
>>
>> Some examples which we can simplify or revert:
>> pve-manager: API2Tools::get_rrd_key can most likely be dropped
>> qemu-server, pve-storage, pve-container,…: rrd API endpoints which check which files exist
>> drop the whole migration step on upgrade
>>
>> other TODOs:
>>
>> RRD::create_rrd_graph -> see if we can combine both files into one graph. But we
>> are relying on RRD itself for this. So that might be interesting. On the other
>> hand, do we want to keep that API endpoint around? People are using it though.
>> So we need to weigh that decision accoringly.
> 
> Keeping it around for the time being is definitively required now that
> PVE 9.0 is out, we might sunset it for a future major release.
> 
> Another option might be to generate both images and merge them manually,
> i.e. literally make the right part the old and the left part the new
> image with some imagemagick command. Or does rrd indeed supports merging
> two files together?

One thing that should be possible is to give it two datasets from 
different files. That would be a middle ground as in, all data is there, 
but it would show up as separata data lines.

I am looking into it to see if we could get it nicer. Manually stitching 
images together will most likely not work because I don't expected the 
scales to match up perfectly.

> 
>>
>>
>> This is currently marked as RFC. Tested by installing it on an existing 9.0.6
>> cluster. Added a 8.4 node with latest updates. Renamed {rrd}.old to {rrd}.
>>
>> cluster:
>>
>> Aaron Lauterer (3):
>>    rrd: fix rrd time frames
>>    RRD: fetch data from old rrd file if present and needed
>>    pmxcfs: status.c: always use 9.0 rrd files
>>
>>   src/PVE/RRD.pm      | 113 ++++++++++++++-------
>>   src/pmxcfs/status.c | 238 ++++++--------------------------------------
>>   2 files changed, 108 insertions(+), 243 deletions(-)
>>
>>
>> Summary over all repositories:
>>    2 files changed, 108 insertions(+), 243 deletions(-)
>>
> 



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel