public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: Proxmox Backup Server development discussion
	<pbs-devel@lists.proxmox.com>,
	Dietmar Maurer <dietmar@proxmox.com>
Subject: [pbs-devel] applied-series: [PATCH proxmox-backup 00/15] RRD database improvements
Date: Wed, 13 Oct 2021 13:58:26 +0200	[thread overview]
Message-ID: <32783331-e2c4-3a19-47b2-c89d670e6291@proxmox.com> (raw)
In-Reply-To: <20211013082452.619406-1-dietmar@proxmox.com>

On 13.10.21 10:24, Dietmar Maurer wrote:
> - use a journal. This way we can reduce the overall number of bytes
>   written by increasing the flush/commit interval (30 minutes).
> 
> - the new CBOR base format is more flexible, and we store much more
>   data points now.
> 
> We previously wrote about 7MB/h. With the new format and journal, we now write about 3MB/h.
> 
> Dietmar Maurer (15):
>   proxmox-rrd: use a journal to reduce amount of bytes written
>   RRD_CACHE: use a OnceCell instead of lazy_static
>   proxmox-backup-proxy: use tokio::task::spawn_blocking instead of
>     block_in_place
>   proxmox-rrd: implement new CBOR based format
>   proxmox-rrd: remove dependency to proxmox-rrd-api-types
>   proxmox-rrd: extract_data: include values from current slot
>   remove proxmox-rrd-api-types crate,
>     s/RRDTimeFrameResolution/RRDTimeFrame/
>   proxmox-rrd: support CF::Last
>   proxmox-rrd: split out load_rrd (cleanup)
>   proxmox-rrd: add binary to create/manage rrd files
>   proxmox-rrd: avoid % inside loop
>   proxmox-rrd: new helper methods - slot() and slot_end_time()
>   proxmox-rrd: protect against negative update time
>   proxmox-rrd: rename last_counter to last_value
>   proxmox-rrd: add more commands to the rrd cli tool

applied whole series, with the diff you replied on patch 14/15 squashed in and the rrd
debug tool moved to examples as quick "fix" for avoiding proxmox-router as hard dependency
on the crate for now, thanks!




      parent reply	other threads:[~2021-10-13 11:58 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-13  8:24 [pbs-devel] " Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 01/15] proxmox-rrd: use a journal to reduce amount of bytes written Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 02/15] RRD_CACHE: use a OnceCell instead of lazy_static Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 03/15] proxmox-backup-proxy: use tokio::task::spawn_blocking instead of block_in_place Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 04/15] proxmox-rrd: implement new CBOR based format Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 05/15] proxmox-rrd: remove dependency to proxmox-rrd-api-types Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 06/15] proxmox-rrd: extract_data: include values from current slot Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 07/15] remove proxmox-rrd-api-types crate, s/RRDTimeFrameResolution/RRDTimeFrame/ Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 08/15] proxmox-rrd: support CF::Last Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 09/15] proxmox-rrd: split out load_rrd (cleanup) Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 10/15] proxmox-rrd: add binary to create/manage rrd files Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 11/15] proxmox-rrd: avoid % inside loop Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 12/15] proxmox-rrd: new helper methods - slot() and slot_end_time() Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 13/15] proxmox-rrd: protect against negative update time Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 14/15] proxmox-rrd: rename last_counter to last_value Dietmar Maurer
2021-10-13  8:24 ` [pbs-devel] [PATCH proxmox-backup 15/15] proxmox-rrd: add more commands to the rrd cli tool Dietmar Maurer
2021-10-13 11:58 ` Thomas Lamprecht [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=32783331-e2c4-3a19-47b2-c89d670e6291@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=dietmar@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal