From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 748A57555C for ; Wed, 13 Oct 2021 13:58:28 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 58E4210F15 for ; Wed, 13 Oct 2021 13:58:28 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 26F2710F05 for ; Wed, 13 Oct 2021 13:58:27 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id D190845DB2 for ; Wed, 13 Oct 2021 13:58:26 +0200 (CEST) Message-ID: <32783331-e2c4-3a19-47b2-c89d670e6291@proxmox.com> Date: Wed, 13 Oct 2021 13:58:26 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Thunderbird/94.0 Content-Language: en-US To: Proxmox Backup Server development discussion , Dietmar Maurer References: <20211013082452.619406-1-dietmar@proxmox.com> From: Thomas Lamprecht In-Reply-To: <20211013082452.619406-1-dietmar@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.205 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] applied-series: [PATCH proxmox-backup 00/15] RRD database improvements X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Oct 2021 11:58:28 -0000 On 13.10.21 10:24, Dietmar Maurer wrote: > - use a journal. This way we can reduce the overall number of bytes > written by increasing the flush/commit interval (30 minutes). > > - the new CBOR base format is more flexible, and we store much more > data points now. > > We previously wrote about 7MB/h. With the new format and journal, we now write about 3MB/h. > > Dietmar Maurer (15): > proxmox-rrd: use a journal to reduce amount of bytes written > RRD_CACHE: use a OnceCell instead of lazy_static > proxmox-backup-proxy: use tokio::task::spawn_blocking instead of > block_in_place > proxmox-rrd: implement new CBOR based format > proxmox-rrd: remove dependency to proxmox-rrd-api-types > proxmox-rrd: extract_data: include values from current slot > remove proxmox-rrd-api-types crate, > s/RRDTimeFrameResolution/RRDTimeFrame/ > proxmox-rrd: support CF::Last > proxmox-rrd: split out load_rrd (cleanup) > proxmox-rrd: add binary to create/manage rrd files > proxmox-rrd: avoid % inside loop > proxmox-rrd: new helper methods - slot() and slot_end_time() > proxmox-rrd: protect against negative update time > proxmox-rrd: rename last_counter to last_value > proxmox-rrd: add more commands to the rrd cli tool applied whole series, with the diff you replied on patch 14/15 squashed in and the rrd debug tool moved to examples as quick "fix" for avoiding proxmox-router as hard dependency on the crate for now, thanks!