From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <c.ebner@proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E434C93F4C for <pbs-devel@lists.proxmox.com>; Wed, 10 Apr 2024 12:04:35 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C4B4BB62B for <pbs-devel@lists.proxmox.com>; Wed, 10 Apr 2024 12:04:35 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for <pbs-devel@lists.proxmox.com>; Wed, 10 Apr 2024 12:04:31 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 4F41343C1A for <pbs-devel@lists.proxmox.com>; Wed, 10 Apr 2024 12:04:31 +0200 (CEST) Message-ID: <90dce332-4dfe-4165-96fe-c05ee2e427b1@proxmox.com> Date: Wed, 10 Apr 2024 12:04:29 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: =?UTF-8?Q?Fabian_Gr=C3=BCnbichler?= <f.gruenbichler@proxmox.com>, pbs-devel@lists.proxmox.com References: <20240328123707.336951-1-c.ebner@proxmox.com> <20240328123707.336951-55-c.ebner@proxmox.com> <171231220563.1926770.17713102356320222919@yuna.proxmox.com> Content-Language: en-US, de-DE From: Christian Ebner <c.ebner@proxmox.com> In-Reply-To: <171231220563.1926770.17713102356320222919@yuna.proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.030 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH v3 proxmox-backup 54/58] client: pxar: add flow chart for metadata change detection X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion <pbs-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/> List-Post: <mailto:pbs-devel@lists.proxmox.com> List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Wed, 10 Apr 2024 10:04:35 -0000 On 4/5/24 12:16, Fabian Grünbichler wrote: > Quoting Christian Ebner (2024-03-28 13:37:03) >> A high level flow chart describing the logic used for the metadata >> based file change detection mode. >> >> Signed-off-by: Christian Ebner <c.ebner@proxmox.com> >> --- >> changes since version 2: >> - not present in previous version >> >> ...ow-chart-metadata-based-file-change-detection.svg | 1 + >> ...ow-chart-metadata-based-file-change-detection.txt | 12 ++++++++++++ >> 2 files changed, 13 insertions(+) >> create mode 100644 pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.svg >> create mode 100644 pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.txt >> >> diff --git a/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.svg b/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.svg >> new file mode 100644 >> index 000000000..5e6df4815 >> --- /dev/null >> +++ b/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.svg >> @@ -0,0 +1 @@ > > [snip] > > something here got broken (I guess mail related somewhere along the way?). it > does work if the contents are manually merged backed into a single line before > applying the patch though. in any case, it probably would be nice to have this > autogenerated and moved to some part of the docs :) > >> diff --git a/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.txt b/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.txt >> new file mode 100644 >> index 000000000..5eace70be >> --- /dev/null >> +++ b/pbs-client/src/pxar/flow-chart-metadata-based-file-change-detection.txt >> @@ -0,0 +1,12 @@ >> +flowchart TD >> + A[Archiver] -->|lookup metadata| B[Accessor] >> + B -->|is reusable entry| C[Lookahead Cache] >> + C -->|lookup reusable chunks| D[Dynamic Index] >> + D -->|insert and deduplicate dynamic entries| E[Reused Chunks] >> + B -->|is not reusable entry| F(re-encode cached entries and current entry) >> + F -->|caching disabled| A >> + E -->|padding above threshold, non-continuous chunks, caching disabled| F >> + E -->|padding above threshold, chunks continuous, caching enabled| A >> + E -->|padding below threshold| G(force boundary, inject chunks, keepback last chunk for potential followup) >> + G -->|caching enabled| A > > the caching enabled/disabled parts here are probably confusing (do those edges > mean caching is enabled/disabled at that point? or are they taken if it is > enabled/disabled?) > > but probably it makes sense to re-visit this in detail once the dust has settled :) > Yes agreed, will drop this patch for now and rethink on how to include this better, or even move some of this into the documentation in text form (although in a more user oriented fashion rather than dev oriented).