From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dietmar@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id CC40571C66
 for <pbs-devel@lists.proxmox.com>; Fri,  9 Apr 2021 13:07:17 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id BE98125B5C
 for <pbs-devel@lists.proxmox.com>; Fri,  9 Apr 2021 13:07:17 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [212.186.127.180])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS id 5C23C25B4C
 for <pbs-devel@lists.proxmox.com>; Fri,  9 Apr 2021 13:07:16 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 26B3941FD3;
 Fri,  9 Apr 2021 13:07:16 +0200 (CEST)
Date: Fri, 9 Apr 2021 13:07:06 +0200 (CEST)
From: Dietmar Maurer <dietmar@proxmox.com>
To: Proxmox Backup Server development discussion <pbs-devel@lists.proxmox.com>, 
 Marco Gabriel <mgabriel@inett.de>
Message-ID: <246338782.1841.1617966426249@webmail.proxmox.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Priority: 3
Importance: Normal
X-Mailer: Open-Xchange Mailer v7.10.5-Rev5
X-Originating-Client: open-xchange-appsuite
X-SPAM-LEVEL: Spam detection results:  0
 AWL 0.111 Adjusted score from AWL reputation of From: address
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 RCVD_IN_DNSWL_MED        -2.3 Sender listed at https://www.dnswl.org/,
 medium trust
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
Subject: Re: [pbs-devel] Status of Garbage Collection marks in memory (not
 atime based)
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
 <pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Fri, 09 Apr 2021 11:07:17 -0000


> >> We use a CephFS as datastore in our test environment and this is rather slow because of the metadata updates. The in-memory based approach could possibly make a huge difference for CephFS based datastores.
> > Using any Network based storage create a big amount of additional network
> > traffic, and makes no real sense to me. You want(should) backup your Cepfs data
> > to Proxmox Backup, not the other way around.
> >
> While this is totally true in most environments, it prevents a scale-out 
> approach where one server is not able to handle everything. That's the 
> whole wish behind it.

But the idea behind backup is to have redundant copies of your data at
different physical locations...

Anyways, I got your point. As mentioned in the code, it would be possible to 
optimize that using an in-memory hash.