public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* Re: [pbs-devel] Status of Garbage Collection marks in memory (not atime based)
@ 2021-04-09  8:13 Dietmar Maurer
       [not found] ` <d16a9279-ec9e-4f8e-905c-2744b4455083.4bf6da22-5cda-4597-b16d-3c88c4c3fb76.9e105e32-a0ed-438f-8146-0b54ba60255f@emailsignatures365.codetwo.com>
  2021-04-09 10:57 ` Maximilian Hill
  0 siblings, 2 replies; 6+ messages in thread
From: Dietmar Maurer @ 2021-04-09  8:13 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Marco Gabriel - inett GmbH

> We use a CephFS as datastore in our test environment and this is rather slow because of the metadata updates. The in-memory based approach could possibly make a huge difference for CephFS based datastores.

Besides, Proxmox Backup is designed to work with local storage. 
Using any Network based storage create a big amount of additional network 
traffic, and makes no real sense to me. You want(should) backup your Cepfs data 
to Proxmox Backup, not the other way around.




^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [pbs-devel] Status of Garbage Collection marks in memory (not atime based)
@ 2021-04-09 11:07 Dietmar Maurer
  0 siblings, 0 replies; 6+ messages in thread
From: Dietmar Maurer @ 2021-04-09 11:07 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Marco Gabriel


> >> We use a CephFS as datastore in our test environment and this is rather slow because of the metadata updates. The in-memory based approach could possibly make a huge difference for CephFS based datastores.
> > Using any Network based storage create a big amount of additional network
> > traffic, and makes no real sense to me. You want(should) backup your Cepfs data
> > to Proxmox Backup, not the other way around.
> >
> While this is totally true in most environments, it prevents a scale-out 
> approach where one server is not able to handle everything. That's the 
> whole wish behind it.

But the idea behind backup is to have redundant copies of your data at
different physical locations...

Anyways, I got your point. As mentioned in the code, it would be possible to 
optimize that using an in-memory hash.




^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [pbs-devel] Status of Garbage Collection marks in memory (not atime based)
@ 2021-04-09  8:08 Dietmar Maurer
  0 siblings, 0 replies; 6+ messages in thread
From: Dietmar Maurer @ 2021-04-09  8:08 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Marco Gabriel - inett GmbH

> on https://git.proxmox.com/?p=proxmox-backup.git;a=blob;f=src/backup.rs;h=cca438818b49724e10594ae2e12302f5fbf021df;hb=HEAD I saw, that besides the currently implemented atime based garbage collection approach, another approach using hashes in memory is under investigation.
> 
> I couldn't find any references besides the lines under that link. Is this investigation still ongoing? Is there any publicly available source?

Nobody is working on that - it is just an idea so far.




^ permalink raw reply	[flat|nested] 6+ messages in thread
* [pbs-devel] Status of Garbage Collection marks in memory (not atime based)
@ 2021-04-08 12:14 Marco Gabriel - inett GmbH
  0 siblings, 0 replies; 6+ messages in thread
From: Marco Gabriel - inett GmbH @ 2021-04-08 12:14 UTC (permalink / raw)
  To: pbs-devel

[-- Attachment #1: Type: text/plain, Size: 659 bytes --]

Hi,

on https://git.proxmox.com/?p=proxmox-backup.git;a=blob;f=src/backup.rs;h=cca438818b49724e10594ae2e12302f5fbf021df;hb=HEAD I saw, that besides the currently implemented atime based garbage collection approach, another approach using hashes in memory is under investigation.

I couldn't find any references besides the lines under that link. Is this investigation still ongoing? Is there any publicly available source?

We use a CephFS as datastore in our test environment and this is rather slow because of the metadata updates. The in-memory based approach could possibly make a huge difference for CephFS based datastores.

Thanks,
Marco



[-- Attachment #2: Type: text/html, Size: 2454 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-04-09 11:07 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-09  8:13 [pbs-devel] Status of Garbage Collection marks in memory (not atime based) Dietmar Maurer
     [not found] ` <d16a9279-ec9e-4f8e-905c-2744b4455083.4bf6da22-5cda-4597-b16d-3c88c4c3fb76.9e105e32-a0ed-438f-8146-0b54ba60255f@emailsignatures365.codetwo.com>
2021-04-09  9:17   ` Marco Gabriel
2021-04-09 10:57 ` Maximilian Hill
  -- strict thread matches above, loose matches on Subject: below --
2021-04-09 11:07 Dietmar Maurer
2021-04-09  8:08 Dietmar Maurer
2021-04-08 12:14 Marco Gabriel - inett GmbH

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal