public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup] gc: attach context to index reader errors and ignore NotFound
@ 2020-09-08  9:18 Stefan Reiter
  2020-09-08 10:07 ` Dietmar Maurer
  2020-09-08 11:12 ` Fabian Grünbichler
  0 siblings, 2 replies; 5+ messages in thread
From: Stefan Reiter @ 2020-09-08  9:18 UTC (permalink / raw)
  To: pbs-devel

Ignore NotFound errors during phase 1, this just means that a snapshot
was forgotten or pruned between scanning for .fidx/.didx files and
actually opening the index to touch the chunks.

ignore_notfound has to be a real function, since generics are not
supported for closures.

The open methods for dynamic and fixed indices are switched from the
usual format_err! to err.context() to allow checking for the root error
(and thus the io::ErrorKind) further up the call chain.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
 src/backup/datastore.rs     | 28 ++++++++++++++++++++++++----
 src/backup/dynamic_index.rs |  5 ++++-
 src/backup/fixed_index.rs   |  7 +++++--
 3 files changed, 33 insertions(+), 7 deletions(-)

diff --git a/src/backup/datastore.rs b/src/backup/datastore.rs
index ebe47487..1f455d64 100644
--- a/src/backup/datastore.rs
+++ b/src/backup/datastore.rs
@@ -426,6 +426,20 @@ impl DataStore {
         Ok(())
     }
 
+    fn ignore_notfound<T>(res: Result<T, Error>) -> Result<Option<T>, Error> {
+        match res {
+            Ok(t) => Ok(Some(t)),
+            Err(err) => {
+                if let Some(ioerr) = err.downcast_ref::<std::io::Error>() {
+                    if ioerr.kind() == std::io::ErrorKind::NotFound {
+                        return Ok(None);
+                    }
+                }
+                Err(err)
+            }
+        }
+    }
+
     fn mark_used_chunks(&self, status: &mut GarbageCollectionStatus, worker: &WorkerTask) -> Result<(), Error> {
 
         let image_list = self.list_images()?;
@@ -443,11 +457,17 @@ impl DataStore {
 
             if let Ok(archive_type) = archive_type(&path) {
                 if archive_type == ArchiveType::FixedIndex {
-                    let index = self.open_fixed_reader(&path)?;
-                    self.index_mark_used_chunks(index, &path, status, worker)?;
+                    if let Some(index) = Self::ignore_notfound(self.open_fixed_reader(&path))? {
+                        self.index_mark_used_chunks(index, &path, status, worker)?;
+                    } else {
+                        worker.warn(format!("warning: could no longer find fixed index '{:?}'", &path));
+                    }
                 } else if archive_type == ArchiveType::DynamicIndex {
-                    let index = self.open_dynamic_reader(&path)?;
-                    self.index_mark_used_chunks(index, &path, status, worker)?;
+                    if let Some(index) = Self::ignore_notfound(self.open_dynamic_reader(&path))? {
+                        self.index_mark_used_chunks(index, &path, status, worker)?;
+                    } else {
+                        worker.warn(format!("warning: could no longer find dynamic index '{:?}'", &path));
+                    }
                 }
             }
             done += 1;
diff --git a/src/backup/dynamic_index.rs b/src/backup/dynamic_index.rs
index f70aa44f..a7ce0f24 100644
--- a/src/backup/dynamic_index.rs
+++ b/src/backup/dynamic_index.rs
@@ -86,7 +86,10 @@ impl DynamicIndexReader {
         File::open(path)
             .map_err(Error::from)
             .and_then(Self::new)
-            .map_err(|err| format_err!("Unable to open dynamic index {:?} - {}", path, err))
+            .map_err(|err| {
+                let msg = format!("Unable to open dynamic index {:?} - {}", path, err);
+                err.context(msg)
+            })
     }
 
     pub fn new(mut file: std::fs::File) -> Result<Self, Error> {
diff --git a/src/backup/fixed_index.rs b/src/backup/fixed_index.rs
index 5d6cc1ff..bf864173 100644
--- a/src/backup/fixed_index.rs
+++ b/src/backup/fixed_index.rs
@@ -1,4 +1,4 @@
-use anyhow::{bail, format_err, Error};
+use anyhow::{bail, Error};
 use std::io::{Seek, SeekFrom};
 
 use super::chunk_stat::*;
@@ -62,7 +62,10 @@ impl FixedIndexReader {
         File::open(path)
             .map_err(Error::from)
             .and_then(|file| Self::new(file))
-            .map_err(|err| format_err!("Unable to open fixed index {:?} - {}", path, err))
+            .map_err(|err| {
+                let msg = format!("Unable to open fixed index {:?} - {}", path, err);
+                err.context(msg)
+            })
     }
 
     pub fn new(mut file: std::fs::File) -> Result<Self, Error> {
-- 
2.20.1





^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] gc: attach context to index reader errors and ignore NotFound
  2020-09-08  9:18 [pbs-devel] [PATCH proxmox-backup] gc: attach context to index reader errors and ignore NotFound Stefan Reiter
@ 2020-09-08 10:07 ` Dietmar Maurer
  2020-09-08 11:12 ` Fabian Grünbichler
  1 sibling, 0 replies; 5+ messages in thread
From: Dietmar Maurer @ 2020-09-08 10:07 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Stefan Reiter

I would prefer to return std::io::Error instead:

impl FixedIndexReader {
-    pub fn open(path: &Path) -> Result<Self, Error> {
+    pub fn open(path: &Path) -> Result<Self, std::io::Error> {

Wolfgang wrote some macros for that;

proxmox::io_bail!() and proxmox::io_format_err!()


> On 09/08/2020 11:18 AM Stefan Reiter <s.reiter@proxmox.com> wrote:
> 
>  
> Ignore NotFound errors during phase 1, this just means that a snapshot
> was forgotten or pruned between scanning for .fidx/.didx files and
> actually opening the index to touch the chunks.
> 
> ignore_notfound has to be a real function, since generics are not
> supported for closures.
> 
> The open methods for dynamic and fixed indices are switched from the
> usual format_err! to err.context() to allow checking for the root error
> (and thus the io::ErrorKind) further up the call chain.
> 
> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
> ---
>  src/backup/datastore.rs     | 28 ++++++++++++++++++++++++----
>  src/backup/dynamic_index.rs |  5 ++++-
>  src/backup/fixed_index.rs   |  7 +++++--
>  3 files changed, 33 insertions(+), 7 deletions(-)
> 
> diff --git a/src/backup/datastore.rs b/src/backup/datastore.rs
> index ebe47487..1f455d64 100644
> --- a/src/backup/datastore.rs
> +++ b/src/backup/datastore.rs
> @@ -426,6 +426,20 @@ impl DataStore {
>          Ok(())
>      }
>  
> +    fn ignore_notfound<T>(res: Result<T, Error>) -> Result<Option<T>, Error> {
> +        match res {
> +            Ok(t) => Ok(Some(t)),
> +            Err(err) => {
> +                if let Some(ioerr) = err.downcast_ref::<std::io::Error>() {
> +                    if ioerr.kind() == std::io::ErrorKind::NotFound {
> +                        return Ok(None);
> +                    }
> +                }
> +                Err(err)
> +            }
> +        }
> +    }
> +
>      fn mark_used_chunks(&self, status: &mut GarbageCollectionStatus, worker: &WorkerTask) -> Result<(), Error> {
>  
>          let image_list = self.list_images()?;
> @@ -443,11 +457,17 @@ impl DataStore {
>  
>              if let Ok(archive_type) = archive_type(&path) {
>                  if archive_type == ArchiveType::FixedIndex {
> -                    let index = self.open_fixed_reader(&path)?;
> -                    self.index_mark_used_chunks(index, &path, status, worker)?;
> +                    if let Some(index) = Self::ignore_notfound(self.open_fixed_reader(&path))? {
> +                        self.index_mark_used_chunks(index, &path, status, worker)?;
> +                    } else {
> +                        worker.warn(format!("warning: could no longer find fixed index '{:?}'", &path));
> +                    }
>                  } else if archive_type == ArchiveType::DynamicIndex {
> -                    let index = self.open_dynamic_reader(&path)?;
> -                    self.index_mark_used_chunks(index, &path, status, worker)?;
> +                    if let Some(index) = Self::ignore_notfound(self.open_dynamic_reader(&path))? {
> +                        self.index_mark_used_chunks(index, &path, status, worker)?;
> +                    } else {
> +                        worker.warn(format!("warning: could no longer find dynamic index '{:?}'", &path));
> +                    }
>                  }
>              }
>              done += 1;
> diff --git a/src/backup/dynamic_index.rs b/src/backup/dynamic_index.rs
> index f70aa44f..a7ce0f24 100644
> --- a/src/backup/dynamic_index.rs
> +++ b/src/backup/dynamic_index.rs
> @@ -86,7 +86,10 @@ impl DynamicIndexReader {
>          File::open(path)
>              .map_err(Error::from)
>              .and_then(Self::new)
> -            .map_err(|err| format_err!("Unable to open dynamic index {:?} - {}", path, err))
> +            .map_err(|err| {
> +                let msg = format!("Unable to open dynamic index {:?} - {}", path, err);
> +                err.context(msg)
> +            })
>      }
>  
>      pub fn new(mut file: std::fs::File) -> Result<Self, Error> {
> diff --git a/src/backup/fixed_index.rs b/src/backup/fixed_index.rs
> index 5d6cc1ff..bf864173 100644
> --- a/src/backup/fixed_index.rs
> +++ b/src/backup/fixed_index.rs
> @@ -1,4 +1,4 @@
> -use anyhow::{bail, format_err, Error};
> +use anyhow::{bail, Error};
>  use std::io::{Seek, SeekFrom};
>  
>  use super::chunk_stat::*;
> @@ -62,7 +62,10 @@ impl FixedIndexReader {
>          File::open(path)
>              .map_err(Error::from)
>              .and_then(|file| Self::new(file))
> -            .map_err(|err| format_err!("Unable to open fixed index {:?} - {}", path, err))
> +            .map_err(|err| {
> +                let msg = format!("Unable to open fixed index {:?} - {}", path, err);
> +                err.context(msg)
> +            })
>      }
>  
>      pub fn new(mut file: std::fs::File) -> Result<Self, Error> {
> -- 
> 2.20.1
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel




^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] gc: attach context to index reader errors and ignore NotFound
  2020-09-08  9:18 [pbs-devel] [PATCH proxmox-backup] gc: attach context to index reader errors and ignore NotFound Stefan Reiter
  2020-09-08 10:07 ` Dietmar Maurer
@ 2020-09-08 11:12 ` Fabian Grünbichler
  2020-09-08 11:18   ` Stefan Reiter
  1 sibling, 1 reply; 5+ messages in thread
From: Fabian Grünbichler @ 2020-09-08 11:12 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion

On September 8, 2020 11:18 am, Stefan Reiter wrote:
> Ignore NotFound errors during phase 1, this just means that a snapshot
> was forgotten or pruned between scanning for .fidx/.didx files and
> actually opening the index to touch the chunks.

I originally had a similar patch already lying around, but I am not sure 
whether this is not too dangerous in the face of transient errors?

I'd much rather get to a point where we are sure that no concurrent 
prune/forget operation can happen, and treat all errors as errors, 
instead of treating all not found errors as benign 'must have happened 
cause of concurrent actions'.

this is not pull, or download/restore, where we can just retry later - 
if we skip the index here, all the chunks it referenced are up for 
garbage collection unless they are saved by another index!

> 
> ignore_notfound has to be a real function, since generics are not
> supported for closures.
> 
> The open methods for dynamic and fixed indices are switched from the
> usual format_err! to err.context() to allow checking for the root error
> (and thus the io::ErrorKind) further up the call chain.
> 
> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
> ---
>  src/backup/datastore.rs     | 28 ++++++++++++++++++++++++----
>  src/backup/dynamic_index.rs |  5 ++++-
>  src/backup/fixed_index.rs   |  7 +++++--
>  3 files changed, 33 insertions(+), 7 deletions(-)
> 
> diff --git a/src/backup/datastore.rs b/src/backup/datastore.rs
> index ebe47487..1f455d64 100644
> --- a/src/backup/datastore.rs
> +++ b/src/backup/datastore.rs
> @@ -426,6 +426,20 @@ impl DataStore {
>          Ok(())
>      }
>  
> +    fn ignore_notfound<T>(res: Result<T, Error>) -> Result<Option<T>, Error> {
> +        match res {
> +            Ok(t) => Ok(Some(t)),
> +            Err(err) => {
> +                if let Some(ioerr) = err.downcast_ref::<std::io::Error>() {
> +                    if ioerr.kind() == std::io::ErrorKind::NotFound {
> +                        return Ok(None);
> +                    }
> +                }
> +                Err(err)
> +            }
> +        }
> +    }
> +
>      fn mark_used_chunks(&self, status: &mut GarbageCollectionStatus, worker: &WorkerTask) -> Result<(), Error> {
>  
>          let image_list = self.list_images()?;
> @@ -443,11 +457,17 @@ impl DataStore {
>  
>              if let Ok(archive_type) = archive_type(&path) {
>                  if archive_type == ArchiveType::FixedIndex {
> -                    let index = self.open_fixed_reader(&path)?;
> -                    self.index_mark_used_chunks(index, &path, status, worker)?;
> +                    if let Some(index) = Self::ignore_notfound(self.open_fixed_reader(&path))? {
> +                        self.index_mark_used_chunks(index, &path, status, worker)?;
> +                    } else {
> +                        worker.warn(format!("warning: could no longer find fixed index '{:?}'", &path));
> +                    }
>                  } else if archive_type == ArchiveType::DynamicIndex {
> -                    let index = self.open_dynamic_reader(&path)?;
> -                    self.index_mark_used_chunks(index, &path, status, worker)?;
> +                    if let Some(index) = Self::ignore_notfound(self.open_dynamic_reader(&path))? {
> +                        self.index_mark_used_chunks(index, &path, status, worker)?;
> +                    } else {
> +                        worker.warn(format!("warning: could no longer find dynamic index '{:?}'", &path));
> +                    }
>                  }
>              }
>              done += 1;
> diff --git a/src/backup/dynamic_index.rs b/src/backup/dynamic_index.rs
> index f70aa44f..a7ce0f24 100644
> --- a/src/backup/dynamic_index.rs
> +++ b/src/backup/dynamic_index.rs
> @@ -86,7 +86,10 @@ impl DynamicIndexReader {
>          File::open(path)
>              .map_err(Error::from)
>              .and_then(Self::new)
> -            .map_err(|err| format_err!("Unable to open dynamic index {:?} - {}", path, err))
> +            .map_err(|err| {
> +                let msg = format!("Unable to open dynamic index {:?} - {}", path, err);
> +                err.context(msg)
> +            })
>      }
>  
>      pub fn new(mut file: std::fs::File) -> Result<Self, Error> {
> diff --git a/src/backup/fixed_index.rs b/src/backup/fixed_index.rs
> index 5d6cc1ff..bf864173 100644
> --- a/src/backup/fixed_index.rs
> +++ b/src/backup/fixed_index.rs
> @@ -1,4 +1,4 @@
> -use anyhow::{bail, format_err, Error};
> +use anyhow::{bail, Error};
>  use std::io::{Seek, SeekFrom};
>  
>  use super::chunk_stat::*;
> @@ -62,7 +62,10 @@ impl FixedIndexReader {
>          File::open(path)
>              .map_err(Error::from)
>              .and_then(|file| Self::new(file))
> -            .map_err(|err| format_err!("Unable to open fixed index {:?} - {}", path, err))
> +            .map_err(|err| {
> +                let msg = format!("Unable to open fixed index {:?} - {}", path, err);
> +                err.context(msg)
> +            })
>      }
>  
>      pub fn new(mut file: std::fs::File) -> Result<Self, Error> {
> -- 
> 2.20.1
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
> 




^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] gc: attach context to index reader errors and ignore NotFound
  2020-09-08 11:12 ` Fabian Grünbichler
@ 2020-09-08 11:18   ` Stefan Reiter
  2020-09-09  6:01     ` Fabian Grünbichler
  0 siblings, 1 reply; 5+ messages in thread
From: Stefan Reiter @ 2020-09-08 11:18 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Fabian Grünbichler

On 9/8/20 1:12 PM, Fabian Grünbichler wrote:
> On September 8, 2020 11:18 am, Stefan Reiter wrote:
>> Ignore NotFound errors during phase 1, this just means that a snapshot
>> was forgotten or pruned between scanning for .fidx/.didx files and
>> actually opening the index to touch the chunks.
> 
> I originally had a similar patch already lying around, but I am not sure
> whether this is not too dangerous in the face of transient errors?
> 
> I'd much rather get to a point where we are sure that no concurrent
> prune/forget operation can happen, and treat all errors as errors,
> instead of treating all not found errors as benign 'must have happened
> cause of concurrent actions'.
> 

So no forget/prune during phase 1 of GC? That sounds like it would cause 
quite some congestion.

> this is not pull, or download/restore, where we can just retry later -
> if we skip the index here, all the chunks it referenced are up for
> garbage collection unless they are saved by another index!
> 

I do see where you're coming from, but what alternative is there? If the 
index file is not found, we can't touch any referenced chunks anyway - 
there are none for us to see.

>>
>> ignore_notfound has to be a real function, since generics are not
>> supported for closures.
>>
>> The open methods for dynamic and fixed indices are switched from the
>> usual format_err! to err.context() to allow checking for the root error
>> (and thus the io::ErrorKind) further up the call chain.
>>
>> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
>> ---
>>   src/backup/datastore.rs     | 28 ++++++++++++++++++++++++----
>>   src/backup/dynamic_index.rs |  5 ++++-
>>   src/backup/fixed_index.rs   |  7 +++++--
>>   3 files changed, 33 insertions(+), 7 deletions(-)
>>
>> diff --git a/src/backup/datastore.rs b/src/backup/datastore.rs
>> index ebe47487..1f455d64 100644
>> --- a/src/backup/datastore.rs
>> +++ b/src/backup/datastore.rs
>> @@ -426,6 +426,20 @@ impl DataStore {
>>           Ok(())
>>       }
>>   
>> +    fn ignore_notfound<T>(res: Result<T, Error>) -> Result<Option<T>, Error> {
>> +        match res {
>> +            Ok(t) => Ok(Some(t)),
>> +            Err(err) => {
>> +                if let Some(ioerr) = err.downcast_ref::<std::io::Error>() {
>> +                    if ioerr.kind() == std::io::ErrorKind::NotFound {
>> +                        return Ok(None);
>> +                    }
>> +                }
>> +                Err(err)
>> +            }
>> +        }
>> +    }
>> +
>>       fn mark_used_chunks(&self, status: &mut GarbageCollectionStatus, worker: &WorkerTask) -> Result<(), Error> {
>>   
>>           let image_list = self.list_images()?;
>> @@ -443,11 +457,17 @@ impl DataStore {
>>   
>>               if let Ok(archive_type) = archive_type(&path) {
>>                   if archive_type == ArchiveType::FixedIndex {
>> -                    let index = self.open_fixed_reader(&path)?;
>> -                    self.index_mark_used_chunks(index, &path, status, worker)?;
>> +                    if let Some(index) = Self::ignore_notfound(self.open_fixed_reader(&path))? {
>> +                        self.index_mark_used_chunks(index, &path, status, worker)?;
>> +                    } else {
>> +                        worker.warn(format!("warning: could no longer find fixed index '{:?}'", &path));
>> +                    }
>>                   } else if archive_type == ArchiveType::DynamicIndex {
>> -                    let index = self.open_dynamic_reader(&path)?;
>> -                    self.index_mark_used_chunks(index, &path, status, worker)?;
>> +                    if let Some(index) = Self::ignore_notfound(self.open_dynamic_reader(&path))? {
>> +                        self.index_mark_used_chunks(index, &path, status, worker)?;
>> +                    } else {
>> +                        worker.warn(format!("warning: could no longer find dynamic index '{:?}'", &path));
>> +                    }
>>                   }
>>               }
>>               done += 1;
>> diff --git a/src/backup/dynamic_index.rs b/src/backup/dynamic_index.rs
>> index f70aa44f..a7ce0f24 100644
>> --- a/src/backup/dynamic_index.rs
>> +++ b/src/backup/dynamic_index.rs
>> @@ -86,7 +86,10 @@ impl DynamicIndexReader {
>>           File::open(path)
>>               .map_err(Error::from)
>>               .and_then(Self::new)
>> -            .map_err(|err| format_err!("Unable to open dynamic index {:?} - {}", path, err))
>> +            .map_err(|err| {
>> +                let msg = format!("Unable to open dynamic index {:?} - {}", path, err);
>> +                err.context(msg)
>> +            })
>>       }
>>   
>>       pub fn new(mut file: std::fs::File) -> Result<Self, Error> {
>> diff --git a/src/backup/fixed_index.rs b/src/backup/fixed_index.rs
>> index 5d6cc1ff..bf864173 100644
>> --- a/src/backup/fixed_index.rs
>> +++ b/src/backup/fixed_index.rs
>> @@ -1,4 +1,4 @@
>> -use anyhow::{bail, format_err, Error};
>> +use anyhow::{bail, Error};
>>   use std::io::{Seek, SeekFrom};
>>   
>>   use super::chunk_stat::*;
>> @@ -62,7 +62,10 @@ impl FixedIndexReader {
>>           File::open(path)
>>               .map_err(Error::from)
>>               .and_then(|file| Self::new(file))
>> -            .map_err(|err| format_err!("Unable to open fixed index {:?} - {}", path, err))
>> +            .map_err(|err| {
>> +                let msg = format!("Unable to open fixed index {:?} - {}", path, err);
>> +                err.context(msg)
>> +            })
>>       }
>>   
>>       pub fn new(mut file: std::fs::File) -> Result<Self, Error> {
>> -- 
>> 2.20.1
>>
>>
>>
>> _______________________________________________
>> pbs-devel mailing list
>> pbs-devel@lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
>>
>>
>>
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 




^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] gc: attach context to index reader errors and ignore NotFound
  2020-09-08 11:18   ` Stefan Reiter
@ 2020-09-09  6:01     ` Fabian Grünbichler
  0 siblings, 0 replies; 5+ messages in thread
From: Fabian Grünbichler @ 2020-09-09  6:01 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Stefan Reiter

On September 8, 2020 1:18 pm, Stefan Reiter wrote:
> On 9/8/20 1:12 PM, Fabian Grünbichler wrote:
>> On September 8, 2020 11:18 am, Stefan Reiter wrote:
>>> Ignore NotFound errors during phase 1, this just means that a snapshot
>>> was forgotten or pruned between scanning for .fidx/.didx files and
>>> actually opening the index to touch the chunks.
>> 
>> I originally had a similar patch already lying around, but I am not sure
>> whether this is not too dangerous in the face of transient errors?
>> 
>> I'd much rather get to a point where we are sure that no concurrent
>> prune/forget operation can happen, and treat all errors as errors,
>> instead of treating all not found errors as benign 'must have happened
>> cause of concurrent actions'.
>> 
> 
> So no forget/prune during phase 1 of GC? That sounds like it would cause 
> quite some congestion.

or locking and touching group-wise, to reduce granularity and 
contention? or let prune/forget wait until GC phase 1 is over, by having 
a higher lock timeout?

phase 1 does not take too long here, but it probably depends a lot on 
datastore setup and size (special vdevs and enough RAM for caching 
probably help a lot here..)

we could also just mark them as deleted (touch $snapshot/.deleted) and 
let GC do the actual deletion of metadata as well, but that would be a 
much more involved change. added benefit that GC is now the only thing 
that deletes stuff (except for cleanup of aborted backup tasks, but that 
could also switch to that mechanism I guess).

> 
>> this is not pull, or download/restore, where we can just retry later -
>> if we skip the index here, all the chunks it referenced are up for
>> garbage collection unless they are saved by another index!
>> 
> 
> I do see where you're coming from, but what alternative is there? If the 
> index file is not found, we can't touch any referenced chunks anyway - 
> there are none for us to see.

the alternatives are

A) treat index files which we expected to read that have vanished as 
'must be benign', and continue GC
B) try to not have a scenario where that can happen benignly (e.g., 
because of a mutex between operations that delete indices and this phase 
of GC), so that we can know that it is an error and treat it as such

I'd like to choose B since it is the safe alternative, and this is the 
one path where having a bug could wipe out whole datastores, but if it's 
too involved then we have to go with A




^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-09-09  6:02 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-08  9:18 [pbs-devel] [PATCH proxmox-backup] gc: attach context to index reader errors and ignore NotFound Stefan Reiter
2020-09-08 10:07 ` Dietmar Maurer
2020-09-08 11:12 ` Fabian Grünbichler
2020-09-08 11:18   ` Stefan Reiter
2020-09-09  6:01     ` Fabian Grünbichler

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal