Re: [pbs-devel] [PATCH proxmox-backup] fix loading chunk_order from config

Re: [pbs-devel] [PATCH proxmox-backup] fix loading chunk_order from config

[Dietmar Maurer]

Maybe we can use the config version cache toö detect changes?

see proxmox-backup/pbs-config/src/config_version_cache.rs


> On 02/24/2022 8:57 AM Dominik Csapak <d.csapak@proxmox.com> wrote:
> 
>  
> when we updated the chunk-order for a datastore, it was not reloaded
> until a reload/restart of the daemon (or if the path or 'verify-new'
> option was changed)
> 
> add it to the options that triggers a reload of the datastore config
> 
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> are we maybe at the point where we want to calculate a checksum of the
> raw options for checking a reload trigger instead of the seperate
> options? i find it slightly irritating that i have to parse the option
> here twice when they have changed...
> 
> a checksum from the raw config lines(in deterministic order, e.g.
> alphabetically) would include every config change, but we would not
> have to maintain a list of options that trigger a change. this
> could prevent someone to forget that mechanism exists (like i did)
> 
>  pbs-datastore/src/datastore.rs | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
> 
> diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
> index 8397da00..e01da2ba 100644
> --- a/pbs-datastore/src/datastore.rs
> +++ b/pbs-datastore/src/datastore.rs
> @@ -76,7 +76,8 @@ impl DataStore {
>          if let Some(datastore) = map.get(name) {
>              // Compare Config - if changed, create new Datastore object!
>              if datastore.chunk_store.base() == path &&
> -                datastore.verify_new == config.verify_new.unwrap_or(false)
> +                datastore.verify_new == config.verify_new.unwrap_or(false) &&
> +                datastore.chunk_order == Self::parse_chunk_order(&config)?
>              {
>                  return Ok(datastore.clone());
>              }
> @@ -120,20 +121,22 @@ impl DataStore {
>              GarbageCollectionStatus::default()
>          };
>  
> -        let tuning: DatastoreTuning = serde_json::from_value(
> -            DatastoreTuning::API_SCHEMA.parse_property_string(config.tuning.as_deref().unwrap_or(""))?
> -        )?;
> -        let chunk_order = tuning.chunk_order.unwrap_or(ChunkOrder::Inode);
> -
>          Ok(Self {
>              chunk_store: Arc::new(chunk_store),
>              gc_mutex: Mutex::new(()),
>              last_gc_status: Mutex::new(gc_status),
>              verify_new: config.verify_new.unwrap_or(false),
> -            chunk_order,
> +            chunk_order: Self::parse_chunk_order(&config)?,
>          })
>      }
>  
> +    fn parse_chunk_order(config: &DataStoreConfig) -> Result<ChunkOrder, Error> {
> +        let tuning: DatastoreTuning = serde_json::from_value(
> +            DatastoreTuning::API_SCHEMA.parse_property_string(config.tuning.as_deref().unwrap_or(""))?
> +        )?;
> +        Ok(tuning.chunk_order.unwrap_or(ChunkOrder::Inode))
> +    }
> +
>      pub fn get_chunk_iterator(
>          &self,
>      ) -> Result<
> -- 
> 2.30.2
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox-backup] fix loading chunk_order from config

[Dominik Csapak]

when we updated the chunk-order for a datastore, it was not reloaded
until a reload/restart of the daemon (or if the path or 'verify-new'
option was changed)

add it to the options that triggers a reload of the datastore config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
are we maybe at the point where we want to calculate a checksum of the
raw options for checking a reload trigger instead of the seperate
options? i find it slightly irritating that i have to parse the option
here twice when they have changed...

a checksum from the raw config lines(in deterministic order, e.g.
alphabetically) would include every config change, but we would not
have to maintain a list of options that trigger a change. this
could prevent someone to forget that mechanism exists (like i did)

 pbs-datastore/src/datastore.rs | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
index 8397da00..e01da2ba 100644
--- a/pbs-datastore/src/datastore.rs
+++ b/pbs-datastore/src/datastore.rs
@@ -76,7 +76,8 @@ impl DataStore {
         if let Some(datastore) = map.get(name) {
             // Compare Config - if changed, create new Datastore object!
             if datastore.chunk_store.base() == path &&
-                datastore.verify_new == config.verify_new.unwrap_or(false)
+                datastore.verify_new == config.verify_new.unwrap_or(false) &&
+                datastore.chunk_order == Self::parse_chunk_order(&config)?
             {
                 return Ok(datastore.clone());
             }
@@ -120,20 +121,22 @@ impl DataStore {
             GarbageCollectionStatus::default()
         };
 
-        let tuning: DatastoreTuning = serde_json::from_value(
-            DatastoreTuning::API_SCHEMA.parse_property_string(config.tuning.as_deref().unwrap_or(""))?
-        )?;
-        let chunk_order = tuning.chunk_order.unwrap_or(ChunkOrder::Inode);
-
         Ok(Self {
             chunk_store: Arc::new(chunk_store),
             gc_mutex: Mutex::new(()),
             last_gc_status: Mutex::new(gc_status),
             verify_new: config.verify_new.unwrap_or(false),
-            chunk_order,
+            chunk_order: Self::parse_chunk_order(&config)?,
         })
     }
 
+    fn parse_chunk_order(config: &DataStoreConfig) -> Result<ChunkOrder, Error> {
+        let tuning: DatastoreTuning = serde_json::from_value(
+            DatastoreTuning::API_SCHEMA.parse_property_string(config.tuning.as_deref().unwrap_or(""))?
+        )?;
+        Ok(tuning.chunk_order.unwrap_or(ChunkOrder::Inode))
+    }
+
     pub fn get_chunk_iterator(
         &self,
     ) -> Result<
-- 
2.30.2