public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup] datastore: simplify update-datastore-cache socket command
@ 2024-07-02 12:29 Hannes Laimer
  2024-08-05 13:19 ` Hannes Laimer
  0 siblings, 1 reply; 5+ messages in thread
From: Hannes Laimer @ 2024-07-02 12:29 UTC (permalink / raw)
  To: pbs-devel

... by always dropping the cache entry and not allowing it to be
re-added based on the maintenance mode. The check is only needed for
`Lookup` operations since lookups with those wouldn't fail based on
maintenance mode alone. Already running tasks won't be affected and
since they're the only ones holding a reference to a DataStore, it,
and the file handles it holds will be dropped automatically once the
(last) task finishes.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
 pbs-datastore/src/datastore.rs  | 26 ++++++++++----------------
 src/api2/config/datastore.rs    |  7 +++----
 src/bin/proxmox-backup-proxy.rs |  6 +++---
 3 files changed, 16 insertions(+), 23 deletions(-)

diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
index f95da761..e1028778 100644
--- a/pbs-datastore/src/datastore.rs
+++ b/pbs-datastore/src/datastore.rs
@@ -190,10 +190,15 @@ impl DataStore {
             )?)
         };
 
+        let is_offline = config
+            .get_maintenance_mode()
+            .map_or(false, |m| m.is_offline());
         let datastore = DataStore::with_store_and_config(chunk_store, config, Some(digest))?;
 
         let datastore = Arc::new(datastore);
-        datastore_cache.insert(name.to_string(), datastore.clone());
+        if !is_offline {
+            datastore_cache.insert(name.to_string(), datastore.clone());
+        }
 
         Ok(Arc::new(Self {
             inner: datastore,
@@ -211,21 +216,10 @@ impl DataStore {
         Ok(())
     }
 
-    /// trigger clearing cache entry based on maintenance mode. Entry will only
-    /// be cleared iff there is no other task running, if there is, the end of the
-    /// last running task will trigger the clearing of the cache entry.
-    pub fn update_datastore_cache(name: &str) -> Result<(), Error> {
-        let (config, _digest) = pbs_config::datastore::config()?;
-        let datastore: DataStoreConfig = config.lookup("datastore", name)?;
-        if datastore
-            .get_maintenance_mode()
-            .map_or(false, |m| m.is_offline())
-        {
-            // the datastore drop handler does the checking if tasks are running and clears the
-            // cache entry, so we just have to trigger it here
-            let _ = DataStore::lookup_datastore(name, Some(Operation::Lookup));
-        }
-
+    /// removes datastore from cache
+    pub fn drop_from_datastore_cache(name: &str) -> Result<(), Error> {
+        let mut datastore_cache = DATASTORE_MAP.lock().unwrap();
+        datastore_cache.remove(name);
         Ok(())
     }
 
diff --git a/src/api2/config/datastore.rs b/src/api2/config/datastore.rs
index 02887b86..5cc3dfb6 100644
--- a/src/api2/config/datastore.rs
+++ b/src/api2/config/datastore.rs
@@ -398,16 +398,15 @@ pub fn update_datastore(
         data.tuning = update.tuning;
     }
 
-    let mut maintenance_mode_changed = false;
+    let mut drop_store_from_cache = false;
     if update.maintenance_mode.is_some() {
-        maintenance_mode_changed = data.maintenance_mode != update.maintenance_mode;
-
         let maintenance_mode = match update.maintenance_mode {
             Some(mode_str) => Some(MaintenanceMode::deserialize(
                 proxmox_schema::de::SchemaDeserializer::new(mode_str, &MaintenanceMode::API_SCHEMA),
             )?),
             None => None,
         };
+        drop_store_from_cache = maintenance_mode.as_ref().map_or(false, |m| m.is_offline());
         data.set_maintenance_mode(maintenance_mode)?;
     }
 
@@ -422,7 +421,7 @@ pub fn update_datastore(
     }
 
     // tell the proxy it might have to clear a cache entry
-    if maintenance_mode_changed {
+    if drop_store_from_cache {
         tokio::spawn(async move {
             if let Ok(proxy_pid) =
                 proxmox_rest_server::read_pid(pbs_buildcfg::PROXMOX_BACKUP_PROXY_PID_FN)
diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index 15444685..4c48defc 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -290,11 +290,11 @@ async fn run() -> Result<(), Error> {
         Ok(Value::Null)
     })?;
 
-    // clear cache entry for datastore that is in a specific maintenance mode
+    // clear cache entry for datastore
     command_sock.register_command("update-datastore-cache".to_string(), |value| {
         if let Some(name) = value.and_then(Value::as_str) {
-            if let Err(err) = DataStore::update_datastore_cache(name) {
-                log::error!("could not trigger update datastore cache: {err}");
+            if let Err(err) = DataStore::drop_from_datastore_cache(name) {
+                log::error!("could not drop datastore from cache: {err}");
             }
         }
         Ok(Value::Null)
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] datastore: simplify update-datastore-cache socket command
  2024-07-02 12:29 [pbs-devel] [PATCH proxmox-backup] datastore: simplify update-datastore-cache socket command Hannes Laimer
@ 2024-08-05 13:19 ` Hannes Laimer
  2024-08-07  9:41   ` Lukas Wagner
  0 siblings, 1 reply; 5+ messages in thread
From: Hannes Laimer @ 2024-08-05 13:19 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion

ping, still applies, and makes the logic more explicit and straight
forward

On Tue Jul 2, 2024 at 2:29 PM CEST, Hannes Laimer wrote:
> ... by always dropping the cache entry and not allowing it to be
> re-added based on the maintenance mode. The check is only needed for
> `Lookup` operations since lookups with those wouldn't fail based on
> maintenance mode alone. Already running tasks won't be affected and
> since they're the only ones holding a reference to a DataStore, it,
> and the file handles it holds will be dropped automatically once the
> (last) task finishes.
>
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
>  pbs-datastore/src/datastore.rs  | 26 ++++++++++----------------
>  src/api2/config/datastore.rs    |  7 +++----
>  src/bin/proxmox-backup-proxy.rs |  6 +++---
>  3 files changed, 16 insertions(+), 23 deletions(-)
>
> diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
> index f95da761..e1028778 100644
> --- a/pbs-datastore/src/datastore.rs
> +++ b/pbs-datastore/src/datastore.rs
> @@ -190,10 +190,15 @@ impl DataStore {
>              )?)
>          };
>  
> +        let is_offline = config
> +            .get_maintenance_mode()
> +            .map_or(false, |m| m.is_offline());
>          let datastore = DataStore::with_store_and_config(chunk_store, config, Some(digest))?;
>  
>          let datastore = Arc::new(datastore);
> -        datastore_cache.insert(name.to_string(), datastore.clone());
> +        if !is_offline {
> +            datastore_cache.insert(name.to_string(), datastore.clone());
> +        }
>  
>          Ok(Arc::new(Self {
>              inner: datastore,
> @@ -211,21 +216,10 @@ impl DataStore {
>          Ok(())
>      }
>  
> -    /// trigger clearing cache entry based on maintenance mode. Entry will only
> -    /// be cleared iff there is no other task running, if there is, the end of the
> -    /// last running task will trigger the clearing of the cache entry.
> -    pub fn update_datastore_cache(name: &str) -> Result<(), Error> {
> -        let (config, _digest) = pbs_config::datastore::config()?;
> -        let datastore: DataStoreConfig = config.lookup("datastore", name)?;
> -        if datastore
> -            .get_maintenance_mode()
> -            .map_or(false, |m| m.is_offline())
> -        {
> -            // the datastore drop handler does the checking if tasks are running and clears the
> -            // cache entry, so we just have to trigger it here
> -            let _ = DataStore::lookup_datastore(name, Some(Operation::Lookup));
> -        }
> -
> +    /// removes datastore from cache
> +    pub fn drop_from_datastore_cache(name: &str) -> Result<(), Error> {
> +        let mut datastore_cache = DATASTORE_MAP.lock().unwrap();
> +        datastore_cache.remove(name);
>          Ok(())
>      }
>  
> diff --git a/src/api2/config/datastore.rs b/src/api2/config/datastore.rs
> index 02887b86..5cc3dfb6 100644
> --- a/src/api2/config/datastore.rs
> +++ b/src/api2/config/datastore.rs
> @@ -398,16 +398,15 @@ pub fn update_datastore(
>          data.tuning = update.tuning;
>      }
>  
> -    let mut maintenance_mode_changed = false;
> +    let mut drop_store_from_cache = false;
>      if update.maintenance_mode.is_some() {
> -        maintenance_mode_changed = data.maintenance_mode != update.maintenance_mode;
> -
>          let maintenance_mode = match update.maintenance_mode {
>              Some(mode_str) => Some(MaintenanceMode::deserialize(
>                  proxmox_schema::de::SchemaDeserializer::new(mode_str, &MaintenanceMode::API_SCHEMA),
>              )?),
>              None => None,
>          };
> +        drop_store_from_cache = maintenance_mode.as_ref().map_or(false, |m| m.is_offline());
>          data.set_maintenance_mode(maintenance_mode)?;
>      }
>  
> @@ -422,7 +421,7 @@ pub fn update_datastore(
>      }
>  
>      // tell the proxy it might have to clear a cache entry
> -    if maintenance_mode_changed {
> +    if drop_store_from_cache {
>          tokio::spawn(async move {
>              if let Ok(proxy_pid) =
>                  proxmox_rest_server::read_pid(pbs_buildcfg::PROXMOX_BACKUP_PROXY_PID_FN)
> diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
> index 15444685..4c48defc 100644
> --- a/src/bin/proxmox-backup-proxy.rs
> +++ b/src/bin/proxmox-backup-proxy.rs
> @@ -290,11 +290,11 @@ async fn run() -> Result<(), Error> {
>          Ok(Value::Null)
>      })?;
>  
> -    // clear cache entry for datastore that is in a specific maintenance mode
> +    // clear cache entry for datastore
>      command_sock.register_command("update-datastore-cache".to_string(), |value| {
>          if let Some(name) = value.and_then(Value::as_str) {
> -            if let Err(err) = DataStore::update_datastore_cache(name) {
> -                log::error!("could not trigger update datastore cache: {err}");
> +            if let Err(err) = DataStore::drop_from_datastore_cache(name) {
> +                log::error!("could not drop datastore from cache: {err}");
>              }
>          }
>          Ok(Value::Null)



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] datastore: simplify update-datastore-cache socket command
  2024-08-05 13:19 ` Hannes Laimer
@ 2024-08-07  9:41   ` Lukas Wagner
  2024-08-07 12:11     ` Hannes Laimer
  0 siblings, 1 reply; 5+ messages in thread
From: Lukas Wagner @ 2024-08-07  9:41 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Hannes Laimer

On  2024-08-05 15:19, Hannes Laimer wrote:
> ping, still applies, and makes the logic more explicit and straight
> forward
> 

Does not apply cleanly for me anymore :) (needs git am -3)

-- 
- Lukas


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] datastore: simplify update-datastore-cache socket command
  2024-08-07  9:41   ` Lukas Wagner
@ 2024-08-07 12:11     ` Hannes Laimer
  2024-08-07 12:19       ` Lukas Wagner
  0 siblings, 1 reply; 5+ messages in thread
From: Hannes Laimer @ 2024-08-07 12:11 UTC (permalink / raw)
  To: Lukas Wagner, Proxmox Backup Server development discussion

On Wed Aug 7, 2024 at 11:41 AM CEST, Lukas Wagner wrote:
> On  2024-08-05 15:19, Hannes Laimer wrote:
> > ping, still applies, and makes the logic more explicit and straight
> > forward
> > 
>
> Does not apply cleanly for me anymore :) (needs git am -3)

Is 3way merge considered less clean? I mean, in my head "applies cleanly" <=> "no
conflicts", but maybe I'm missing something. I can send it again after
applying it with 3way merge and creating a new patch file.


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] datastore: simplify update-datastore-cache socket command
  2024-08-07 12:11     ` Hannes Laimer
@ 2024-08-07 12:19       ` Lukas Wagner
  0 siblings, 0 replies; 5+ messages in thread
From: Lukas Wagner @ 2024-08-07 12:19 UTC (permalink / raw)
  To: Hannes Laimer, Proxmox Backup Server development discussion

On  2024-08-07 14:11, Hannes Laimer wrote:
> On Wed Aug 7, 2024 at 11:41 AM CEST, Lukas Wagner wrote:
>> On  2024-08-05 15:19, Hannes Laimer wrote:
>>> ping, still applies, and makes the logic more explicit and straight
>>> forward
>>>
>>
>> Does not apply cleanly for me anymore :) (needs git am -3)
> 
> Is 3way merge considered less clean? I mean, in my head "applies cleanly" <=> "no
> conflicts", but maybe I'm missing something. I can send it again after
> applying it with 3way merge and creating a new patch file.

I would say for smaller stuff a 3-way merge on reviewer/applier side is fine, but for
bigger patches it's nicer if it can be merged without it. It gives me the assurance
that everything is applied just the way you tested it and want it to be.
There is always the chance that a 3-way merge introduces slight changes in behavior/
subtle bugs.

But that's just my 2ct on this, maybe others see this differently 🙂 .

-- 
- Lukas


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2024-08-07 12:19 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-07-02 12:29 [pbs-devel] [PATCH proxmox-backup] datastore: simplify update-datastore-cache socket command Hannes Laimer
2024-08-05 13:19 ` Hannes Laimer
2024-08-07  9:41   ` Lukas Wagner
2024-08-07 12:11     ` Hannes Laimer
2024-08-07 12:19       ` Lukas Wagner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal