[RFC proxmox{,-backup} 0/6] add scheduled fstrim job for datastore's backing filesystems

[RFC proxmox{,-backup} 0/6] add scheduled fstrim job for datastore's backing filesystems

[Christian Ebner]

As reported in the community forum [0], the default systemd service
to run fstrim does not cover datastores mounted via systemd mount
unit, since the fstrim command is invoked via:
```
fstrim --listed-in /etc/fstab:/proc/self/mountinfo ...
```
which however only considers the list up to the first non-empty
file according to the man page [1].

To allow for easy configuration of scheduled fstrims also on
filesystems backing datastores in PBS, implement a scheduled job
with per-datastore schedule configuration. Enable and default to
executing the fstrim job for new datastores (except crated via ZFS
dialog).

Open question remaining:
How to best handle datastores located on ZFS? Should the command default
to zpool trim? Should it set `autotrim=on` on datastore creation instead
and silently ignore as it is now?

[0] https://forum.proxmox.com/threads/181764/
[1] https://www.man7.org/linux/man-pages/man8/fstrim.8.html


proxmox:

Christian Ebner (1):
  pbs-api-types: define fstrim schedule on datastore config

 pbs-api-types/src/datastore.rs | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)


proxmox-backup:

Christian Ebner (5):
  tools: add helper to run fstrim command on path or mountpoint
  api: config: expose fstrim schedule for datastores
  bin: proxy: periodically schedule fstrim on datastore's filesystems
  ui: expose per-datastore fstrim job schedule
  api: set default fstrim schedule on datastore create

 src/api2/config/datastore.rs     | 10 ++++++
 src/api2/node/disks/directory.rs |  6 ++--
 src/bin/proxmox-backup-proxy.rs  | 59 ++++++++++++++++++++++++++++++++
 src/tools/disks/mod.rs           | 10 ++++++
 www/Makefile                     |  1 +
 www/datastore/OptionView.js      |  7 ++++
 www/window/FstrimJobEdit.js      | 27 +++++++++++++++
 7 files changed, 118 insertions(+), 2 deletions(-)
 create mode 100644 www/window/FstrimJobEdit.js


Summary over all repositories:
  8 files changed, 134 insertions(+), 2 deletions(-)

-- 
Generated by murpp 0.9.0

[PATCH proxmox 1/1] pbs-api-types: define fstrim schedule on datastore config

[Christian Ebner]

Allows to configure whether and how often to execute a fstrim on
the filesystem backing the datastore. Defaults to not run such
jobs if no schedule is given.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-api-types/src/datastore.rs | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
index 3e87eaaa..7b5c7027 100644
--- a/pbs-api-types/src/datastore.rs
+++ b/pbs-api-types/src/datastore.rs
@@ -419,6 +419,14 @@ impl FromStr for DatastoreBackendConfig {
     }
 }
 
+pub const FSTRIM_SCHEDULE_SCHEMA: Schema =
+    StringSchema::new("Run fstrim job at specified schedule.")
+        .format(&ApiStringFormat::VerifyFn(
+            proxmox_time::verify_calendar_event,
+        ))
+        .type_text("<calendar-event>")
+        .schema();
+
 #[api(
     properties: {
         name: {
@@ -447,6 +455,10 @@ impl FromStr for DatastoreBackendConfig {
             optional: true,
             schema: PRUNE_SCHEDULE_SCHEMA,
         },
+        "fstrim-schedule": {
+            optional: true,
+            schema: FSTRIM_SCHEDULE_SCHEMA,
+        },
         keep: {
             type: crate::KeepOptions,
         },
@@ -495,6 +507,9 @@ pub struct DataStoreConfig {
     #[serde(skip_serializing_if = "Option::is_none")]
     pub prune_schedule: Option<String>,
 
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub fstrim_schedule: Option<String>,
+
     #[serde(flatten)]
     pub keep: crate::KeepOptions,
 
@@ -561,6 +576,7 @@ impl DataStoreConfig {
             comment: None,
             gc_schedule: None,
             prune_schedule: None,
+            fstrim_schedule: None,
             keep: Default::default(),
             verify_new: None,
             notify_user: None,
-- 
2.47.3

[PATCH proxmox-backup 1/5] tools: add helper to run fstrim command on path or mountpoint

[Christian Ebner]

Runs the fstrim on the filesystem mounted at given mountpoint or path
on the filesystem, allowing to discard unused blocks on the backing
disk. By invoking with `verbose` flag enabled, statistics are
gathered in the output to show on result. Setting `quiet-unsupported`
allows to run the command without error also on filesystems without
support [0].

With the intention to run this as scheduled job on a datastore's
backing filesystem.

[0] https://www.man7.org/linux/man-pages/man8/fstrim.8.html

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/tools/disks/mod.rs | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/tools/disks/mod.rs b/src/tools/disks/mod.rs
index a86cbdf79..a5cb091a7 100644
--- a/src/tools/disks/mod.rs
+++ b/src/tools/disks/mod.rs
@@ -1392,3 +1392,13 @@ pub fn unmount_by_mountpoint(path: &Path) -> Result<(), Error> {
     proxmox_sys::command::run_command(command, None)?;
     Ok(())
 }
+
+/// Run fstrim on mounted filesystem backing given path.
+pub fn fstrim(path: &Path) -> Result<String, Error> {
+    let mut command = std::process::Command::new("fstrim");
+    command.arg(path);
+    command.arg("--verbose");
+    command.arg("--quiet-unsupported");
+
+    proxmox_sys::command::run_command(command, Some(|exit_code| exit_code == 0))
+}
-- 
2.47.3

[PATCH proxmox-backup 2/5] api: config: expose fstrim schedule for datastores

[Christian Ebner]

Allows to create/edit/delete schedules to run fstrim on datastore's
backing filesystems.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/api2/config/datastore.rs | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/src/api2/config/datastore.rs b/src/api2/config/datastore.rs
index f845fe2d0..b10b7fb0a 100644
--- a/src/api2/config/datastore.rs
+++ b/src/api2/config/datastore.rs
@@ -412,6 +412,8 @@ pub enum DeletableProperty {
     GcSchedule,
     /// Delete the prune job schedule.
     PruneSchedule,
+    /// Delete the fstrim job schedule.
+    FstrimSchedule,
     /// Delete the keep-last property
     KeepLast,
     /// Delete the keep-hourly property
@@ -498,6 +500,9 @@ pub fn update_datastore(
                 DeletableProperty::PruneSchedule => {
                     data.prune_schedule = None;
                 }
+                DeletableProperty::FstrimSchedule => {
+                    data.fstrim_schedule = None;
+                }
                 DeletableProperty::KeepLast => {
                     data.keep.keep_last = None;
                 }
@@ -553,6 +558,10 @@ pub fn update_datastore(
         data.gc_schedule = update.gc_schedule;
     }
 
+    if update.fstrim_schedule.is_some() {
+        data.fstrim_schedule = update.fstrim_schedule;
+    }
+
     macro_rules! prune_disabled {
         ($(($param:literal, $($member:tt)+)),+) => {
             $(
-- 
2.47.3

[PATCH proxmox-backup 3/5] bin: proxy: periodically schedule fstrim on datastore's filesystems

[Christian Ebner]

Run the fstrim command as scheduled job on datastores having a
schedule defined in the config.

This is done since by default the systemd service to execute fstrim
invokes the command with:

`/sbin/fstrim --listed-in /etc/fstab:/proc/self/mountinfo ...`

which however does not cover datastores created on top of a disk,
since these are mounted via systemd mount units. fstrim however only
evaluates the given list above up to the first non-empty file as
stated in the man page [0].

[0] https://www.man7.org/linux/man-pages/man8/fstrim.8.html

Fixes: https://forum.proxmox.com/threads/181764/
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/bin/proxmox-backup-proxy.rs | 59 +++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index c1fe3ac15..1d375e69d 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -508,6 +508,7 @@ async fn schedule_tasks() -> Result<(), Error> {
     schedule_datastore_verify_jobs().await;
     schedule_tape_backup_jobs().await;
     schedule_task_log_rotate().await;
+    schedule_fstrim().await;
 
     Ok(())
 }
@@ -879,6 +880,64 @@ async fn schedule_task_log_rotate() {
     }
 }
 
+async fn schedule_fstrim() {
+    let config = match pbs_config::datastore::config() {
+        Err(err) => {
+            eprintln!("unable to read datastore config - {err}");
+            return;
+        }
+        Ok((config, _digest)) => config,
+    };
+
+    for (store, (_, store_config)) in config.sections {
+        let store_config: DataStoreConfig = match serde_json::from_value(store_config) {
+            Ok(c) => c,
+            Err(err) => {
+                eprintln!("datastore config from_value failed - {err}");
+                continue;
+            }
+        };
+
+        let event_schedule = match &store_config.fstrim_schedule {
+            Some(event_schedule) => event_schedule,
+            None => continue,
+        };
+
+        let worker_type = "fstrim";
+        if check_schedule(worker_type, event_schedule, &store) {
+            let mut job = match Job::new(worker_type, &store) {
+                Ok(job) => job,
+                Err(_) => continue, // could not get lock
+            };
+
+            if let Err(err) = WorkerTask::new_thread(
+                worker_type,
+                None,
+                Authid::root_auth_id().to_string(),
+                false,
+                move |worker| {
+                    job.start(&worker.upid().to_string())?;
+                    info!("executing fstrim on filesystem for {store}");
+
+                    let path = store_config.absolute_path();
+                    let result = proxmox_backup::tools::disks::fstrim(Path::new(&path))
+                        .map(|output| log::info!("{output}"));
+
+                    let status = worker.create_state(&result);
+
+                    if let Err(err) = job.finish(status) {
+                        eprintln!("could not finish job state for {worker_type}: {err}");
+                    }
+
+                    result
+                },
+            ) {
+                eprintln!("unable to start fstrim task: {err}");
+            }
+        }
+    }
+}
+
 async fn command_reopen_access_logfiles() -> Result<(), Error> {
     // only care about the most recent daemon instance for each, proxy & api, as other older ones
     // should not respond to new requests anyway, but only finish their current one and then exit.
-- 
2.47.3

[PATCH proxmox-backup 4/5] ui: expose per-datastore fstrim job schedule

[Christian Ebner]

Allows to configure the fstrim job schedule in the datastore options.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 www/Makefile                |  1 +
 www/datastore/OptionView.js |  7 +++++++
 www/window/FstrimJobEdit.js | 27 +++++++++++++++++++++++++++
 3 files changed, 35 insertions(+)
 create mode 100644 www/window/FstrimJobEdit.js

diff --git a/www/Makefile b/www/Makefile
index 9ebf0445f..32ae479ea 100644
--- a/www/Makefile
+++ b/www/Makefile
@@ -87,6 +87,7 @@ JSSRC=							\
 	window/SyncJobEdit.js				\
 	window/PruneJobEdit.js				\
 	window/GCJobEdit.js				\
+	window/FstrimJobEdit.js				\
 	window/UserEdit.js				\
 	window/S3ClientEdit.js				\
 	window/Settings.js				\
diff --git a/www/datastore/OptionView.js b/www/datastore/OptionView.js
index 7854e64d7..7eb498ee8 100644
--- a/www/datastore/OptionView.js
+++ b/www/datastore/OptionView.js
@@ -334,5 +334,12 @@ Ext.define('PBS.Datastore.Options', {
                 },
             },
         },
+        'fstrim-schedule': {
+            required: true,
+            header: gettext('fstrim Schedule'),
+            editor: {
+                xtype: 'pbsFstrimJobEdit',
+            },
+        },
     },
 });
diff --git a/www/window/FstrimJobEdit.js b/www/window/FstrimJobEdit.js
new file mode 100644
index 000000000..daf355eb2
--- /dev/null
+++ b/www/window/FstrimJobEdit.js
@@ -0,0 +1,27 @@
+Ext.define('PBS.window.FstrimJobEdit', {
+    extend: 'Proxmox.window.Edit',
+    alias: 'widget.pbsFstrimJobEdit',
+    mixins: ['Proxmox.Mixin.CBind'],
+
+    userid: undefined,
+    isAdd: false,
+
+    subject: gettext('fstrim Schedule'),
+
+    cbindData: function (initial) {
+        let me = this;
+
+        me.datastore = encodeURIComponent(me.datastore);
+        me.url = `/api2/extjs/config/datastore/${me.datastore}`;
+        me.method = 'PUT';
+        me.autoLoad = true;
+        return {};
+    },
+
+    items: {
+        xtype: 'pbsCalendarEvent',
+        name: 'fstrim-schedule',
+        fieldLabel: gettext('fstrim Schedule'),
+        emptyText: gettext('none (disabled)'),
+    },
+});
-- 
2.47.3

[PATCH proxmox-backup 5/5] api: set default fstrim schedule on datastore create

[Christian Ebner]

Default to set a weekly fstrim schedule for newly created datastores.

Do not set when crating a datastore via the ZFS dialog, as it does
not support discard.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 src/api2/config/datastore.rs     | 1 +
 src/api2/node/disks/directory.rs | 6 ++++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/src/api2/config/datastore.rs b/src/api2/config/datastore.rs
index b10b7fb0a..45df53c72 100644
--- a/src/api2/config/datastore.rs
+++ b/src/api2/config/datastore.rs
@@ -336,6 +336,7 @@ pub fn create_datastore(
     // clearing prune settings in the datastore config, as they are now handled by prune jobs
     let config = DataStoreConfig {
         prune_schedule: None,
+        fstrim_schedule: config.fstrim_schedule.or(Some("weekly".to_string())),
         keep: KeepOptions::default(),
         ..config
     };
diff --git a/src/api2/node/disks/directory.rs b/src/api2/node/disks/directory.rs
index c37d65b8d..42cd1d2d7 100644
--- a/src/api2/node/disks/directory.rs
+++ b/src/api2/node/disks/directory.rs
@@ -242,10 +242,12 @@ pub fn create_datastore_disk(
                 let lock = pbs_config::datastore::lock_config()?;
                 let datastore: DataStoreConfig = if removable_datastore {
                     serde_json::from_value(
-                        json!({ "name": name, "path": name, "backing-device": uuid }),
+                        json!({ "name": name, "path": name, "backing-device": uuid , "fstrim-schedule": "weekly" }),
                     )?
                 } else {
-                    serde_json::from_value(json!({ "name": name, "path": mount_point }))?
+                    serde_json::from_value(
+                        json!({ "name": name, "path": mount_point, "fstrim-schedule": "weekly" }),
+                    )?
                 };
                 let (config, _digest) = pbs_config::datastore::config()?;
 
-- 
2.47.3

Re: [PATCH proxmox-backup 3/5] bin: proxy: periodically schedule fstrim on datastore's filesystems

[Christian Ebner]

On 3/19/26 3:36 PM, Christian Ebner wrote:
> Run the fstrim command as scheduled job on datastores having a
> schedule defined in the config.
> 
> This is done since by default the systemd service to execute fstrim
> invokes the command with:
> 
> `/sbin/fstrim --listed-in /etc/fstab:/proc/self/mountinfo ...`
> 
> which however does not cover datastores created on top of a disk,
> since these are mounted via systemd mount units. fstrim however only
> evaluates the given list above up to the first non-empty file as
> stated in the man page [0].
> 
> [0] https://www.man7.org/linux/man-pages/man8/fstrim.8.html
> 
> Fixes: https://forum.proxmox.com/threads/181764/
> Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
> ---
>   src/bin/proxmox-backup-proxy.rs | 59 +++++++++++++++++++++++++++++++++
>   1 file changed, 59 insertions(+)
> 
> diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
> index c1fe3ac15..1d375e69d 100644
> --- a/src/bin/proxmox-backup-proxy.rs
> +++ b/src/bin/proxmox-backup-proxy.rs
> @@ -508,6 +508,7 @@ async fn schedule_tasks() -> Result<(), Error> {
>       schedule_datastore_verify_jobs().await;
>       schedule_tape_backup_jobs().await;
>       schedule_task_log_rotate().await;
> +    schedule_fstrim().await;
>   
>       Ok(())
>   }
> @@ -879,6 +880,64 @@ async fn schedule_task_log_rotate() {
>       }
>   }
>   
> +async fn schedule_fstrim() {
> +    let config = match pbs_config::datastore::config() {
> +        Err(err) => {
> +            eprintln!("unable to read datastore config - {err}");
> +            return;
> +        }
> +        Ok((config, _digest)) => config,
> +    };
> +
> +    for (store, (_, store_config)) in config.sections {
> +        let store_config: DataStoreConfig = match serde_json::from_value(store_config) {
> +            Ok(c) => c,
> +            Err(err) => {
> +                eprintln!("datastore config from_value failed - {err}");
> +                continue;
> +            }
> +        };
> +
> +        let event_schedule = match &store_config.fstrim_schedule {
> +            Some(event_schedule) => event_schedule,
> +            None => continue,
> +        };
> +
> +        let worker_type = "fstrim";
> +        if check_schedule(worker_type, event_schedule, &store) {
> +            let mut job = match Job::new(worker_type, &store) {
> +                Ok(job) => job,
> +                Err(_) => continue, // could not get lock
> +            };
> +
> +            if let Err(err) = WorkerTask::new_thread(
> +                worker_type,
> +                None,
> +                Authid::root_auth_id().to_string(),
> +                false,
> +                move |worker| {
> +                    job.start(&worker.upid().to_string())?;
> +                    info!("executing fstrim on filesystem for {store}");
> +
> +                    let path = store_config.absolute_path();
> +                    let result = proxmox_backup::tools::disks::fstrim(Path::new(&path))
> +                        .map(|output| log::info!("{output}"));

Only realized now that this does not work since unprivileged, but this 
needs to be run as root. So this needs additional rework...

Other open questions still remain though.

> +
> +                    let status = worker.create_state(&result);
> +
> +                    if let Err(err) = job.finish(status) {
> +                        eprintln!("could not finish job state for {worker_type}: {err}");
> +                    }
> +
> +                    result
> +                },
> +            ) {
> +                eprintln!("unable to start fstrim task: {err}");
> +            }
> +        }
> +    }
> +}
> +
>   async fn command_reopen_access_logfiles() -> Result<(), Error> {
>       // only care about the most recent daemon instance for each, proxy & api, as other older ones
>       // should not respond to new requests anyway, but only finish their current one and then exit.