[pve-devel] [PATCH docs] qm: guest trim: add note mentioning issue with ext4

[pve-devel] [PATCH docs] qm: guest trim: add note mentioning issue with ext4

[Fiona Ebner]

It is rather unexpected and seems worth mentioning. Reported in the
community forum [0] and the explanation found by Alwin [1].

[0]: https://forum.proxmox.com/threads/123819/
[1]: https://serverfault.com/questions/1113127/fstrim-is-very-slow-on-xfs-and-always-return-same-value-unlike-ext4/1113129#1113129

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 qm.adoc | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/qm.adoc b/qm.adoc
index bd535a2..3ba2a3c 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -1063,6 +1063,12 @@ operations that have the potential to write out zeros to the storage:
 
 On a thin provisioned storage, this can help to free up unused space.
 
+NOTE: There is a caveat with ext4 on Linux, because it uses an in-memory
+optimization to avoid issuing duplicate TRIM requests. Since the guest doesn't
+know about the change in the underlying storage, only the first guest-trim will
+run as expected. Subsequent ones, until the next reboot, will only consider
+parts of the filesystem that changed since then.
+
 Troubleshooting
 ^^^^^^^^^^^^^^^
 
-- 
2.30.2

Re: [pve-devel] [PATCH docs] qm: guest trim: add note mentioning issue with ext4

[Laurent GUERBY]

On Fri, 2023-03-10 at 10:06 +0100, Fiona Ebner wrote:
> It is rather unexpected and seems worth mentioning. Reported in the
> community forum [0] and the explanation found by Alwin [1].
> 
> [0]: https://forum.proxmox.com/threads/123819/
> [1]: https://serverfault.com/questions/1113127/fstrim-is-very-slow-
> on-xfs-and-always-return-same-value-unlike-ext4/1113129#1113129

Hi,

Here a workaround proposed by kernel developper when I reported the
issue:

https://lore.kernel.org/all/1634984680.26818.10.camel@guerby.net/
"How to force EXT4_MB_GRP_CLEAR_TRIMMED on a live ext4?"

"My use case is having live migrated a virtual machine root disk from
one storage to another, the target supporting trim, but since fstrim in
the VM post migration does mostly nothing (assumes most space was
trimmed) I cannot release space to the new storage."
(...)
"
Meanwhile other than umount/mount, or actually writing to the dummy
files,
you can try to use fallocate to allocate all the remaining space in the
file system and subsequently removing it. That should be more
efficient,
but don't forget to sync after remove to make sure the space is
released
before you call fstrim."


I haven't followed up to see if code was added to deal with this
(tune2fs?) without using fallocate.

Sincerely,

Laurent

> 
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
>  qm.adoc | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/qm.adoc b/qm.adoc
> index bd535a2..3ba2a3c 100644
> --- a/qm.adoc
> +++ b/qm.adoc
> @@ -1063,6 +1063,12 @@ operations that have the potential to write
> out zeros to the storage:
>  
>  On a thin provisioned storage, this can help to free up unused
> space.
>  
> +NOTE: There is a caveat with ext4 on Linux, because it uses an in-
> memory
> +optimization to avoid issuing duplicate TRIM requests. Since the
> guest doesn't
> +know about the change in the underlying storage, only the first
> guest-trim will
> +run as expected. Subsequent ones, until the next reboot, will only
> consider
> +parts of the filesystem that changed since then.
> +
>  Troubleshooting
>  ^^^^^^^^^^^^^^^
>  

Re: [pve-devel] [PATCH docs] qm: guest trim: add note mentioning issue with ext4

[Fiona Ebner]

Am 10.03.23 um 10:43 schrieb Laurent GUERBY:
> On Fri, 2023-03-10 at 10:06 +0100, Fiona Ebner wrote:
>> It is rather unexpected and seems worth mentioning. Reported in the
>> community forum [0] and the explanation found by Alwin [1].
>>
>> [0]: https://forum.proxmox.com/threads/123819/
>> [1]: https://serverfault.com/questions/1113127/fstrim-is-very-slow-
>> on-xfs-and-always-return-same-value-unlike-ext4/1113129#1113129
> 
> Hi,
> 
> Here a workaround proposed by kernel developper when I reported the
> issue:
> 
> https://lore.kernel.org/all/1634984680.26818.10.camel@guerby.net/
> "How to force EXT4_MB_GRP_CLEAR_TRIMMED on a live ext4?"
> 
> "My use case is having live migrated a virtual machine root disk from
> one storage to another, the target supporting trim, but since fstrim in
> the VM post migration does mostly nothing (assumes most space was
> trimmed) I cannot release space to the new storage."
> (...)
> "
> Meanwhile other than umount/mount, or actually writing to the dummy
> files,
> you can try to use fallocate to allocate all the remaining space in the
> file system and subsequently removing it. That should be more
> efficient,
> but don't forget to sync after remove to make sure the space is
> released
> before you call fstrim."
> 

Hi,
while we could mention that workaround, I'm not sure it's not too hacky
and filling up (almost) all space is not something I'd like to do in
production even if it's just very briefly ;)

> 
> I haven't followed up to see if code was added to deal with this
> (tune2fs?) without using fallocate.
> 

Remounting my root partition (even with the 'discard' mount option) in a
Debian 11 VM didn't help at least and I don't see anything other than
the 'discard' mount option in 'man tune2fs' either (searched for 'trim'
and 'discard').

Best Regards,
Fiona

[pve-devel] applied: [PATCH docs] qm: guest trim: add note mentioning issue with ext4

[Thomas Lamprecht]

Am 10/03/2023 um 10:06 schrieb Fiona Ebner:
> It is rather unexpected and seems worth mentioning. Reported in the
> community forum [0] and the explanation found by Alwin [1].
> 
> [0]: https://forum.proxmox.com/threads/123819/
> [1]: https://serverfault.com/questions/1113127/fstrim-is-very-slow-on-xfs-and-always-return-same-value-unlike-ext4/1113129#1113129
> 
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
>  qm.adoc | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
>

applied, thanks!