[pve-devel] [RFC pve-kernel] add disable_write_zeroes quirk for Crucial T700 and T705

[pve-devel] [RFC pve-kernel] add disable_write_zeroes quirk for Crucial T700 and T705

[Stoiko Ivanov]

The patch seems to indeed improve the performance of BLKZEROOUT and
other operations that use write_zeroes for both drives.

Tested with LVM-thick LVs on both drives, as LVM seems to pass-through the
write_zeroes_max_bytes property (going by the output of:
/sys/block/nvmeXn1/queue/write_zeroes_max_bytes).

however as the T705 drives in our pool are all in use the test there was
limited to a rather small LV of 350M (still the change was from 5
seconds to 0.5 seconds).

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
sending as RFC, as I'm not certain if that approach makes sense in
general, or if maybe other drives are also affected (or if some other
sizes of T705, T700 are not affected).

I checked quite a few NVMEs and SATA SSDs we have in our pool - and
most consumer/prosumer/enterprise drives report a
write_zeroes_max_bytes of 0, which AFAICT means that they don't support
the write_zeroes command at all. This includes a Crucial T500, Crucial P5
several Samsung Pro, Evo drives.  Some drives report that they support
write zeroes - e.g.  Samsung Evo Plus 500, Kioxia KM7
All of the above have in common that they were much faster done with
the blkdiscard --zerout --length 16G command than the T700/T705 without
this patch.

 ...-write-zeroes-for-Crucial-T700-and-T.patch | 51 +++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100644 patches/kernel/0016-nvme-pci-disable-write-zeroes-for-Crucial-T700-and-T.patch

diff --git a/patches/kernel/0016-nvme-pci-disable-write-zeroes-for-Crucial-T700-and-T.patch b/patches/kernel/0016-nvme-pci-disable-write-zeroes-for-Crucial-T700-and-T.patch
new file mode 100644
index 000000000000..b9e2cfabf66c
--- /dev/null
+++ b/patches/kernel/0016-nvme-pci-disable-write-zeroes-for-Crucial-T700-and-T.patch
@@ -0,0 +1,51 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Stoiko Ivanov <s.ivanov@proxmox.com>
+Date: Wed, 19 Mar 2025 10:28:12 +0100
+Subject: [PATCH] nvme-pci: disable write zeroes for Crucial T700 and T705
+
+Both drives are really slow when using ioctl(fd, BLKZEROOUT,...)
+easily reproduced with the command:
+`blkdiscard --zeroout --step 1M --length 16G /dev/nvmeXn1` [0]
+
+With this patch applied the above speeds up significantly (10x)
+
+The intended change can be observed with bpftrace:
+bpftrace -e 'kprobe:blkdev_issue_zeroout { printf("%s %s", comm, kstack()); } '
+bpftrace -e 'kprobe:__blkdev_issue_write_zeroes { printf("%s %s", comm, kstack()); }'
+bpftrace -e 'kprobe:__blkdev_issue_zero_pages { printf("%s %s", comm, kstack()); }'
+
+w/o the patch the first 2 kprobes produce output, w/ the patch applied
+the kernel falls back an uses __blkdev_issue_zero_pages, as it does
+for devices that don't support write zeroes, so probe 1 and 3 produce
+output.
+
+The patch is based on commit:
+d14c273132ae (nvme-pci: disable Write Zeroes on Phison E3C/E4C)
+https://github.com/torvalds/linux/commit/d14c273132aec81a1a8107c9ab4865b89e7910a7
+The discussion and rationale seems sensible in this case as well:
+https://lore.kernel.org/lkml/20220922142626.GB28397@lst.de/T/#m52275e0dc8081a9b779bd201b2bdc51ce7cd045b
+
+[0] The above command emulates the wiping done by
+`lvmconvert --type thinpool` for the metadata volume.
+
+Reported-by: Dietmar Maurer <dietmar@proxmox.com>
+Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
+---
+ drivers/nvme/host/pci.c | 4 ++++
+ 1 file changed, 4 insertions(+)
+
+diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
+index 76b3f7b396c86b6672b8bfe9e03bfdca51c6b726..da7a6e5f2d7d1b3fc48d1fcff4e8264dbef88445 100644
+--- a/drivers/nvme/host/pci.c
++++ b/drivers/nvme/host/pci.c
+@@ -3560,6 +3560,10 @@ static const struct pci_device_id nvme_id_table[] = {
+ 		.driver_data = NVME_QUIRK_NO_DEEPEST_PS, },
+ 	{ PCI_DEVICE(0xc0a9, 0x540a),   /* Crucial P2 */
+ 		.driver_data = NVME_QUIRK_BOGUS_NID, },
++	{ PCI_DEVICE(0xc0a9, 0x5419),   /* Crucial T700 */
++		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
++	{ PCI_DEVICE(0xc0a9, 0x542b),   /* Crucial T705 */
++		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
+ 	{ PCI_DEVICE(0x1d97, 0x2263), /* Lexar NM610 */
+ 		.driver_data = NVME_QUIRK_BOGUS_NID, },
+ 	{ PCI_DEVICE(0x1d97, 0x1d97), /* Lexar NM620 */
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC pve-kernel] add disable_write_zeroes quirk for Crucial T700 and T705

[Thomas Lamprecht]

Am 19.03.25 um 21:33 schrieb Stoiko Ivanov:
> The patch seems to indeed improve the performance of BLKZEROOUT and
> other operations that use write_zeroes for both drives.
> 
> Tested with LVM-thick LVs on both drives, as LVM seems to pass-through the
> write_zeroes_max_bytes property (going by the output of:
> /sys/block/nvmeXn1/queue/write_zeroes_max_bytes).
> 
> however as the T705 drives in our pool are all in use the test there was
> limited to a rather small LV of 350M (still the change was from 5
> seconds to 0.5 seconds).
> 
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
> sending as RFC, as I'm not certain if that approach makes sense in
> general, or if maybe other drives are also affected (or if some other
> sizes of T705, T700 are not affected).

not sure how much anybody here can comment on that, maybe send it upstream
and CC a relevant maintainer, ideally one working for crucial/micron.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel