* [PATCH docs/manager/storage 0/4] fix #8196: lvm: add a throughput limit for discards
@ 2026-11-20 13:32 Lukas Sichert
2026-11-20 13:32 ` [PATCH storage 1/4] lvm: rename the saferemove throughput parser to parse_throughput Lukas Sichert
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Lukas Sichert @ 2026-11-20 13:32 UTC (permalink / raw)
To: pve-devel; +Cc: Lukas Sichert
Unthrottled discards can overload the controllers of enterprise storage
arrays. The cleanup worker discards removed volumes in bounded batches
that share the loop pacing the saferemove writes, so limiting the rate
there throttles discards too.
This series adds a 'discard_throughput' option to 'on-volume-remove'. It
takes the same values as 'saferemove_throughput'. If both are set while
wiping, the lower limit is used, since both go through the same pacing
loop. Not setting the option, or setting it to zero, means no limit from
this option.
The option only has an effect if 'discard' is enabled. It can be set
through the API or CLI. The storage edit dialog has no field for it, but
saving the dialog no longer drops it.
Link: https://bugzilla.proxmox.com/show_bug.cgi?id=8196
storage:
Lukas Sichert (2):
lvm: rename the saferemove throughput parser to parse_throughput
fix #8196: lvm: add discard throughput limit
src/PVE/Storage/LVMPlugin.pm | 30 +++++++++++++++++-----
src/test/run_test_lvm_throughput.pl | 40 +++++++++++++++++++++++++++++
2 files changed, 64 insertions(+), 6 deletions(-)
manager:
Lukas Sichert (1):
ui: lvm: keep other on-volume-remove options when editing
www/manager6/storage/LVMEdit.js | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
docs:
Lukas Sichert (1):
storage: lvm: document the discard throughput limit
pve-storage-lvm.adoc | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
Summary over all repositories:
4 files changed, 74 insertions(+), 8 deletions(-)
--
Generated by murpp 0.12.0
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH storage 1/4] lvm: rename the saferemove throughput parser to parse_throughput 2026-11-20 13:32 [PATCH docs/manager/storage 0/4] fix #8196: lvm: add a throughput limit for discards Lukas Sichert @ 2026-11-20 13:32 ` Lukas Sichert 2026-11-20 13:32 ` [PATCH storage 2/4] fix #8196: lvm: add discard throughput limit Lukas Sichert ` (2 subsequent siblings) 3 siblings, 0 replies; 5+ messages in thread From: Lukas Sichert @ 2026-11-20 13:32 UTC (permalink / raw) To: pve-devel; +Cc: Lukas Sichert The parser only handles the value syntax of a throughput limit, so it is not specific to saferemove. Signed-off-by: Lukas Sichert <l.sichert@proxmox.com> --- src/PVE/Storage/LVMPlugin.pm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm index c8384d7..3f74cfc 100644 --- a/src/PVE/Storage/LVMPlugin.pm +++ b/src/PVE/Storage/LVMPlugin.pm @@ -684,11 +684,11 @@ my sub blockdev_ioctl_range { # The limit is in bytes per second, with an optional k, m or g suffix for powers of 1024, as taken by # cstream's -t option, which existing configurations were written for. -my sub parse_saferemove_throughput { +my sub parse_throughput { my ($value) = @_; my ($number, $unit) = $value =~ m/^(-?\d+)([kmg])?$/i - or die "invalid saferemove throughput '$value'\n"; + or die "invalid throughput '$value'\n"; my $multiplier = { k => 1024, m => 1024**2, g => 1024**3 }->{ lc($unit // '') } // 1; return $number * $multiplier; @@ -762,7 +762,7 @@ my sub free_lvm_volumes_locked { my $throughput = undef; # discards alone are not rate limited if ($zero_out && (my $value = $scfg->{saferemove_throughput})) { # Normalize zero spellings before selecting the default or dividing by the rate. - $throughput = abs(parse_saferemove_throughput($value)) || undef; + $throughput = abs(parse_throughput($value)) || undef; } if (defined($throughput)) { my $rendered_throughput = render_bytes($throughput); -- 2.47.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH storage 2/4] fix #8196: lvm: add discard throughput limit 2026-11-20 13:32 [PATCH docs/manager/storage 0/4] fix #8196: lvm: add a throughput limit for discards Lukas Sichert 2026-11-20 13:32 ` [PATCH storage 1/4] lvm: rename the saferemove throughput parser to parse_throughput Lukas Sichert @ 2026-11-20 13:32 ` Lukas Sichert 2026-11-20 13:32 ` [PATCH manager 3/4] ui: lvm: keep other on-volume-remove options when editing Lukas Sichert 2026-11-20 13:32 ` [PATCH docs 4/4] storage: lvm: document the discard throughput limit Lukas Sichert 3 siblings, 0 replies; 5+ messages in thread From: Lukas Sichert @ 2026-11-20 13:32 UTC (permalink / raw) To: pve-devel; +Cc: Lukas Sichert Unthrottled discards can overload the controllers of enterprise storage arrays. The cleanup worker already issues discards in bounded batches through the loop that paces the saferemove writes, so applying a rate limit there throttles discards as well. Add a 'discard_throughput' option to on-volume-remove. It takes the same values as saferemove_throughput and only has an effect if discard is enabled. If wiping is enabled and saferemove_throughput is set as well, the lower of both limits is used, as both share the one pacing loop. The task log line announcing the limit no longer says "saferemove", as the limit can now come from either option. Link: https://bugzilla.proxmox.com/show_bug.cgi?id=8196 Signed-off-by: Lukas Sichert <l.sichert@proxmox.com> --- src/PVE/Storage/LVMPlugin.pm | 24 ++++++++++++++--- src/test/run_test_lvm_throughput.pl | 40 +++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 3 deletions(-) diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm index 3f74cfc..ca274b8 100644 --- a/src/PVE/Storage/LVMPlugin.pm +++ b/src/PVE/Storage/LVMPlugin.pm @@ -759,14 +759,21 @@ my sub free_lvm_volumes_locked { my $zero_out = $scfg->{saferemove}; my $zeroout_variant = $zero_out ? 'blkzeroout' : 'none'; - my $throughput = undef; # discards alone are not rate limited + my $throughput = undef; if ($zero_out && (my $value = $scfg->{saferemove_throughput})) { # Normalize zero spellings before selecting the default or dividing by the rate. $throughput = abs(parse_throughput($value)) || undef; } + if ($on_remove_opts->{discard} && (my $value = $on_remove_opts->{discard_throughput})) { + # Wiping and discarding share one pacing loop, so the lower of both limits applies. + my $limit = abs(parse_throughput($value)) || undef; + if (defined($limit) && (!defined($throughput) || $limit < $throughput)) { + $throughput = $limit; + } + } if (defined($throughput)) { my $rendered_throughput = render_bytes($throughput); - print "using saferemove throughput limit: $rendered_throughput/s\n"; + print "using throughput limit: $rendered_throughput/s\n"; } # If the storage does not support write_zeroes fall back to writing zeroes manually using @@ -1068,6 +1075,15 @@ my $on_volume_remove_format = { . " range before removing it, allowing thin-provisioned storage to reclaim previously" . " allocated physical space, provided the storage supports discard.", }, + discard_throughput => { + description => "Discard throughput in bytes per second (optional suffix: k, m, or g)." + . " Only has an effect if 'discard' is enabled. If 'saferemove_throughput' is set" + . " as well, the lower limit is used.", + type => 'string', + format_description => 'bytes per second', + pattern => '-?\d+[kmgKMG]?', + optional => 1, + }, }; sub verify_on_volume_remove { @@ -1119,7 +1135,9 @@ sub properties { }, saferemove_throughput => { title => "Wipe Throughput", - description => "Wipe throughput in bytes per second (optional suffix: k, m, or g).", + description => "Wipe throughput in bytes per second (optional suffix: k, m, or g)." + . " If 'discard' is enabled and 'discard_throughput' in 'on-volume-remove' is set" + . " as well, the lower limit is used.", type => 'string', pattern => '-?\d+[kmgKMG]?', }, diff --git a/src/test/run_test_lvm_throughput.pl b/src/test/run_test_lvm_throughput.pl index b6f6fa8..d71f05a 100755 --- a/src/test/run_test_lvm_throughput.pl +++ b/src/test/run_test_lvm_throughput.pl @@ -51,4 +51,44 @@ for my $value ('0m', '-0', '00', '1m') { ); } +my $discard_granularity = 0; +$plugin->mock( + file_read_firstline => sub { + return + $_[0] =~ m!/size$! ? 2048 + : $_[0] =~ m!/discard_granularity$! ? $discard_granularity + : 0; + }, +); +my @cases = ( + ['wipe, lower on-volume-remove wins', 1, '2m', '1m', 1, 0, 1], + ['wipe, lower saferemove_throughput wins', 1, '1m', '2m', 1, 0, 1], + ['wipe, zero on-volume-remove leaves the other', 1, '2m', '0', 0.5, 0, 1], + ['wipe, discard_throughput ignored without discard', 1, '2m', '1m', 0.5, 0, 0], + ['discard only', 0, undef, '2m', 0.5, 512, 1], + ['discard only, saferemove_throughput ignored', 0, '1m', undef, undef, 512, 1], +); +for my $case (@cases) { + my ($label, $wipe, $old, $new, $expected, $granularity, $discard) = @$case; + $discard_granularity = $granularity; + open(my $fh, '>', "$dir/del-$name") or die "create test volume: $!\n"; + close($fh) or die "close test volume: $!\n"; + @delays = (); + @commands = (); + my $output = ''; + my $scfg = { vgname => "..$dir", saferemove => $wipe }; + $scfg->{saferemove_throughput} = $old if defined($old); + my @remove_opts = ( + $discard ? 'discard=1' : (), defined($new) ? "discard_throughput=$new" : (), + ); + $scfg->{'on-volume-remove'} = join(',', @remove_opts) if @remove_opts; + open(my $stdout, '>', \$output) or die "capture output: $!\n"; + my $previous = select($stdout); + eval { PVE::Storage::LVMPlugin->free_image('test', $scfg, $name, 0, 'raw')->(); }; + select($previous); + close($stdout) or die "close captured output: $!\n"; + like($output, qr/^(?:zeroed out|discarded) /m, "$label: processed the volume"); + is_deeply(\@delays, defined($expected) ? [$expected] : [], "$label: delays"); +} + done_testing(); -- 2.47.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH manager 3/4] ui: lvm: keep other on-volume-remove options when editing 2026-11-20 13:32 [PATCH docs/manager/storage 0/4] fix #8196: lvm: add a throughput limit for discards Lukas Sichert 2026-11-20 13:32 ` [PATCH storage 1/4] lvm: rename the saferemove throughput parser to parse_throughput Lukas Sichert 2026-11-20 13:32 ` [PATCH storage 2/4] fix #8196: lvm: add discard throughput limit Lukas Sichert @ 2026-11-20 13:32 ` Lukas Sichert 2026-11-20 13:32 ` [PATCH docs 4/4] storage: lvm: document the discard throughput limit Lukas Sichert 3 siblings, 0 replies; 5+ messages in thread From: Lukas Sichert @ 2026-11-20 13:32 UTC (permalink / raw) To: pve-devel; +Cc: Lukas Sichert The edit dialog rebuilt the on-volume-remove property string from the discard checkbox alone, so saving the dialog dropped every other option in it. This would silently remove options that can only be set through the API or CLI, such as discard_throughput. Remember the options without a field of their own when loading the storage and submit them again with the checkbox state. Signed-off-by: Lukas Sichert <l.sichert@proxmox.com> --- www/manager6/storage/LVMEdit.js | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/www/manager6/storage/LVMEdit.js b/www/manager6/storage/LVMEdit.js index 9308ef4e..75861214 100644 --- a/www/manager6/storage/LVMEdit.js +++ b/www/manager6/storage/LVMEdit.js @@ -151,7 +151,8 @@ Ext.define('PVE.storage.LVMInputPanel', { onGetValues: function (values) { let me = this; - let onRemove = {}; + // options without a field of their own, like discard_throughput, must survive an edit + let onRemove = { ...me.otherOnRemoveOptions }; if (values['on-remove-discard']) { onRemove.discard = 1; } @@ -178,6 +179,8 @@ Ext.define('PVE.storage.LVMInputPanel', { if (values['on-volume-remove']) { let onRemove = PVE.Parser.parsePropertyString(values['on-volume-remove']); values['on-remove-discard'] = onRemove.discard; + delete onRemove.discard; + this.otherOnRemoveOptions = onRemove; } delete values['on-volume-remove']; -- 2.47.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH docs 4/4] storage: lvm: document the discard throughput limit 2026-11-20 13:32 [PATCH docs/manager/storage 0/4] fix #8196: lvm: add a throughput limit for discards Lukas Sichert ` (2 preceding siblings ...) 2026-11-20 13:32 ` [PATCH manager 3/4] ui: lvm: keep other on-volume-remove options when editing Lukas Sichert @ 2026-11-20 13:32 ` Lukas Sichert 3 siblings, 0 replies; 5+ messages in thread From: Lukas Sichert @ 2026-11-20 13:32 UTC (permalink / raw) To: pve-devel; +Cc: Lukas Sichert Describe the discard_throughput option of on-volume-remove and how it combines with saferemove_throughput. Unthrottled discards can overload the controllers of enterprise storage arrays. Signed-off-by: Lukas Sichert <l.sichert@proxmox.com> --- pve-storage-lvm.adoc | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/pve-storage-lvm.adoc b/pve-storage-lvm.adoc index d09206b..5d9c601 100644 --- a/pve-storage-lvm.adoc +++ b/pve-storage-lvm.adoc @@ -67,6 +67,10 @@ zeroing step size and the discard granularity. The accumulated range is then discarded before processing continues. This prevents complete discard chunks from being skipped and avoids allocating the whole LV with zeroes on thin-provisioned backing storage before the space can be reclaimed again. ++ +Set `discard_throughput` to limit the discard rate in bytes per second. It only +takes effect if `discard` is enabled. If `saferemove_throughput` is set as well, +the lower limit applies. `saferemove-stepsize`:: @@ -77,7 +81,8 @@ underlying storage. Up to 32 MiB (maximum) by default. Limits wipe throughput. If the backing storage supports the "write zeroes" operation, throughput is unlimited by default. Otherwise, manually written -zeroes are limited to 10 MiB/s by default. +zeroes are limited to 10 MiB/s by default. If `discard` is enabled and +`discard_throughput` is set as well, the lower limit applies. `snapshot-as-volume-chain`:: -- 2.47.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-20 13:33 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-20 13:32 [PATCH docs/manager/storage 0/4] fix #8196: lvm: add a throughput limit for discards Lukas Sichert 2026-11-20 13:32 ` [PATCH storage 1/4] lvm: rename the saferemove throughput parser to parse_throughput Lukas Sichert 2026-11-20 13:32 ` [PATCH storage 2/4] fix #8196: lvm: add discard throughput limit Lukas Sichert 2026-11-20 13:32 ` [PATCH manager 3/4] ui: lvm: keep other on-volume-remove options when editing Lukas Sichert 2026-11-20 13:32 ` [PATCH docs 4/4] storage: lvm: document the discard throughput limit Lukas Sichert
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.