* [pve-devel] [PATCH installer 1/3] run env: always return proper value for default zfs max arc size
2025-04-04 12:46 [pve-devel] [PATCH installer 0/3] fix #6285: always set up zfs modprobe configuration Christoph Heiss
@ 2025-04-04 12:46 ` Christoph Heiss
2025-04-04 12:46 ` [pve-devel] [PATCH installer 2/3] tui: bootdisk: " Christoph Heiss
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Christoph Heiss @ 2025-04-04 12:46 UTC (permalink / raw)
To: pve-devel
In preparation for fixing #6285 [0].
`0` means to just skip writing the module parameter. But (especially)
with the upcoming change in ZFS 2.3 - which makes the size basically
that of the system memory minus 1 GiB - we want to always write some
value.
[0] https://bugzilla.proxmox.com/show_bug.cgi?id=6285
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Proxmox/Install/RunEnv.pm | 9 +++++----
test/zfs-arc-max.pl | 42 +++++++++++++++++++++++++--------------
2 files changed, 32 insertions(+), 19 deletions(-)
diff --git a/Proxmox/Install/RunEnv.pm b/Proxmox/Install/RunEnv.pm
index 407d05c..02c603d 100644
--- a/Proxmox/Install/RunEnv.pm
+++ b/Proxmox/Install/RunEnv.pm
@@ -337,14 +337,15 @@ sub default_zfs_arc_max {
my $product = Proxmox::Install::ISOEnv::get('product');
my $total_memory = query_total_memory();
- # By default limit PVE and low-memory systems, for all others let ZFS decide on its own by
- # returning `0`, which causes the installer to skip writing the `zfs_arc_max` module parameter.
+ # By default limit PVE and low-memory systems, for all just use the 50% of system memory
if ($product ne 'pve') {
- return 0 if $total_memory >= 2048 && $product ne 'pmg';
- return 0 if $total_memory >= 4096; # PMG's base memory requirement is much higer
+ my $zfs_default_mib = int(sprintf('%.0f', $total_memory / 2));
+ return $zfs_default_mib if $total_memory >= 2048 && $product ne 'pmg';
+ return $zfs_default_mib if $total_memory >= 4096; # PMG's base memory requirement is much higher
}
my $default_mib = $total_memory * $ZFS_ARC_SYSMEM_PERCENTAGE;
+ # int(sprintf()) rounds up instead of truncating
my $rounded_mib = int(sprintf('%.0f', $default_mib));
if ($rounded_mib > $ZFS_ARC_MAX_SIZE_MIB) {
diff --git a/test/zfs-arc-max.pl b/test/zfs-arc-max.pl
index 8cf093f..a8c0a37 100755
--- a/test/zfs-arc-max.pl
+++ b/test/zfs-arc-max.pl
@@ -21,8 +21,10 @@ sub mock_product {
);
}
-my %default_tests = (
- 16 => 64, # at least 64 MiB
+use constant ZFS_ARC_MIN_MIB => 64;
+
+my %default_tests_pve = (
+ 16 => ZFS_ARC_MIN_MIB, # at least 64 MiB
1024 => 102,
4 * 1024 => 410,
8 * 1024 => 819,
@@ -31,26 +33,36 @@ my %default_tests = (
1024 * 1024 => 16384, # maximum of 16 GiB
);
-while (my ($total_mem, $expected) = each %default_tests) {
+my %default_tests_others = (
+ 16 => ZFS_ARC_MIN_MIB, # at least 64 MiB
+ 1024 => 102,
+ 4 * 1024 => 2048,
+ 8 * 1024 => 4096,
+ 150 * 1024 => 76800,
+ 160 * 1024 => 81920,
+ 1024 * 1024 => 524288,
+);
+
+mock_product('pve');
+while (my ($total_mem, $expected) = each %default_tests_pve) {
$proxmox_install_runenv->redefine(
query_total_memory => sub { return $total_mem; },
);
- mock_product('pve');
is(Proxmox::Install::RunEnv::default_zfs_arc_max(), $expected,
- "$expected MiB should be zfs_arc_max for PVE with $total_mem MiB system memory");
+ "zfs_arc_max should default to $expected for pve with $total_mem MiB system memory");
+}
- mock_product('pbs');
- is(Proxmox::Install::RunEnv::default_zfs_arc_max(), $total_mem < 2048 ? $expected : 0,
- "zfs_arc_max should default to `0` for PBS with $total_mem MiB system memory");
+while (my ($total_mem, $expected) = each %default_tests_others) {
+ $proxmox_install_runenv->redefine(
+ query_total_memory => sub { return $total_mem; },
+ );
- mock_product('pmg');
- is(Proxmox::Install::RunEnv::default_zfs_arc_max(), $total_mem < 4096 ? $expected : 0,
- "zfs_arc_max should default to `0` for PMG with $total_mem MiB system memory");
-
- mock_product('pdm');
- is(Proxmox::Install::RunEnv::default_zfs_arc_max(), $total_mem < 2048 ? $expected : 0,
- "zfs_arc_max should default to `0` for PDM with $total_mem MiB system memory");
+ foreach my $product ('pbs', 'pmg', 'pdm') {
+ mock_product($product);
+ is(Proxmox::Install::RunEnv::default_zfs_arc_max(), $expected,
+ "zfs_arc_max should default to $expected for $product with $total_mem MiB system memory");
+ }
}
my @clamp_tests = (
--
2.48.1
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 5+ messages in thread* [pve-devel] [PATCH installer 2/3] tui: bootdisk: always return proper value for default zfs max arc size
2025-04-04 12:46 [pve-devel] [PATCH installer 0/3] fix #6285: always set up zfs modprobe configuration Christoph Heiss
2025-04-04 12:46 ` [pve-devel] [PATCH installer 1/3] run env: always return proper value for default zfs max arc size Christoph Heiss
@ 2025-04-04 12:46 ` Christoph Heiss
2025-04-04 12:46 ` [pve-devel] [PATCH installer 3/3] fix #6285: install: always set up zfs modprobe configuration Christoph Heiss
2025-04-04 13:01 ` [pve-devel] applied-series: [PATCH installer 0/3] fix #6285: " Thomas Lamprecht
3 siblings, 0 replies; 5+ messages in thread
From: Christoph Heiss @ 2025-04-04 12:46 UTC (permalink / raw)
To: pve-devel
In preparation for fixing #6285 [0].
`0` means to just skip writing the module parameter. But (especially)
with the upcoming change in ZFS 2.3 - which makes the size basically
that of the system memory minus 1 GiB - we want to always write some
value.
[0] https://bugzilla.proxmox.com/show_bug.cgi?id=6285
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
proxmox-tui-installer/src/views/bootdisk.rs | 32 ++++++++++++---------
1 file changed, 19 insertions(+), 13 deletions(-)
diff --git a/proxmox-tui-installer/src/views/bootdisk.rs b/proxmox-tui-installer/src/views/bootdisk.rs
index b0fc131..313a3c9 100644
--- a/proxmox-tui-installer/src/views/bootdisk.rs
+++ b/proxmox-tui-installer/src/views/bootdisk.rs
@@ -626,26 +626,27 @@ impl ViewWrapper for BtrfsBootdiskOptionsView {
struct ZfsBootdiskOptionsView {
view: MultiDiskOptionsView<FormView>,
+ zfs_arc_max_default: usize,
}
impl ZfsBootdiskOptionsView {
// TODO: Re-apply previous disk selection from `options` correctly
fn new(runinfo: &RuntimeInfo, options: &ZfsBootdiskOptions) -> Self {
+ let zfs_arc_max_default = runinfo.total_memory.div_ceil(2);
+
let arc_max_view = {
// Always leave a GiB of headroom for the OS.
let view =
IntegerEditView::new_with_suffix("MiB").max_value(runinfo.total_memory - 1024);
- // If the runtime environment provides a non-zero value, that is
- // also not the built-in ZFS default of half the system memory, use
- // that as default.
- // Otherwise, just place the ZFS default into the placeholder.
- if runinfo.default_zfs_arc_max > 0
- && runinfo.default_zfs_arc_max != runinfo.total_memory / 2
- {
+ // If the runtime environment provides a value that is not the (ZFS) default of
+ // half the system memory, use that as default.
+ // Otherwise, just place the (ZFS) default of 50% of available system memory into the
+ // placeholder.
+ if runinfo.default_zfs_arc_max != zfs_arc_max_default {
view.content(options.arc_max)
} else {
- view.placeholder(runinfo.total_memory / 2)
+ view.placeholder(zfs_arc_max_default)
}
};
@@ -687,7 +688,10 @@ impl ZfsBootdiskOptionsView {
"ZFS is not compatible with hardware RAID controllers, for details see the documentation."
).center());
- Self { view }
+ Self {
+ view,
+ zfs_arc_max_default,
+ }
}
fn new_with_defaults(runinfo: &RuntimeInfo) -> Self {
@@ -706,13 +710,15 @@ impl ZfsBootdiskOptionsView {
// If a value is set, return that and clamp it to at least [`ZFS_ARC_MIN_SIZE_MIB`].
//
- // Otherwise, if no value was set or an error occurred return `0`. The former simply means
- // that the placeholder value is still there.
+ // Otherwise, if no value was set or an error occurred return the default value. The former
+ // simply means that the placeholder value is still there.
let arc_max = view
.get_child::<IntegerEditView>(4)?
.get_content_maybe()
- .map_or(Ok(0), |v| v.map(|v| v.max(ZFS_ARC_MIN_SIZE_MIB)))
- .unwrap_or(0);
+ .map_or(Ok(self.zfs_arc_max_default), |v| {
+ v.map(|v| v.max(ZFS_ARC_MIN_SIZE_MIB))
+ })
+ .unwrap_or(self.zfs_arc_max_default);
Some((
disks,
--
2.48.1
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 5+ messages in thread* [pve-devel] [PATCH installer 3/3] fix #6285: install: always set up zfs modprobe configuration
2025-04-04 12:46 [pve-devel] [PATCH installer 0/3] fix #6285: always set up zfs modprobe configuration Christoph Heiss
2025-04-04 12:46 ` [pve-devel] [PATCH installer 1/3] run env: always return proper value for default zfs max arc size Christoph Heiss
2025-04-04 12:46 ` [pve-devel] [PATCH installer 2/3] tui: bootdisk: " Christoph Heiss
@ 2025-04-04 12:46 ` Christoph Heiss
2025-04-04 13:01 ` [pve-devel] applied-series: [PATCH installer 0/3] fix #6285: " Thomas Lamprecht
3 siblings, 0 replies; 5+ messages in thread
From: Christoph Heiss @ 2025-04-04 12:46 UTC (permalink / raw)
To: pve-devel
Fixes #6285 [0].
Came up a few times now in the forum (most recently [0], german) and is
a potential source of confusion for users, if the file does not exist on
(new) installations.
It makes indeed sense to just unconditionally write to
/etc/modprobe.d/zfs.conf. Often users create a separate ZFS pool after
installation, on separate disks, where it still makes sense to have a
more sensible zfs_arc_max default, at least on PVE.
It has been also exposed for all products, with sensible defaults each,
for some time now [1][2].
[0] https://bugzilla.proxmox.com/show_bug.cgi?id=6285
[1] https://forum.proxmox.com/threads/ram-auslastung-nimmt-%C3%BCber-die-zeit-drastisch-zu.164527/#post-760989
[2] https://git.proxmox.com/?p=pve-installer.git;a=commitdiff;h=a42a9db20976fbc0abb35d53416cee926b6efafe
[3] https://git.proxmox.com/?p=pve-installer.git;a=commitdiff;h=8772ebc35a7f43b97f5433c4d328ea784eaf902c
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Proxmox/Install.pm | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/Proxmox/Install.pm b/Proxmox/Install.pm
index 57fd899..2e9b131 100644
--- a/Proxmox/Install.pm
+++ b/Proxmox/Install.pm
@@ -340,9 +340,7 @@ my sub zfs_setup_module_conf {
my $arc_max_mib = Proxmox::Install::Config::get_zfs_opt('arc_max');
my $arc_max = Proxmox::Install::RunEnv::clamp_zfs_arc_max($arc_max_mib) * 1024 * 1024;
- if ($arc_max > 0) {
- file_write_all("$targetdir/etc/modprobe.d/zfs.conf", "options zfs zfs_arc_max=$arc_max\n")
- }
+ file_write_all("$targetdir/etc/modprobe.d/zfs.conf", "options zfs zfs_arc_max=$arc_max\n");
}
sub get_btrfs_raid_setup {
@@ -1329,10 +1327,13 @@ _EOD
file_write_all("$targetdir/etc/default/grub.d/zfs.cfg", $zfs_snippet);
file_write_all("$targetdir/etc/kernel/cmdline", "root=ZFS=$zfs_pool_name/ROOT/$zfs_root_volume_name boot=zfs $target_cmdline\n");
-
- zfs_setup_module_conf($targetdir);
}
+ # Always write zfs module parameter - even if the user did not select ZFS-on-root.
+ # It still makes sense to provide a sensible default for zfs_arc_max, in case a
+ # separate zfs pool is created afterwards.
+ zfs_setup_module_conf($targetdir);
+
diversion_remove($targetdir, "/usr/sbin/update-grub");
diversion_remove($targetdir, "/usr/sbin/update-initramfs");
--
2.48.1
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 5+ messages in thread* [pve-devel] applied-series: [PATCH installer 0/3] fix #6285: always set up zfs modprobe configuration
2025-04-04 12:46 [pve-devel] [PATCH installer 0/3] fix #6285: always set up zfs modprobe configuration Christoph Heiss
` (2 preceding siblings ...)
2025-04-04 12:46 ` [pve-devel] [PATCH installer 3/3] fix #6285: install: always set up zfs modprobe configuration Christoph Heiss
@ 2025-04-04 13:01 ` Thomas Lamprecht
3 siblings, 0 replies; 5+ messages in thread
From: Thomas Lamprecht @ 2025-04-04 13:01 UTC (permalink / raw)
To: Proxmox VE development discussion, Christoph Heiss
Am 04.04.25 um 14:46 schrieb Christoph Heiss:
> Fixes #6285 [0].
>
> This also came up quite a few times already in the forum, most recently
> [1] (german).
>
> Since we do expose the option in the UI for all products nowadays (on
> ZFS-on-root installations only tho), just always write to
> /etc/modprobe.d/zfs.conf.
>
> [0] https://bugzilla.proxmox.com/show_bug.cgi?id=6285
> [1] https://forum.proxmox.com/threads/ram-auslastung-nimmt-%C3%BCber-die-zeit-drastisch-zu.164527/#post-760989
>
> Testing
> =======
>
> Tested for each product PVE, PBS, PMG the following
> - install on ZFS, not touching the ARC size setting
> - install on ZFS, set some explicit value for ARC size
> - installing on non-ZFS such as ext4
>
> .. and checking afterwards that /etc/modprobe.d/zfs.conf was written and
> contained respectively
> - the default value for the product
> - the set value
> - again the default value for the product
>
> Also tested each case with the auto-installer for PVE and PMG.
>
> Diffstat
> ========
>
> Christoph Heiss (3):
> run env: always return proper value for default zfs max arc size
> tui: bootdisk: always return proper value for default zfs max arc size
> fix #6285: install: always set up zfs modprobe configuration
>
> Proxmox/Install.pm | 11 +++---
> Proxmox/Install/RunEnv.pm | 9 +++--
> proxmox-tui-installer/src/views/bootdisk.rs | 32 +++++++++-------
> test/zfs-arc-max.pl | 42 +++++++++++++--------
> 4 files changed, 57 insertions(+), 37 deletions(-)
>
applied series, thanks for the quick implementation!
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 5+ messages in thread