From: "Max Carrara" <m.carrara@proxmox.com>
To: "Proxmox VE development discussion" <pve-devel@lists.proxmox.com>
Subject: Re: [pve-devel] [PATCH zfsonlinux v2 2/2] update arc_summary arcstat patch with new introduced values
Date: Tue, 21 May 2024 15:32:52 +0200 [thread overview]
Message-ID: <D1FCZVMCBBRO.14VE7B5ZPG0DM@proxmox.com> (raw)
In-Reply-To: <20240507150210.1391522-3-s.ivanov@proxmox.com>
On Tue May 7, 2024 at 5:02 PM CEST, Stoiko Ivanov wrote:
> ZFS 2.2.4 added new kstats for speculative prefetch in:
> 026fe796465e3da7b27d06ef5338634ee6dd30d8
>
> Adapt our patch introduced with ZFS 2.1 (for the then added MFU/MRU
> stats), to also deal with the now introduced values not being present
> (because an old kernel-module does not offer them).
>
> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
See my reply to the cover letter.
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
> ...-guard-access-to-freshly-introduced-.patch | 438 ++++++++++++++++++
> ...-guard-access-to-l2arc-MFU-MRU-stats.patch | 113 -----
> debian/patches/series | 2 +-
> 3 files changed, 439 insertions(+), 114 deletions(-)
> create mode 100644 debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
> delete mode 100644 debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
>
> diff --git a/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch b/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
> new file mode 100644
> index 00000000..bc7db2a9
> --- /dev/null
> +++ b/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
> @@ -0,0 +1,438 @@
> +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
> +From: Thomas Lamprecht <t.lamprecht@proxmox.com>
> +Date: Wed, 10 Nov 2021 09:29:47 +0100
> +Subject: [PATCH] arc stat/summary: guard access to freshly introduced stats
> +
> +l2arc MFU/MRU and zfetch past future and stride stats were introduced
> +in 2.1 and 2.2.4 respectively:
> +
> +commit 085321621e79a75bea41c2b6511da6ebfbf2ba0a added printing MFU
> +and MRU stats for 2.1 user space tools, but those keys are not
> +available in the 2.0 module. That means it may break the arcstat and
> +arc_summary tools after upgrade to 2.1 (user space), before a reboot
> +to the new 2.1 ZFS kernel-module happened, due to python raising a
> +KeyError on the dict access then.
> +
> +Move those two keys to a .get accessor with `0` as fallback, as it
> +should be better to show some possible wrong data for new stat-keys
> +than throwing an exception.
> +
> +also move l2_mfu_asize l2_mru_asize l2_prefetch_asize
> +l2_bufc_data_asize l2_bufc_metadata_asize to .get accessor
> +(these are only present with a cache device in the pool)
> +
> +guard access to iohits and uncached state introduced in
> +792a6ee462efc15a7614f27e13f0f8aaa9414a08
> +
> +guard access to zfetch past future stride stats introduced in
> +026fe796465e3da7b27d06ef5338634ee6dd30d8
> +
> +These are present in the current kernel, but lead to an exception, if
> +running the new user-space with an old kernel module.
> +
> +Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> +---
> + cmd/arc_summary | 132 ++++++++++++++++++++++++------------------------
> + cmd/arcstat.in | 48 +++++++++---------
> + 2 files changed, 90 insertions(+), 90 deletions(-)
> +
> +diff --git a/cmd/arc_summary b/cmd/arc_summary
> +index 100fb1987..30f5d23e9 100755
> +--- a/cmd/arc_summary
> ++++ b/cmd/arc_summary
> +@@ -551,21 +551,21 @@ def section_arc(kstats_dict):
> + arc_target_size = arc_stats['c']
> + arc_max = arc_stats['c_max']
> + arc_min = arc_stats['c_min']
> +- meta = arc_stats['meta']
> +- pd = arc_stats['pd']
> +- pm = arc_stats['pm']
> +- anon_data = arc_stats['anon_data']
> +- anon_metadata = arc_stats['anon_metadata']
> +- mfu_data = arc_stats['mfu_data']
> +- mfu_metadata = arc_stats['mfu_metadata']
> +- mru_data = arc_stats['mru_data']
> +- mru_metadata = arc_stats['mru_metadata']
> +- mfug_data = arc_stats['mfu_ghost_data']
> +- mfug_metadata = arc_stats['mfu_ghost_metadata']
> +- mrug_data = arc_stats['mru_ghost_data']
> +- mrug_metadata = arc_stats['mru_ghost_metadata']
> +- unc_data = arc_stats['uncached_data']
> +- unc_metadata = arc_stats['uncached_metadata']
> ++ meta = arc_stats.get('meta', 0)
> ++ pd = arc_stats.get('pd', 0)
> ++ pm = arc_stats.get('pm', 0)
> ++ anon_data = arc_stats.get('anon_data', 0)
> ++ anon_metadata = arc_stats.get('anon_metadata', 0)
> ++ mfu_data = arc_stats.get('mfu_data', 0)
> ++ mfu_metadata = arc_stats.get('mfu_metadata', 0)
> ++ mru_data = arc_stats.get('mru_data', 0)
> ++ mru_metadata = arc_stats.get('mru_metadata', 0)
> ++ mfug_data = arc_stats.get('mfu_ghost_data', 0)
> ++ mfug_metadata = arc_stats.get('mfu_ghost_metadata', 0)
> ++ mrug_data = arc_stats.get('mru_ghost_data', 0)
> ++ mrug_metadata = arc_stats.get('mru_ghost_metadata', 0)
> ++ unc_data = arc_stats.get('uncached_data', 0)
> ++ unc_metadata = arc_stats.get('uncached_metadata', 0)
> + bonus_size = arc_stats['bonus_size']
> + dnode_limit = arc_stats['arc_dnode_limit']
> + dnode_size = arc_stats['dnode_size']
> +@@ -655,13 +655,13 @@ def section_arc(kstats_dict):
> + prt_i1('L2 cached evictions:', f_bytes(arc_stats['evict_l2_cached']))
> + prt_i1('L2 eligible evictions:', f_bytes(arc_stats['evict_l2_eligible']))
> + prt_i2('L2 eligible MFU evictions:',
> +- f_perc(arc_stats['evict_l2_eligible_mfu'],
> ++ f_perc(arc_stats.get('evict_l2_eligible_mfu', 0), # 2.0 module compat
> + arc_stats['evict_l2_eligible']),
> +- f_bytes(arc_stats['evict_l2_eligible_mfu']))
> ++ f_bytes(arc_stats.get('evict_l2_eligible_mfu', 0)))
> + prt_i2('L2 eligible MRU evictions:',
> +- f_perc(arc_stats['evict_l2_eligible_mru'],
> ++ f_perc(arc_stats.get('evict_l2_eligible_mru', 0), # 2.0 module compat
> + arc_stats['evict_l2_eligible']),
> +- f_bytes(arc_stats['evict_l2_eligible_mru']))
> ++ f_bytes(arc_stats.get('evict_l2_eligible_mru', 0)))
> + prt_i1('L2 ineligible evictions:',
> + f_bytes(arc_stats['evict_l2_ineligible']))
> + print()
> +@@ -672,106 +672,106 @@ def section_archits(kstats_dict):
> + """
> +
> + arc_stats = isolate_section('arcstats', kstats_dict)
> +- all_accesses = int(arc_stats['hits'])+int(arc_stats['iohits'])+\
> ++ all_accesses = int(arc_stats['hits'])+int(arc_stats.get('iohits', 0))+\
> + int(arc_stats['misses'])
> +
> + prt_1('ARC total accesses:', f_hits(all_accesses))
> + ta_todo = (('Total hits:', arc_stats['hits']),
> +- ('Total I/O hits:', arc_stats['iohits']),
> ++ ('Total I/O hits:', arc_stats.get('iohits', 0)),
> + ('Total misses:', arc_stats['misses']))
> + for title, value in ta_todo:
> + prt_i2(title, f_perc(value, all_accesses), f_hits(value))
> + print()
> +
> + dd_total = int(arc_stats['demand_data_hits']) +\
> +- int(arc_stats['demand_data_iohits']) +\
> ++ int(arc_stats.get('demand_data_iohits', 0)) +\
> + int(arc_stats['demand_data_misses'])
> + prt_2('ARC demand data accesses:', f_perc(dd_total, all_accesses),
> + f_hits(dd_total))
> + dd_todo = (('Demand data hits:', arc_stats['demand_data_hits']),
> +- ('Demand data I/O hits:', arc_stats['demand_data_iohits']),
> ++ ('Demand data I/O hits:', arc_stats.get('demand_data_iohits', 0)),
> + ('Demand data misses:', arc_stats['demand_data_misses']))
> + for title, value in dd_todo:
> + prt_i2(title, f_perc(value, dd_total), f_hits(value))
> + print()
> +
> + dm_total = int(arc_stats['demand_metadata_hits']) +\
> +- int(arc_stats['demand_metadata_iohits']) +\
> ++ int(arc_stats.get('demand_metadata_iohits', 0)) +\
> + int(arc_stats['demand_metadata_misses'])
> + prt_2('ARC demand metadata accesses:', f_perc(dm_total, all_accesses),
> + f_hits(dm_total))
> + dm_todo = (('Demand metadata hits:', arc_stats['demand_metadata_hits']),
> + ('Demand metadata I/O hits:',
> +- arc_stats['demand_metadata_iohits']),
> ++ arc_stats.get('demand_metadata_iohits', 0)),
> + ('Demand metadata misses:', arc_stats['demand_metadata_misses']))
> + for title, value in dm_todo:
> + prt_i2(title, f_perc(value, dm_total), f_hits(value))
> + print()
> +
> + pd_total = int(arc_stats['prefetch_data_hits']) +\
> +- int(arc_stats['prefetch_data_iohits']) +\
> ++ int(arc_stats.get('prefetch_data_iohits', 0)) +\
> + int(arc_stats['prefetch_data_misses'])
> + prt_2('ARC prefetch data accesses:', f_perc(pd_total, all_accesses),
> + f_hits(pd_total))
> + pd_todo = (('Prefetch data hits:', arc_stats['prefetch_data_hits']),
> +- ('Prefetch data I/O hits:', arc_stats['prefetch_data_iohits']),
> ++ ('Prefetch data I/O hits:', arc_stats.get('prefetch_data_iohits', 0)),
> + ('Prefetch data misses:', arc_stats['prefetch_data_misses']))
> + for title, value in pd_todo:
> + prt_i2(title, f_perc(value, pd_total), f_hits(value))
> + print()
> +
> + pm_total = int(arc_stats['prefetch_metadata_hits']) +\
> +- int(arc_stats['prefetch_metadata_iohits']) +\
> ++ int(arc_stats.get('prefetch_metadata_iohits', 0)) +\
> + int(arc_stats['prefetch_metadata_misses'])
> + prt_2('ARC prefetch metadata accesses:', f_perc(pm_total, all_accesses),
> + f_hits(pm_total))
> + pm_todo = (('Prefetch metadata hits:',
> + arc_stats['prefetch_metadata_hits']),
> + ('Prefetch metadata I/O hits:',
> +- arc_stats['prefetch_metadata_iohits']),
> ++ arc_stats.get('prefetch_metadata_iohits', 0)),
> + ('Prefetch metadata misses:',
> + arc_stats['prefetch_metadata_misses']))
> + for title, value in pm_todo:
> + prt_i2(title, f_perc(value, pm_total), f_hits(value))
> + print()
> +
> +- all_prefetches = int(arc_stats['predictive_prefetch'])+\
> +- int(arc_stats['prescient_prefetch'])
> ++ all_prefetches = int(arc_stats.get('predictive_prefetch', 0))+\
> ++ int(arc_stats.get('prescient_prefetch', 0))
> + prt_2('ARC predictive prefetches:',
> +- f_perc(arc_stats['predictive_prefetch'], all_prefetches),
> +- f_hits(arc_stats['predictive_prefetch']))
> ++ f_perc(arc_stats.get('predictive_prefetch', 0), all_prefetches),
> ++ f_hits(arc_stats.get('predictive_prefetch', 0)))
> + prt_i2('Demand hits after predictive:',
> + f_perc(arc_stats['demand_hit_predictive_prefetch'],
> +- arc_stats['predictive_prefetch']),
> ++ arc_stats.get('predictive_prefetch', 0)),
> + f_hits(arc_stats['demand_hit_predictive_prefetch']))
> + prt_i2('Demand I/O hits after predictive:',
> +- f_perc(arc_stats['demand_iohit_predictive_prefetch'],
> +- arc_stats['predictive_prefetch']),
> +- f_hits(arc_stats['demand_iohit_predictive_prefetch']))
> +- never = int(arc_stats['predictive_prefetch']) -\
> ++ f_perc(arc_stats.get('demand_iohit_predictive_prefetch', 0),
> ++ arc_stats.get('predictive_prefetch', 0)),
> ++ f_hits(arc_stats.get('demand_iohit_predictive_prefetch', 0)))
> ++ never = int(arc_stats.get('predictive_prefetch', 0)) -\
> + int(arc_stats['demand_hit_predictive_prefetch']) -\
> +- int(arc_stats['demand_iohit_predictive_prefetch'])
> ++ int(arc_stats.get('demand_iohit_predictive_prefetch', 0))
> + prt_i2('Never demanded after predictive:',
> +- f_perc(never, arc_stats['predictive_prefetch']),
> ++ f_perc(never, arc_stats.get('predictive_prefetch', 0)),
> + f_hits(never))
> + print()
> +
> + prt_2('ARC prescient prefetches:',
> +- f_perc(arc_stats['prescient_prefetch'], all_prefetches),
> +- f_hits(arc_stats['prescient_prefetch']))
> ++ f_perc(arc_stats.get('prescient_prefetch', 0), all_prefetches),
> ++ f_hits(arc_stats.get('prescient_prefetch', 0)))
> + prt_i2('Demand hits after prescient:',
> + f_perc(arc_stats['demand_hit_prescient_prefetch'],
> +- arc_stats['prescient_prefetch']),
> ++ arc_stats.get('prescient_prefetch', 0)),
> + f_hits(arc_stats['demand_hit_prescient_prefetch']))
> + prt_i2('Demand I/O hits after prescient:',
> +- f_perc(arc_stats['demand_iohit_prescient_prefetch'],
> +- arc_stats['prescient_prefetch']),
> +- f_hits(arc_stats['demand_iohit_prescient_prefetch']))
> +- never = int(arc_stats['prescient_prefetch'])-\
> ++ f_perc(arc_stats.get('demand_iohit_prescient_prefetch', 0),
> ++ arc_stats.get('prescient_prefetch', 0)),
> ++ f_hits(arc_stats.get('demand_iohit_prescient_prefetch', 0)))
> ++ never = int(arc_stats.get('prescient_prefetch', 0))-\
> + int(arc_stats['demand_hit_prescient_prefetch'])-\
> +- int(arc_stats['demand_iohit_prescient_prefetch'])
> ++ int(arc_stats.get('demand_iohit_prescient_prefetch', 0))
> + prt_i2('Never demanded after prescient:',
> +- f_perc(never, arc_stats['prescient_prefetch']),
> ++ f_perc(never, arc_stats.get('prescient_prefetch', 0)),
> + f_hits(never))
> + print()
> +
> +@@ -782,7 +782,7 @@ def section_archits(kstats_dict):
> + arc_stats['mfu_ghost_hits']),
> + ('Most recently used (MRU) ghost:',
> + arc_stats['mru_ghost_hits']),
> +- ('Uncached:', arc_stats['uncached_hits']))
> ++ ('Uncached:', arc_stats.get('uncached_hits', 0)))
> + for title, value in cl_todo:
> + prt_i2(title, f_perc(value, all_accesses), f_hits(value))
> + print()
> +@@ -794,26 +794,26 @@ def section_dmu(kstats_dict):
> + zfetch_stats = isolate_section('zfetchstats', kstats_dict)
> +
> + zfetch_access_total = int(zfetch_stats['hits']) +\
> +- int(zfetch_stats['future']) + int(zfetch_stats['stride']) +\
> +- int(zfetch_stats['past']) + int(zfetch_stats['misses'])
> ++ int(zfetch_stats.get('future', 0)) + int(zfetch_stats.get('stride', 0)) +\
> ++ int(zfetch_stats.get('past', 0)) + int(zfetch_stats['misses'])
> +
> + prt_1('DMU predictive prefetcher calls:', f_hits(zfetch_access_total))
> + prt_i2('Stream hits:',
> + f_perc(zfetch_stats['hits'], zfetch_access_total),
> + f_hits(zfetch_stats['hits']))
> +- future = int(zfetch_stats['future']) + int(zfetch_stats['stride'])
> ++ future = int(zfetch_stats.get('future', 0)) + int(zfetch_stats.get('stride', 0))
> + prt_i2('Hits ahead of stream:', f_perc(future, zfetch_access_total),
> + f_hits(future))
> + prt_i2('Hits behind stream:',
> +- f_perc(zfetch_stats['past'], zfetch_access_total),
> +- f_hits(zfetch_stats['past']))
> ++ f_perc(zfetch_stats.get('past', 0), zfetch_access_total),
> ++ f_hits(zfetch_stats.get('past', 0)))
> + prt_i2('Stream misses:',
> + f_perc(zfetch_stats['misses'], zfetch_access_total),
> + f_hits(zfetch_stats['misses']))
> + prt_i2('Streams limit reached:',
> + f_perc(zfetch_stats['max_streams'], zfetch_stats['misses']),
> + f_hits(zfetch_stats['max_streams']))
> +- prt_i1('Stream strides:', f_hits(zfetch_stats['stride']))
> ++ prt_i1('Stream strides:', f_hits(zfetch_stats.get('stride', 0)))
> + prt_i1('Prefetches issued', f_hits(zfetch_stats['io_issued']))
> + print()
> +
> +@@ -860,20 +860,20 @@ def section_l2arc(kstats_dict):
> + f_perc(arc_stats['l2_hdr_size'], arc_stats['l2_size']),
> + f_bytes(arc_stats['l2_hdr_size']))
> + prt_i2('MFU allocated size:',
> +- f_perc(arc_stats['l2_mfu_asize'], arc_stats['l2_asize']),
> +- f_bytes(arc_stats['l2_mfu_asize']))
> ++ f_perc(arc_stats.get('l2_mfu_asize', 0), arc_stats['l2_asize']),
> ++ f_bytes(arc_stats.get('l2_mfu_asize', 0))) # 2.0 module compat
> + prt_i2('MRU allocated size:',
> +- f_perc(arc_stats['l2_mru_asize'], arc_stats['l2_asize']),
> +- f_bytes(arc_stats['l2_mru_asize']))
> ++ f_perc(arc_stats.get('l2_mru_asize', 0), arc_stats['l2_asize']),
> ++ f_bytes(arc_stats.get('l2_mru_asize', 0))) # 2.0 module compat
> + prt_i2('Prefetch allocated size:',
> +- f_perc(arc_stats['l2_prefetch_asize'], arc_stats['l2_asize']),
> +- f_bytes(arc_stats['l2_prefetch_asize']))
> ++ f_perc(arc_stats.get('l2_prefetch_asize', 0), arc_stats['l2_asize']),
> ++ f_bytes(arc_stats.get('l2_prefetch_asize',0))) # 2.0 module compat
> + prt_i2('Data (buffer content) allocated size:',
> +- f_perc(arc_stats['l2_bufc_data_asize'], arc_stats['l2_asize']),
> +- f_bytes(arc_stats['l2_bufc_data_asize']))
> ++ f_perc(arc_stats.get('l2_bufc_data_asize', 0), arc_stats['l2_asize']),
> ++ f_bytes(arc_stats.get('l2_bufc_data_asize', 0))) # 2.0 module compat
> + prt_i2('Metadata (buffer content) allocated size:',
> +- f_perc(arc_stats['l2_bufc_metadata_asize'], arc_stats['l2_asize']),
> +- f_bytes(arc_stats['l2_bufc_metadata_asize']))
> ++ f_perc(arc_stats.get('l2_bufc_metadata_asize', 0), arc_stats['l2_asize']),
> ++ f_bytes(arc_stats.get('l2_bufc_metadata_asize', 0))) # 2.0 module compat
> +
> + print()
> + prt_1('L2ARC breakdown:', f_hits(l2_access_total))
> +diff --git a/cmd/arcstat.in b/cmd/arcstat.in
> +index c4f10a1d6..bf47ec90e 100755
> +--- a/cmd/arcstat.in
> ++++ b/cmd/arcstat.in
> +@@ -510,7 +510,7 @@ def calculate():
> + v = dict()
> + v["time"] = time.strftime("%H:%M:%S", time.localtime())
> + v["hits"] = d["hits"] // sint
> +- v["iohs"] = d["iohits"] // sint
> ++ v["iohs"] = d.get("iohits", 0) // sint
> + v["miss"] = d["misses"] // sint
> + v["read"] = v["hits"] + v["iohs"] + v["miss"]
> + v["hit%"] = 100 * v["hits"] // v["read"] if v["read"] > 0 else 0
> +@@ -518,7 +518,7 @@ def calculate():
> + v["miss%"] = 100 - v["hit%"] - v["ioh%"] if v["read"] > 0 else 0
> +
> + v["dhit"] = (d["demand_data_hits"] + d["demand_metadata_hits"]) // sint
> +- v["dioh"] = (d["demand_data_iohits"] + d["demand_metadata_iohits"]) // sint
> ++ v["dioh"] = (d.get("demand_data_iohits", 0) + d.get("demand_metadata_iohits", 0)) // sint
> + v["dmis"] = (d["demand_data_misses"] + d["demand_metadata_misses"]) // sint
> +
> + v["dread"] = v["dhit"] + v["dioh"] + v["dmis"]
> +@@ -527,7 +527,7 @@ def calculate():
> + v["dm%"] = 100 - v["dh%"] - v["di%"] if v["dread"] > 0 else 0
> +
> + v["ddhit"] = d["demand_data_hits"] // sint
> +- v["ddioh"] = d["demand_data_iohits"] // sint
> ++ v["ddioh"] = d.get("demand_data_iohits", 0) // sint
> + v["ddmis"] = d["demand_data_misses"] // sint
> +
> + v["ddread"] = v["ddhit"] + v["ddioh"] + v["ddmis"]
> +@@ -536,7 +536,7 @@ def calculate():
> + v["ddm%"] = 100 - v["ddh%"] - v["ddi%"] if v["ddread"] > 0 else 0
> +
> + v["dmhit"] = d["demand_metadata_hits"] // sint
> +- v["dmioh"] = d["demand_metadata_iohits"] // sint
> ++ v["dmioh"] = d.get("demand_metadata_iohits", 0) // sint
> + v["dmmis"] = d["demand_metadata_misses"] // sint
> +
> + v["dmread"] = v["dmhit"] + v["dmioh"] + v["dmmis"]
> +@@ -545,8 +545,8 @@ def calculate():
> + v["dmm%"] = 100 - v["dmh%"] - v["dmi%"] if v["dmread"] > 0 else 0
> +
> + v["phit"] = (d["prefetch_data_hits"] + d["prefetch_metadata_hits"]) // sint
> +- v["pioh"] = (d["prefetch_data_iohits"] +
> +- d["prefetch_metadata_iohits"]) // sint
> ++ v["pioh"] = (d.get("prefetch_data_iohits", 0) +
> ++ d.get("prefetch_metadata_iohits", 0)) // sint
> + v["pmis"] = (d["prefetch_data_misses"] +
> + d["prefetch_metadata_misses"]) // sint
> +
> +@@ -556,7 +556,7 @@ def calculate():
> + v["pm%"] = 100 - v["ph%"] - v["pi%"] if v["pread"] > 0 else 0
> +
> + v["pdhit"] = d["prefetch_data_hits"] // sint
> +- v["pdioh"] = d["prefetch_data_iohits"] // sint
> ++ v["pdioh"] = d.get("prefetch_data_iohits", 0) // sint
> + v["pdmis"] = d["prefetch_data_misses"] // sint
> +
> + v["pdread"] = v["pdhit"] + v["pdioh"] + v["pdmis"]
> +@@ -565,7 +565,7 @@ def calculate():
> + v["pdm%"] = 100 - v["pdh%"] - v["pdi%"] if v["pdread"] > 0 else 0
> +
> + v["pmhit"] = d["prefetch_metadata_hits"] // sint
> +- v["pmioh"] = d["prefetch_metadata_iohits"] // sint
> ++ v["pmioh"] = d.get("prefetch_metadata_iohits", 0) // sint
> + v["pmmis"] = d["prefetch_metadata_misses"] // sint
> +
> + v["pmread"] = v["pmhit"] + v["pmioh"] + v["pmmis"]
> +@@ -575,8 +575,8 @@ def calculate():
> +
> + v["mhit"] = (d["prefetch_metadata_hits"] +
> + d["demand_metadata_hits"]) // sint
> +- v["mioh"] = (d["prefetch_metadata_iohits"] +
> +- d["demand_metadata_iohits"]) // sint
> ++ v["mioh"] = (d.get("prefetch_metadata_iohits", 0) +
> ++ d.get("demand_metadata_iohits", 0)) // sint
> + v["mmis"] = (d["prefetch_metadata_misses"] +
> + d["demand_metadata_misses"]) // sint
> +
> +@@ -592,24 +592,24 @@ def calculate():
> + v["mru"] = d["mru_hits"] // sint
> + v["mrug"] = d["mru_ghost_hits"] // sint
> + v["mfug"] = d["mfu_ghost_hits"] // sint
> +- v["unc"] = d["uncached_hits"] // sint
> ++ v["unc"] = d.get("uncached_hits", 0) // sint
> + v["eskip"] = d["evict_skip"] // sint
> + v["el2skip"] = d["evict_l2_skip"] // sint
> + v["el2cach"] = d["evict_l2_cached"] // sint
> + v["el2el"] = d["evict_l2_eligible"] // sint
> +- v["el2mfu"] = d["evict_l2_eligible_mfu"] // sint
> +- v["el2mru"] = d["evict_l2_eligible_mru"] // sint
> ++ v["el2mfu"] = d.get("evict_l2_eligible_mfu", 0) // sint
> ++ v["el2mru"] = d.get("evict_l2_eligible_mru", 0) // sint
> + v["el2inel"] = d["evict_l2_ineligible"] // sint
> + v["mtxmis"] = d["mutex_miss"] // sint
> +- v["ztotal"] = (d["zfetch_hits"] + d["zfetch_future"] + d["zfetch_stride"] +
> +- d["zfetch_past"] + d["zfetch_misses"]) // sint
> ++ v["ztotal"] = (d["zfetch_hits"] + d.get("zfetch_future", 0) + d.get("zfetch_stride", 0) +
> ++ d.get("zfetch_past", 0) + d["zfetch_misses"]) // sint
> + v["zhits"] = d["zfetch_hits"] // sint
> +- v["zahead"] = (d["zfetch_future"] + d["zfetch_stride"]) // sint
> +- v["zpast"] = d["zfetch_past"] // sint
> ++ v["zahead"] = (d.get("zfetch_future", 0) + d.get("zfetch_stride", 0)) // sint
> ++ v["zpast"] = d.get("zfetch_past", 0) // sint
> + v["zmisses"] = d["zfetch_misses"] // sint
> + v["zmax"] = d["zfetch_max_streams"] // sint
> +- v["zfuture"] = d["zfetch_future"] // sint
> +- v["zstride"] = d["zfetch_stride"] // sint
> ++ v["zfuture"] = d.get("zfetch_future", 0) // sint
> ++ v["zstride"] = d.get("zfetch_stride", 0) // sint
> + v["zissued"] = d["zfetch_io_issued"] // sint
> + v["zactive"] = d["zfetch_io_active"] // sint
> +
> +@@ -624,11 +624,11 @@ def calculate():
> + v["l2size"] = cur["l2_size"]
> + v["l2bytes"] = d["l2_read_bytes"] // sint
> +
> +- v["l2pref"] = cur["l2_prefetch_asize"]
> +- v["l2mfu"] = cur["l2_mfu_asize"]
> +- v["l2mru"] = cur["l2_mru_asize"]
> +- v["l2data"] = cur["l2_bufc_data_asize"]
> +- v["l2meta"] = cur["l2_bufc_metadata_asize"]
> ++ v["l2pref"] = cur.get("l2_prefetch_asize", 0)
> ++ v["l2mfu"] = cur.get("l2_mfu_asize", 0)
> ++ v["l2mru"] = cur.get("l2_mru_asize", 0)
> ++ v["l2data"] = cur.get("l2_bufc_data_asize", 0)
> ++ v["l2meta"] = cur.get("l2_bufc_metadata_asize", 0)
> + v["l2pref%"] = 100 * v["l2pref"] // v["l2asize"]
> + v["l2mfu%"] = 100 * v["l2mfu"] // v["l2asize"]
> + v["l2mru%"] = 100 * v["l2mru"] // v["l2asize"]
> diff --git a/debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch b/debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
> deleted file mode 100644
> index 2e7c207d..00000000
> --- a/debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
> +++ /dev/null
> @@ -1,113 +0,0 @@
> -From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
> -From: Thomas Lamprecht <t.lamprecht@proxmox.com>
> -Date: Wed, 10 Nov 2021 09:29:47 +0100
> -Subject: [PATCH] arc stat/summary: guard access to l2arc MFU/MRU stats
> -
> -commit 085321621e79a75bea41c2b6511da6ebfbf2ba0a added printing MFU
> -and MRU stats for 2.1 user space tools, but those keys are not
> -available in the 2.0 module. That means it may break the arcstat and
> -arc_summary tools after upgrade to 2.1 (user space), before a reboot
> -to the new 2.1 ZFS kernel-module happened, due to python raising a
> -KeyError on the dict access then.
> -
> -Move those two keys to a .get accessor with `0` as fallback, as it
> -should be better to show some possible wrong data for new stat-keys
> -than throwing an exception.
> -
> -Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
> -
> -also move l2_mfu_asize l2_mru_asize l2_prefetch_asize
> -l2_bufc_data_asize l2_bufc_metadata_asize to .get accessor
> -(these are only present with a cache device in the pool)
> -Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> -Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
> ----
> - cmd/arc_summary | 28 ++++++++++++++--------------
> - cmd/arcstat.in | 14 +++++++-------
> - 2 files changed, 21 insertions(+), 21 deletions(-)
> -
> -diff --git a/cmd/arc_summary b/cmd/arc_summary
> -index 100fb1987..86b2260a1 100755
> ---- a/cmd/arc_summary
> -+++ b/cmd/arc_summary
> -@@ -655,13 +655,13 @@ def section_arc(kstats_dict):
> - prt_i1('L2 cached evictions:', f_bytes(arc_stats['evict_l2_cached']))
> - prt_i1('L2 eligible evictions:', f_bytes(arc_stats['evict_l2_eligible']))
> - prt_i2('L2 eligible MFU evictions:',
> -- f_perc(arc_stats['evict_l2_eligible_mfu'],
> -+ f_perc(arc_stats.get('evict_l2_eligible_mfu', 0), # 2.0 module compat
> - arc_stats['evict_l2_eligible']),
> -- f_bytes(arc_stats['evict_l2_eligible_mfu']))
> -+ f_bytes(arc_stats.get('evict_l2_eligible_mfu', 0)))
> - prt_i2('L2 eligible MRU evictions:',
> -- f_perc(arc_stats['evict_l2_eligible_mru'],
> -+ f_perc(arc_stats.get('evict_l2_eligible_mru', 0), # 2.0 module compat
> - arc_stats['evict_l2_eligible']),
> -- f_bytes(arc_stats['evict_l2_eligible_mru']))
> -+ f_bytes(arc_stats.get('evict_l2_eligible_mru', 0)))
> - prt_i1('L2 ineligible evictions:',
> - f_bytes(arc_stats['evict_l2_ineligible']))
> - print()
> -@@ -860,20 +860,20 @@ def section_l2arc(kstats_dict):
> - f_perc(arc_stats['l2_hdr_size'], arc_stats['l2_size']),
> - f_bytes(arc_stats['l2_hdr_size']))
> - prt_i2('MFU allocated size:',
> -- f_perc(arc_stats['l2_mfu_asize'], arc_stats['l2_asize']),
> -- f_bytes(arc_stats['l2_mfu_asize']))
> -+ f_perc(arc_stats.get('l2_mfu_asize', 0), arc_stats['l2_asize']),
> -+ f_bytes(arc_stats.get('l2_mfu_asize', 0))) # 2.0 module compat
> - prt_i2('MRU allocated size:',
> -- f_perc(arc_stats['l2_mru_asize'], arc_stats['l2_asize']),
> -- f_bytes(arc_stats['l2_mru_asize']))
> -+ f_perc(arc_stats.get('l2_mru_asize', 0), arc_stats['l2_asize']),
> -+ f_bytes(arc_stats.get('l2_mru_asize', 0))) # 2.0 module compat
> - prt_i2('Prefetch allocated size:',
> -- f_perc(arc_stats['l2_prefetch_asize'], arc_stats['l2_asize']),
> -- f_bytes(arc_stats['l2_prefetch_asize']))
> -+ f_perc(arc_stats.get('l2_prefetch_asize', 0), arc_stats['l2_asize']),
> -+ f_bytes(arc_stats.get('l2_prefetch_asize',0))) # 2.0 module compat
> - prt_i2('Data (buffer content) allocated size:',
> -- f_perc(arc_stats['l2_bufc_data_asize'], arc_stats['l2_asize']),
> -- f_bytes(arc_stats['l2_bufc_data_asize']))
> -+ f_perc(arc_stats.get('l2_bufc_data_asize', 0), arc_stats['l2_asize']),
> -+ f_bytes(arc_stats.get('l2_bufc_data_asize', 0))) # 2.0 module compat
> - prt_i2('Metadata (buffer content) allocated size:',
> -- f_perc(arc_stats['l2_bufc_metadata_asize'], arc_stats['l2_asize']),
> -- f_bytes(arc_stats['l2_bufc_metadata_asize']))
> -+ f_perc(arc_stats.get('l2_bufc_metadata_asize', 0), arc_stats['l2_asize']),
> -+ f_bytes(arc_stats.get('l2_bufc_metadata_asize', 0))) # 2.0 module compat
> -
> - print()
> - prt_1('L2ARC breakdown:', f_hits(l2_access_total))
> -diff --git a/cmd/arcstat.in b/cmd/arcstat.in
> -index c4f10a1d6..c570dca88 100755
> ---- a/cmd/arcstat.in
> -+++ b/cmd/arcstat.in
> -@@ -597,8 +597,8 @@ def calculate():
> - v["el2skip"] = d["evict_l2_skip"] // sint
> - v["el2cach"] = d["evict_l2_cached"] // sint
> - v["el2el"] = d["evict_l2_eligible"] // sint
> -- v["el2mfu"] = d["evict_l2_eligible_mfu"] // sint
> -- v["el2mru"] = d["evict_l2_eligible_mru"] // sint
> -+ v["el2mfu"] = d.get("evict_l2_eligible_mfu", 0) // sint
> -+ v["el2mru"] = d.get("evict_l2_eligible_mru", 0) // sint
> - v["el2inel"] = d["evict_l2_ineligible"] // sint
> - v["mtxmis"] = d["mutex_miss"] // sint
> - v["ztotal"] = (d["zfetch_hits"] + d["zfetch_future"] + d["zfetch_stride"] +
> -@@ -624,11 +624,11 @@ def calculate():
> - v["l2size"] = cur["l2_size"]
> - v["l2bytes"] = d["l2_read_bytes"] // sint
> -
> -- v["l2pref"] = cur["l2_prefetch_asize"]
> -- v["l2mfu"] = cur["l2_mfu_asize"]
> -- v["l2mru"] = cur["l2_mru_asize"]
> -- v["l2data"] = cur["l2_bufc_data_asize"]
> -- v["l2meta"] = cur["l2_bufc_metadata_asize"]
> -+ v["l2pref"] = cur.get("l2_prefetch_asize", 0)
> -+ v["l2mfu"] = cur.get("l2_mfu_asize", 0)
> -+ v["l2mru"] = cur.get("l2_mru_asize", 0)
> -+ v["l2data"] = cur.get("l2_bufc_data_asize", 0)
> -+ v["l2meta"] = cur.get("l2_bufc_metadata_asize", 0)
> - v["l2pref%"] = 100 * v["l2pref"] // v["l2asize"]
> - v["l2mfu%"] = 100 * v["l2mfu"] // v["l2asize"]
> - v["l2mru%"] = 100 * v["l2mru"] // v["l2asize"]
> diff --git a/debian/patches/series b/debian/patches/series
> index 35f81d13..229027ff 100644
> --- a/debian/patches/series
> +++ b/debian/patches/series
> @@ -6,6 +6,6 @@
> 0006-dont-symlink-zed-scripts.patch
> 0007-Add-systemd-unit-for-importing-specific-pools.patch
> 0008-Patch-move-manpage-arcstat-1-to-arcstat-8.patch
> -0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
> +0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
> 0010-Fix-nfs_truncate_shares-without-etc-exports.d.patch
> 0011-zpool-status-tighten-bounds-for-noalloc-stat-availab.patch
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next prev parent reply other threads:[~2024-05-21 13:33 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-07 15:02 [pve-devel] [PATCH zfsonlinux v2 0/2] Update to ZFS 2.2.4 Stoiko Ivanov
2024-05-07 15:02 ` [pve-devel] [PATCH zfsonlinux v2 1/2] update zfs submodule to 2.2.4 and refresh patches Stoiko Ivanov
2024-05-21 13:56 ` Max Carrara
2024-05-07 15:02 ` [pve-devel] [PATCH zfsonlinux v2 2/2] update arc_summary arcstat patch with new introduced values Stoiko Ivanov
2024-05-21 13:32 ` Max Carrara [this message]
2024-05-21 13:31 ` [pve-devel] [PATCH zfsonlinux v2 0/2] Update to ZFS 2.2.4 Max Carrara
2024-05-21 14:06 ` [pve-devel] applied-series: " Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=D1FCZVMCBBRO.14VE7B5ZPG0DM@proxmox.com \
--to=m.carrara@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal