From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 9F8241FF389 for ; Tue, 7 May 2024 17:02:19 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C300510F71; Tue, 7 May 2024 17:02:18 +0200 (CEST) From: Stoiko Ivanov To: pve-devel@lists.proxmox.com Date: Tue, 7 May 2024 17:02:10 +0200 Message-Id: <20240507150210.1391522-3-s.ivanov@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240507150210.1391522-1-s.ivanov@proxmox.com> References: <20240507150210.1391522-1-s.ivanov@proxmox.com> MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.099 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LOTSOFHASH 0.25 Emails with lots of hash-like gibberish PROLO_LEO1 0.1 Meta Catches all Leo drug variations so far SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [arcstat.in] Subject: [pve-devel] [PATCH zfsonlinux v2 2/2] update arc_summary arcstat patch with new introduced values X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" ZFS 2.2.4 added new kstats for speculative prefetch in: 026fe796465e3da7b27d06ef5338634ee6dd30d8 Adapt our patch introduced with ZFS 2.1 (for the then added MFU/MRU stats), to also deal with the now introduced values not being present (because an old kernel-module does not offer them). Signed-off-by: Stoiko Ivanov --- ...-guard-access-to-freshly-introduced-.patch | 438 ++++++++++++++++++ ...-guard-access-to-l2arc-MFU-MRU-stats.patch | 113 ----- debian/patches/series | 2 +- 3 files changed, 439 insertions(+), 114 deletions(-) create mode 100644 debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch delete mode 100644 debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch diff --git a/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch b/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch new file mode 100644 index 00000000..bc7db2a9 --- /dev/null +++ b/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch @@ -0,0 +1,438 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Thomas Lamprecht +Date: Wed, 10 Nov 2021 09:29:47 +0100 +Subject: [PATCH] arc stat/summary: guard access to freshly introduced stats + +l2arc MFU/MRU and zfetch past future and stride stats were introduced +in 2.1 and 2.2.4 respectively: + +commit 085321621e79a75bea41c2b6511da6ebfbf2ba0a added printing MFU +and MRU stats for 2.1 user space tools, but those keys are not +available in the 2.0 module. That means it may break the arcstat and +arc_summary tools after upgrade to 2.1 (user space), before a reboot +to the new 2.1 ZFS kernel-module happened, due to python raising a +KeyError on the dict access then. + +Move those two keys to a .get accessor with `0` as fallback, as it +should be better to show some possible wrong data for new stat-keys +than throwing an exception. + +also move l2_mfu_asize l2_mru_asize l2_prefetch_asize +l2_bufc_data_asize l2_bufc_metadata_asize to .get accessor +(these are only present with a cache device in the pool) + +guard access to iohits and uncached state introduced in +792a6ee462efc15a7614f27e13f0f8aaa9414a08 + +guard access to zfetch past future stride stats introduced in +026fe796465e3da7b27d06ef5338634ee6dd30d8 + +These are present in the current kernel, but lead to an exception, if +running the new user-space with an old kernel module. + +Signed-off-by: Stoiko Ivanov +--- + cmd/arc_summary | 132 ++++++++++++++++++++++++------------------------ + cmd/arcstat.in | 48 +++++++++--------- + 2 files changed, 90 insertions(+), 90 deletions(-) + +diff --git a/cmd/arc_summary b/cmd/arc_summary +index 100fb1987..30f5d23e9 100755 +--- a/cmd/arc_summary ++++ b/cmd/arc_summary +@@ -551,21 +551,21 @@ def section_arc(kstats_dict): + arc_target_size = arc_stats['c'] + arc_max = arc_stats['c_max'] + arc_min = arc_stats['c_min'] +- meta = arc_stats['meta'] +- pd = arc_stats['pd'] +- pm = arc_stats['pm'] +- anon_data = arc_stats['anon_data'] +- anon_metadata = arc_stats['anon_metadata'] +- mfu_data = arc_stats['mfu_data'] +- mfu_metadata = arc_stats['mfu_metadata'] +- mru_data = arc_stats['mru_data'] +- mru_metadata = arc_stats['mru_metadata'] +- mfug_data = arc_stats['mfu_ghost_data'] +- mfug_metadata = arc_stats['mfu_ghost_metadata'] +- mrug_data = arc_stats['mru_ghost_data'] +- mrug_metadata = arc_stats['mru_ghost_metadata'] +- unc_data = arc_stats['uncached_data'] +- unc_metadata = arc_stats['uncached_metadata'] ++ meta = arc_stats.get('meta', 0) ++ pd = arc_stats.get('pd', 0) ++ pm = arc_stats.get('pm', 0) ++ anon_data = arc_stats.get('anon_data', 0) ++ anon_metadata = arc_stats.get('anon_metadata', 0) ++ mfu_data = arc_stats.get('mfu_data', 0) ++ mfu_metadata = arc_stats.get('mfu_metadata', 0) ++ mru_data = arc_stats.get('mru_data', 0) ++ mru_metadata = arc_stats.get('mru_metadata', 0) ++ mfug_data = arc_stats.get('mfu_ghost_data', 0) ++ mfug_metadata = arc_stats.get('mfu_ghost_metadata', 0) ++ mrug_data = arc_stats.get('mru_ghost_data', 0) ++ mrug_metadata = arc_stats.get('mru_ghost_metadata', 0) ++ unc_data = arc_stats.get('uncached_data', 0) ++ unc_metadata = arc_stats.get('uncached_metadata', 0) + bonus_size = arc_stats['bonus_size'] + dnode_limit = arc_stats['arc_dnode_limit'] + dnode_size = arc_stats['dnode_size'] +@@ -655,13 +655,13 @@ def section_arc(kstats_dict): + prt_i1('L2 cached evictions:', f_bytes(arc_stats['evict_l2_cached'])) + prt_i1('L2 eligible evictions:', f_bytes(arc_stats['evict_l2_eligible'])) + prt_i2('L2 eligible MFU evictions:', +- f_perc(arc_stats['evict_l2_eligible_mfu'], ++ f_perc(arc_stats.get('evict_l2_eligible_mfu', 0), # 2.0 module compat + arc_stats['evict_l2_eligible']), +- f_bytes(arc_stats['evict_l2_eligible_mfu'])) ++ f_bytes(arc_stats.get('evict_l2_eligible_mfu', 0))) + prt_i2('L2 eligible MRU evictions:', +- f_perc(arc_stats['evict_l2_eligible_mru'], ++ f_perc(arc_stats.get('evict_l2_eligible_mru', 0), # 2.0 module compat + arc_stats['evict_l2_eligible']), +- f_bytes(arc_stats['evict_l2_eligible_mru'])) ++ f_bytes(arc_stats.get('evict_l2_eligible_mru', 0))) + prt_i1('L2 ineligible evictions:', + f_bytes(arc_stats['evict_l2_ineligible'])) + print() +@@ -672,106 +672,106 @@ def section_archits(kstats_dict): + """ + + arc_stats = isolate_section('arcstats', kstats_dict) +- all_accesses = int(arc_stats['hits'])+int(arc_stats['iohits'])+\ ++ all_accesses = int(arc_stats['hits'])+int(arc_stats.get('iohits', 0))+\ + int(arc_stats['misses']) + + prt_1('ARC total accesses:', f_hits(all_accesses)) + ta_todo = (('Total hits:', arc_stats['hits']), +- ('Total I/O hits:', arc_stats['iohits']), ++ ('Total I/O hits:', arc_stats.get('iohits', 0)), + ('Total misses:', arc_stats['misses'])) + for title, value in ta_todo: + prt_i2(title, f_perc(value, all_accesses), f_hits(value)) + print() + + dd_total = int(arc_stats['demand_data_hits']) +\ +- int(arc_stats['demand_data_iohits']) +\ ++ int(arc_stats.get('demand_data_iohits', 0)) +\ + int(arc_stats['demand_data_misses']) + prt_2('ARC demand data accesses:', f_perc(dd_total, all_accesses), + f_hits(dd_total)) + dd_todo = (('Demand data hits:', arc_stats['demand_data_hits']), +- ('Demand data I/O hits:', arc_stats['demand_data_iohits']), ++ ('Demand data I/O hits:', arc_stats.get('demand_data_iohits', 0)), + ('Demand data misses:', arc_stats['demand_data_misses'])) + for title, value in dd_todo: + prt_i2(title, f_perc(value, dd_total), f_hits(value)) + print() + + dm_total = int(arc_stats['demand_metadata_hits']) +\ +- int(arc_stats['demand_metadata_iohits']) +\ ++ int(arc_stats.get('demand_metadata_iohits', 0)) +\ + int(arc_stats['demand_metadata_misses']) + prt_2('ARC demand metadata accesses:', f_perc(dm_total, all_accesses), + f_hits(dm_total)) + dm_todo = (('Demand metadata hits:', arc_stats['demand_metadata_hits']), + ('Demand metadata I/O hits:', +- arc_stats['demand_metadata_iohits']), ++ arc_stats.get('demand_metadata_iohits', 0)), + ('Demand metadata misses:', arc_stats['demand_metadata_misses'])) + for title, value in dm_todo: + prt_i2(title, f_perc(value, dm_total), f_hits(value)) + print() + + pd_total = int(arc_stats['prefetch_data_hits']) +\ +- int(arc_stats['prefetch_data_iohits']) +\ ++ int(arc_stats.get('prefetch_data_iohits', 0)) +\ + int(arc_stats['prefetch_data_misses']) + prt_2('ARC prefetch data accesses:', f_perc(pd_total, all_accesses), + f_hits(pd_total)) + pd_todo = (('Prefetch data hits:', arc_stats['prefetch_data_hits']), +- ('Prefetch data I/O hits:', arc_stats['prefetch_data_iohits']), ++ ('Prefetch data I/O hits:', arc_stats.get('prefetch_data_iohits', 0)), + ('Prefetch data misses:', arc_stats['prefetch_data_misses'])) + for title, value in pd_todo: + prt_i2(title, f_perc(value, pd_total), f_hits(value)) + print() + + pm_total = int(arc_stats['prefetch_metadata_hits']) +\ +- int(arc_stats['prefetch_metadata_iohits']) +\ ++ int(arc_stats.get('prefetch_metadata_iohits', 0)) +\ + int(arc_stats['prefetch_metadata_misses']) + prt_2('ARC prefetch metadata accesses:', f_perc(pm_total, all_accesses), + f_hits(pm_total)) + pm_todo = (('Prefetch metadata hits:', + arc_stats['prefetch_metadata_hits']), + ('Prefetch metadata I/O hits:', +- arc_stats['prefetch_metadata_iohits']), ++ arc_stats.get('prefetch_metadata_iohits', 0)), + ('Prefetch metadata misses:', + arc_stats['prefetch_metadata_misses'])) + for title, value in pm_todo: + prt_i2(title, f_perc(value, pm_total), f_hits(value)) + print() + +- all_prefetches = int(arc_stats['predictive_prefetch'])+\ +- int(arc_stats['prescient_prefetch']) ++ all_prefetches = int(arc_stats.get('predictive_prefetch', 0))+\ ++ int(arc_stats.get('prescient_prefetch', 0)) + prt_2('ARC predictive prefetches:', +- f_perc(arc_stats['predictive_prefetch'], all_prefetches), +- f_hits(arc_stats['predictive_prefetch'])) ++ f_perc(arc_stats.get('predictive_prefetch', 0), all_prefetches), ++ f_hits(arc_stats.get('predictive_prefetch', 0))) + prt_i2('Demand hits after predictive:', + f_perc(arc_stats['demand_hit_predictive_prefetch'], +- arc_stats['predictive_prefetch']), ++ arc_stats.get('predictive_prefetch', 0)), + f_hits(arc_stats['demand_hit_predictive_prefetch'])) + prt_i2('Demand I/O hits after predictive:', +- f_perc(arc_stats['demand_iohit_predictive_prefetch'], +- arc_stats['predictive_prefetch']), +- f_hits(arc_stats['demand_iohit_predictive_prefetch'])) +- never = int(arc_stats['predictive_prefetch']) -\ ++ f_perc(arc_stats.get('demand_iohit_predictive_prefetch', 0), ++ arc_stats.get('predictive_prefetch', 0)), ++ f_hits(arc_stats.get('demand_iohit_predictive_prefetch', 0))) ++ never = int(arc_stats.get('predictive_prefetch', 0)) -\ + int(arc_stats['demand_hit_predictive_prefetch']) -\ +- int(arc_stats['demand_iohit_predictive_prefetch']) ++ int(arc_stats.get('demand_iohit_predictive_prefetch', 0)) + prt_i2('Never demanded after predictive:', +- f_perc(never, arc_stats['predictive_prefetch']), ++ f_perc(never, arc_stats.get('predictive_prefetch', 0)), + f_hits(never)) + print() + + prt_2('ARC prescient prefetches:', +- f_perc(arc_stats['prescient_prefetch'], all_prefetches), +- f_hits(arc_stats['prescient_prefetch'])) ++ f_perc(arc_stats.get('prescient_prefetch', 0), all_prefetches), ++ f_hits(arc_stats.get('prescient_prefetch', 0))) + prt_i2('Demand hits after prescient:', + f_perc(arc_stats['demand_hit_prescient_prefetch'], +- arc_stats['prescient_prefetch']), ++ arc_stats.get('prescient_prefetch', 0)), + f_hits(arc_stats['demand_hit_prescient_prefetch'])) + prt_i2('Demand I/O hits after prescient:', +- f_perc(arc_stats['demand_iohit_prescient_prefetch'], +- arc_stats['prescient_prefetch']), +- f_hits(arc_stats['demand_iohit_prescient_prefetch'])) +- never = int(arc_stats['prescient_prefetch'])-\ ++ f_perc(arc_stats.get('demand_iohit_prescient_prefetch', 0), ++ arc_stats.get('prescient_prefetch', 0)), ++ f_hits(arc_stats.get('demand_iohit_prescient_prefetch', 0))) ++ never = int(arc_stats.get('prescient_prefetch', 0))-\ + int(arc_stats['demand_hit_prescient_prefetch'])-\ +- int(arc_stats['demand_iohit_prescient_prefetch']) ++ int(arc_stats.get('demand_iohit_prescient_prefetch', 0)) + prt_i2('Never demanded after prescient:', +- f_perc(never, arc_stats['prescient_prefetch']), ++ f_perc(never, arc_stats.get('prescient_prefetch', 0)), + f_hits(never)) + print() + +@@ -782,7 +782,7 @@ def section_archits(kstats_dict): + arc_stats['mfu_ghost_hits']), + ('Most recently used (MRU) ghost:', + arc_stats['mru_ghost_hits']), +- ('Uncached:', arc_stats['uncached_hits'])) ++ ('Uncached:', arc_stats.get('uncached_hits', 0))) + for title, value in cl_todo: + prt_i2(title, f_perc(value, all_accesses), f_hits(value)) + print() +@@ -794,26 +794,26 @@ def section_dmu(kstats_dict): + zfetch_stats = isolate_section('zfetchstats', kstats_dict) + + zfetch_access_total = int(zfetch_stats['hits']) +\ +- int(zfetch_stats['future']) + int(zfetch_stats['stride']) +\ +- int(zfetch_stats['past']) + int(zfetch_stats['misses']) ++ int(zfetch_stats.get('future', 0)) + int(zfetch_stats.get('stride', 0)) +\ ++ int(zfetch_stats.get('past', 0)) + int(zfetch_stats['misses']) + + prt_1('DMU predictive prefetcher calls:', f_hits(zfetch_access_total)) + prt_i2('Stream hits:', + f_perc(zfetch_stats['hits'], zfetch_access_total), + f_hits(zfetch_stats['hits'])) +- future = int(zfetch_stats['future']) + int(zfetch_stats['stride']) ++ future = int(zfetch_stats.get('future', 0)) + int(zfetch_stats.get('stride', 0)) + prt_i2('Hits ahead of stream:', f_perc(future, zfetch_access_total), + f_hits(future)) + prt_i2('Hits behind stream:', +- f_perc(zfetch_stats['past'], zfetch_access_total), +- f_hits(zfetch_stats['past'])) ++ f_perc(zfetch_stats.get('past', 0), zfetch_access_total), ++ f_hits(zfetch_stats.get('past', 0))) + prt_i2('Stream misses:', + f_perc(zfetch_stats['misses'], zfetch_access_total), + f_hits(zfetch_stats['misses'])) + prt_i2('Streams limit reached:', + f_perc(zfetch_stats['max_streams'], zfetch_stats['misses']), + f_hits(zfetch_stats['max_streams'])) +- prt_i1('Stream strides:', f_hits(zfetch_stats['stride'])) ++ prt_i1('Stream strides:', f_hits(zfetch_stats.get('stride', 0))) + prt_i1('Prefetches issued', f_hits(zfetch_stats['io_issued'])) + print() + +@@ -860,20 +860,20 @@ def section_l2arc(kstats_dict): + f_perc(arc_stats['l2_hdr_size'], arc_stats['l2_size']), + f_bytes(arc_stats['l2_hdr_size'])) + prt_i2('MFU allocated size:', +- f_perc(arc_stats['l2_mfu_asize'], arc_stats['l2_asize']), +- f_bytes(arc_stats['l2_mfu_asize'])) ++ f_perc(arc_stats.get('l2_mfu_asize', 0), arc_stats['l2_asize']), ++ f_bytes(arc_stats.get('l2_mfu_asize', 0))) # 2.0 module compat + prt_i2('MRU allocated size:', +- f_perc(arc_stats['l2_mru_asize'], arc_stats['l2_asize']), +- f_bytes(arc_stats['l2_mru_asize'])) ++ f_perc(arc_stats.get('l2_mru_asize', 0), arc_stats['l2_asize']), ++ f_bytes(arc_stats.get('l2_mru_asize', 0))) # 2.0 module compat + prt_i2('Prefetch allocated size:', +- f_perc(arc_stats['l2_prefetch_asize'], arc_stats['l2_asize']), +- f_bytes(arc_stats['l2_prefetch_asize'])) ++ f_perc(arc_stats.get('l2_prefetch_asize', 0), arc_stats['l2_asize']), ++ f_bytes(arc_stats.get('l2_prefetch_asize',0))) # 2.0 module compat + prt_i2('Data (buffer content) allocated size:', +- f_perc(arc_stats['l2_bufc_data_asize'], arc_stats['l2_asize']), +- f_bytes(arc_stats['l2_bufc_data_asize'])) ++ f_perc(arc_stats.get('l2_bufc_data_asize', 0), arc_stats['l2_asize']), ++ f_bytes(arc_stats.get('l2_bufc_data_asize', 0))) # 2.0 module compat + prt_i2('Metadata (buffer content) allocated size:', +- f_perc(arc_stats['l2_bufc_metadata_asize'], arc_stats['l2_asize']), +- f_bytes(arc_stats['l2_bufc_metadata_asize'])) ++ f_perc(arc_stats.get('l2_bufc_metadata_asize', 0), arc_stats['l2_asize']), ++ f_bytes(arc_stats.get('l2_bufc_metadata_asize', 0))) # 2.0 module compat + + print() + prt_1('L2ARC breakdown:', f_hits(l2_access_total)) +diff --git a/cmd/arcstat.in b/cmd/arcstat.in +index c4f10a1d6..bf47ec90e 100755 +--- a/cmd/arcstat.in ++++ b/cmd/arcstat.in +@@ -510,7 +510,7 @@ def calculate(): + v = dict() + v["time"] = time.strftime("%H:%M:%S", time.localtime()) + v["hits"] = d["hits"] // sint +- v["iohs"] = d["iohits"] // sint ++ v["iohs"] = d.get("iohits", 0) // sint + v["miss"] = d["misses"] // sint + v["read"] = v["hits"] + v["iohs"] + v["miss"] + v["hit%"] = 100 * v["hits"] // v["read"] if v["read"] > 0 else 0 +@@ -518,7 +518,7 @@ def calculate(): + v["miss%"] = 100 - v["hit%"] - v["ioh%"] if v["read"] > 0 else 0 + + v["dhit"] = (d["demand_data_hits"] + d["demand_metadata_hits"]) // sint +- v["dioh"] = (d["demand_data_iohits"] + d["demand_metadata_iohits"]) // sint ++ v["dioh"] = (d.get("demand_data_iohits", 0) + d.get("demand_metadata_iohits", 0)) // sint + v["dmis"] = (d["demand_data_misses"] + d["demand_metadata_misses"]) // sint + + v["dread"] = v["dhit"] + v["dioh"] + v["dmis"] +@@ -527,7 +527,7 @@ def calculate(): + v["dm%"] = 100 - v["dh%"] - v["di%"] if v["dread"] > 0 else 0 + + v["ddhit"] = d["demand_data_hits"] // sint +- v["ddioh"] = d["demand_data_iohits"] // sint ++ v["ddioh"] = d.get("demand_data_iohits", 0) // sint + v["ddmis"] = d["demand_data_misses"] // sint + + v["ddread"] = v["ddhit"] + v["ddioh"] + v["ddmis"] +@@ -536,7 +536,7 @@ def calculate(): + v["ddm%"] = 100 - v["ddh%"] - v["ddi%"] if v["ddread"] > 0 else 0 + + v["dmhit"] = d["demand_metadata_hits"] // sint +- v["dmioh"] = d["demand_metadata_iohits"] // sint ++ v["dmioh"] = d.get("demand_metadata_iohits", 0) // sint + v["dmmis"] = d["demand_metadata_misses"] // sint + + v["dmread"] = v["dmhit"] + v["dmioh"] + v["dmmis"] +@@ -545,8 +545,8 @@ def calculate(): + v["dmm%"] = 100 - v["dmh%"] - v["dmi%"] if v["dmread"] > 0 else 0 + + v["phit"] = (d["prefetch_data_hits"] + d["prefetch_metadata_hits"]) // sint +- v["pioh"] = (d["prefetch_data_iohits"] + +- d["prefetch_metadata_iohits"]) // sint ++ v["pioh"] = (d.get("prefetch_data_iohits", 0) + ++ d.get("prefetch_metadata_iohits", 0)) // sint + v["pmis"] = (d["prefetch_data_misses"] + + d["prefetch_metadata_misses"]) // sint + +@@ -556,7 +556,7 @@ def calculate(): + v["pm%"] = 100 - v["ph%"] - v["pi%"] if v["pread"] > 0 else 0 + + v["pdhit"] = d["prefetch_data_hits"] // sint +- v["pdioh"] = d["prefetch_data_iohits"] // sint ++ v["pdioh"] = d.get("prefetch_data_iohits", 0) // sint + v["pdmis"] = d["prefetch_data_misses"] // sint + + v["pdread"] = v["pdhit"] + v["pdioh"] + v["pdmis"] +@@ -565,7 +565,7 @@ def calculate(): + v["pdm%"] = 100 - v["pdh%"] - v["pdi%"] if v["pdread"] > 0 else 0 + + v["pmhit"] = d["prefetch_metadata_hits"] // sint +- v["pmioh"] = d["prefetch_metadata_iohits"] // sint ++ v["pmioh"] = d.get("prefetch_metadata_iohits", 0) // sint + v["pmmis"] = d["prefetch_metadata_misses"] // sint + + v["pmread"] = v["pmhit"] + v["pmioh"] + v["pmmis"] +@@ -575,8 +575,8 @@ def calculate(): + + v["mhit"] = (d["prefetch_metadata_hits"] + + d["demand_metadata_hits"]) // sint +- v["mioh"] = (d["prefetch_metadata_iohits"] + +- d["demand_metadata_iohits"]) // sint ++ v["mioh"] = (d.get("prefetch_metadata_iohits", 0) + ++ d.get("demand_metadata_iohits", 0)) // sint + v["mmis"] = (d["prefetch_metadata_misses"] + + d["demand_metadata_misses"]) // sint + +@@ -592,24 +592,24 @@ def calculate(): + v["mru"] = d["mru_hits"] // sint + v["mrug"] = d["mru_ghost_hits"] // sint + v["mfug"] = d["mfu_ghost_hits"] // sint +- v["unc"] = d["uncached_hits"] // sint ++ v["unc"] = d.get("uncached_hits", 0) // sint + v["eskip"] = d["evict_skip"] // sint + v["el2skip"] = d["evict_l2_skip"] // sint + v["el2cach"] = d["evict_l2_cached"] // sint + v["el2el"] = d["evict_l2_eligible"] // sint +- v["el2mfu"] = d["evict_l2_eligible_mfu"] // sint +- v["el2mru"] = d["evict_l2_eligible_mru"] // sint ++ v["el2mfu"] = d.get("evict_l2_eligible_mfu", 0) // sint ++ v["el2mru"] = d.get("evict_l2_eligible_mru", 0) // sint + v["el2inel"] = d["evict_l2_ineligible"] // sint + v["mtxmis"] = d["mutex_miss"] // sint +- v["ztotal"] = (d["zfetch_hits"] + d["zfetch_future"] + d["zfetch_stride"] + +- d["zfetch_past"] + d["zfetch_misses"]) // sint ++ v["ztotal"] = (d["zfetch_hits"] + d.get("zfetch_future", 0) + d.get("zfetch_stride", 0) + ++ d.get("zfetch_past", 0) + d["zfetch_misses"]) // sint + v["zhits"] = d["zfetch_hits"] // sint +- v["zahead"] = (d["zfetch_future"] + d["zfetch_stride"]) // sint +- v["zpast"] = d["zfetch_past"] // sint ++ v["zahead"] = (d.get("zfetch_future", 0) + d.get("zfetch_stride", 0)) // sint ++ v["zpast"] = d.get("zfetch_past", 0) // sint + v["zmisses"] = d["zfetch_misses"] // sint + v["zmax"] = d["zfetch_max_streams"] // sint +- v["zfuture"] = d["zfetch_future"] // sint +- v["zstride"] = d["zfetch_stride"] // sint ++ v["zfuture"] = d.get("zfetch_future", 0) // sint ++ v["zstride"] = d.get("zfetch_stride", 0) // sint + v["zissued"] = d["zfetch_io_issued"] // sint + v["zactive"] = d["zfetch_io_active"] // sint + +@@ -624,11 +624,11 @@ def calculate(): + v["l2size"] = cur["l2_size"] + v["l2bytes"] = d["l2_read_bytes"] // sint + +- v["l2pref"] = cur["l2_prefetch_asize"] +- v["l2mfu"] = cur["l2_mfu_asize"] +- v["l2mru"] = cur["l2_mru_asize"] +- v["l2data"] = cur["l2_bufc_data_asize"] +- v["l2meta"] = cur["l2_bufc_metadata_asize"] ++ v["l2pref"] = cur.get("l2_prefetch_asize", 0) ++ v["l2mfu"] = cur.get("l2_mfu_asize", 0) ++ v["l2mru"] = cur.get("l2_mru_asize", 0) ++ v["l2data"] = cur.get("l2_bufc_data_asize", 0) ++ v["l2meta"] = cur.get("l2_bufc_metadata_asize", 0) + v["l2pref%"] = 100 * v["l2pref"] // v["l2asize"] + v["l2mfu%"] = 100 * v["l2mfu"] // v["l2asize"] + v["l2mru%"] = 100 * v["l2mru"] // v["l2asize"] diff --git a/debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch b/debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch deleted file mode 100644 index 2e7c207d..00000000 --- a/debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch +++ /dev/null @@ -1,113 +0,0 @@ -From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 -From: Thomas Lamprecht -Date: Wed, 10 Nov 2021 09:29:47 +0100 -Subject: [PATCH] arc stat/summary: guard access to l2arc MFU/MRU stats - -commit 085321621e79a75bea41c2b6511da6ebfbf2ba0a added printing MFU -and MRU stats for 2.1 user space tools, but those keys are not -available in the 2.0 module. That means it may break the arcstat and -arc_summary tools after upgrade to 2.1 (user space), before a reboot -to the new 2.1 ZFS kernel-module happened, due to python raising a -KeyError on the dict access then. - -Move those two keys to a .get accessor with `0` as fallback, as it -should be better to show some possible wrong data for new stat-keys -than throwing an exception. - -Signed-off-by: Thomas Lamprecht - -also move l2_mfu_asize l2_mru_asize l2_prefetch_asize -l2_bufc_data_asize l2_bufc_metadata_asize to .get accessor -(these are only present with a cache device in the pool) -Signed-off-by: Stoiko Ivanov -Signed-off-by: Thomas Lamprecht ---- - cmd/arc_summary | 28 ++++++++++++++-------------- - cmd/arcstat.in | 14 +++++++------- - 2 files changed, 21 insertions(+), 21 deletions(-) - -diff --git a/cmd/arc_summary b/cmd/arc_summary -index 100fb1987..86b2260a1 100755 ---- a/cmd/arc_summary -+++ b/cmd/arc_summary -@@ -655,13 +655,13 @@ def section_arc(kstats_dict): - prt_i1('L2 cached evictions:', f_bytes(arc_stats['evict_l2_cached'])) - prt_i1('L2 eligible evictions:', f_bytes(arc_stats['evict_l2_eligible'])) - prt_i2('L2 eligible MFU evictions:', -- f_perc(arc_stats['evict_l2_eligible_mfu'], -+ f_perc(arc_stats.get('evict_l2_eligible_mfu', 0), # 2.0 module compat - arc_stats['evict_l2_eligible']), -- f_bytes(arc_stats['evict_l2_eligible_mfu'])) -+ f_bytes(arc_stats.get('evict_l2_eligible_mfu', 0))) - prt_i2('L2 eligible MRU evictions:', -- f_perc(arc_stats['evict_l2_eligible_mru'], -+ f_perc(arc_stats.get('evict_l2_eligible_mru', 0), # 2.0 module compat - arc_stats['evict_l2_eligible']), -- f_bytes(arc_stats['evict_l2_eligible_mru'])) -+ f_bytes(arc_stats.get('evict_l2_eligible_mru', 0))) - prt_i1('L2 ineligible evictions:', - f_bytes(arc_stats['evict_l2_ineligible'])) - print() -@@ -860,20 +860,20 @@ def section_l2arc(kstats_dict): - f_perc(arc_stats['l2_hdr_size'], arc_stats['l2_size']), - f_bytes(arc_stats['l2_hdr_size'])) - prt_i2('MFU allocated size:', -- f_perc(arc_stats['l2_mfu_asize'], arc_stats['l2_asize']), -- f_bytes(arc_stats['l2_mfu_asize'])) -+ f_perc(arc_stats.get('l2_mfu_asize', 0), arc_stats['l2_asize']), -+ f_bytes(arc_stats.get('l2_mfu_asize', 0))) # 2.0 module compat - prt_i2('MRU allocated size:', -- f_perc(arc_stats['l2_mru_asize'], arc_stats['l2_asize']), -- f_bytes(arc_stats['l2_mru_asize'])) -+ f_perc(arc_stats.get('l2_mru_asize', 0), arc_stats['l2_asize']), -+ f_bytes(arc_stats.get('l2_mru_asize', 0))) # 2.0 module compat - prt_i2('Prefetch allocated size:', -- f_perc(arc_stats['l2_prefetch_asize'], arc_stats['l2_asize']), -- f_bytes(arc_stats['l2_prefetch_asize'])) -+ f_perc(arc_stats.get('l2_prefetch_asize', 0), arc_stats['l2_asize']), -+ f_bytes(arc_stats.get('l2_prefetch_asize',0))) # 2.0 module compat - prt_i2('Data (buffer content) allocated size:', -- f_perc(arc_stats['l2_bufc_data_asize'], arc_stats['l2_asize']), -- f_bytes(arc_stats['l2_bufc_data_asize'])) -+ f_perc(arc_stats.get('l2_bufc_data_asize', 0), arc_stats['l2_asize']), -+ f_bytes(arc_stats.get('l2_bufc_data_asize', 0))) # 2.0 module compat - prt_i2('Metadata (buffer content) allocated size:', -- f_perc(arc_stats['l2_bufc_metadata_asize'], arc_stats['l2_asize']), -- f_bytes(arc_stats['l2_bufc_metadata_asize'])) -+ f_perc(arc_stats.get('l2_bufc_metadata_asize', 0), arc_stats['l2_asize']), -+ f_bytes(arc_stats.get('l2_bufc_metadata_asize', 0))) # 2.0 module compat - - print() - prt_1('L2ARC breakdown:', f_hits(l2_access_total)) -diff --git a/cmd/arcstat.in b/cmd/arcstat.in -index c4f10a1d6..c570dca88 100755 ---- a/cmd/arcstat.in -+++ b/cmd/arcstat.in -@@ -597,8 +597,8 @@ def calculate(): - v["el2skip"] = d["evict_l2_skip"] // sint - v["el2cach"] = d["evict_l2_cached"] // sint - v["el2el"] = d["evict_l2_eligible"] // sint -- v["el2mfu"] = d["evict_l2_eligible_mfu"] // sint -- v["el2mru"] = d["evict_l2_eligible_mru"] // sint -+ v["el2mfu"] = d.get("evict_l2_eligible_mfu", 0) // sint -+ v["el2mru"] = d.get("evict_l2_eligible_mru", 0) // sint - v["el2inel"] = d["evict_l2_ineligible"] // sint - v["mtxmis"] = d["mutex_miss"] // sint - v["ztotal"] = (d["zfetch_hits"] + d["zfetch_future"] + d["zfetch_stride"] + -@@ -624,11 +624,11 @@ def calculate(): - v["l2size"] = cur["l2_size"] - v["l2bytes"] = d["l2_read_bytes"] // sint - -- v["l2pref"] = cur["l2_prefetch_asize"] -- v["l2mfu"] = cur["l2_mfu_asize"] -- v["l2mru"] = cur["l2_mru_asize"] -- v["l2data"] = cur["l2_bufc_data_asize"] -- v["l2meta"] = cur["l2_bufc_metadata_asize"] -+ v["l2pref"] = cur.get("l2_prefetch_asize", 0) -+ v["l2mfu"] = cur.get("l2_mfu_asize", 0) -+ v["l2mru"] = cur.get("l2_mru_asize", 0) -+ v["l2data"] = cur.get("l2_bufc_data_asize", 0) -+ v["l2meta"] = cur.get("l2_bufc_metadata_asize", 0) - v["l2pref%"] = 100 * v["l2pref"] // v["l2asize"] - v["l2mfu%"] = 100 * v["l2mfu"] // v["l2asize"] - v["l2mru%"] = 100 * v["l2mru"] // v["l2asize"] diff --git a/debian/patches/series b/debian/patches/series index 35f81d13..229027ff 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -6,6 +6,6 @@ 0006-dont-symlink-zed-scripts.patch 0007-Add-systemd-unit-for-importing-specific-pools.patch 0008-Patch-move-manpage-arcstat-1-to-arcstat-8.patch -0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch +0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch 0010-Fix-nfs_truncate_shares-without-etc-exports.d.patch 0011-zpool-status-tighten-bounds-for-noalloc-stat-availab.patch -- 2.39.2 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel