From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 72000C0BAB for ; Fri, 12 Jan 2024 11:28:26 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 53B6B1FB55 for ; Fri, 12 Jan 2024 11:28:26 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 12 Jan 2024 11:28:25 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 10A354536F for ; Fri, 12 Jan 2024 11:28:25 +0100 (CET) Date: Fri, 12 Jan 2024 11:28:18 +0100 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox VE development discussion References: <20240111165826.804669-1-f.weber@proxmox.com> <1705047462.e8upimiin2.astroid@yuna.none> <5970d7e6-ec71-4484-9f59-339f8c1aadcd@proxmox.com> In-Reply-To: <5970d7e6-ec71-4484-9f59-339f8c1aadcd@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.16.0 (https://github.com/astroidmail/astroid) Message-Id: <1705055166.gmeldwg7ib.astroid@yuna.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.065 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [lvmplugin.pm] Subject: Re: [pve-devel] [PATCH storage] lvm: avoid warning due to human-readable text in vgs output X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2024 10:28:26 -0000 On January 12, 2024 11:06 am, Friedrich Weber wrote: > On 12/01/2024 10:22, Fabian Gr=C3=BCnbichler wrote: >>> --- a/src/PVE/Storage/LVMPlugin.pm >>> +++ b/src/PVE/Storage/LVMPlugin.pm >>> @@ -130,6 +130,9 @@ sub lvm_vgs { >>> =20 >>> my ($name, $size, $free, $lvcount, $pvname, $pvsize, $pvfree) =3D= split (':', $line); >>> =20 >>> + # skip human-readable messages that vgs occasionally prints to st= dout >>> + return if !defined($size); >>> + >>=20 >> we might want to either log this message (like anything printed to >> STDERR), so that the admin at least can notice something weird is going >> on, >=20 > The vgs message is printed to stdout, so we could do something like >=20 > warn $line if !defined($size); >=20 > ? yep, that would be an option (warn+return ;)) [..] >> AFAICT this message only gets printed if the archives grew very big, and >> the backup file does not exist? at least for me using your reproducer, >> it's only printed once, and I have to rename rename rm again afterwards >> to get it to show up again, which would mean it's not too bad to log it >> (as long as it doesn't confuse our code). >=20 > For the user in enterprise support I mentioned in the commit message, > the warnings were logged to the journal by pvestatd every few seconds > (without any manual deletion of backup files or similar). I'd need to > look at the source again to be sure, but IIRC the message is also > printed if the backup exists but is outdated (and the archives are very > big). And because LVM got confused by the duplicate VG names, this > situation seemed to occur every few seconds. >=20 > Another complication I forgot about: For that user, /etc/lvm/archive had > 800000 files amounting to >1GiB, which also slowed down `vgs -o vg_name` > considerably (to >10s), presumably because `vgs -o vg_name` read all > those files. But unexpectedly, as soon as `-o` included `pv_name` the > command was fast again, presumably because it does not do the reads. So > I was considering to modify `sub lvm_vgs` to always include `-o pv_name` > in the command (not only if $includepvs is true), but was unsure if the > edge case warranted this (somewhat weird) workaround. that sounds weird ;) >> the `lvmscan` endpoint also picks up the line btw, which means it ends >> up being included in the "VG" selector on the web UI when adding an LVM >> storage ;) >=20 > Hah, fun :) >=20 > By the way, the message also causes `vgs` to print invalid JSON: >=20 > # rm -f /etc/lvm/backup/spam ; vgs -o vg_name -q --reportformat json > 2>/dev/null > { > "report": [ > Consider pruning spam VG archive with more then 8 MiB in 8305 files > (check archiving is needed in lvm.conf). > { > "vg": [ > {"vg_name":"pve"}, > {"vg_name":"spam"}, > {"vg_name":"testvg"} > ] > } > ] > } >=20 > Dominik suggested that this very much looks like an LVM bug, so I'll > check whether it still happens on latest upstream LVM and, if yes, file > a bug report there. yes, that definitely sounds like a bug. potentially they'd also be open to switching the log level/target so that it ends up on STDERR at least..