From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 4EFBA72ED5 for ; Mon, 5 Jul 2021 09:18:11 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 3E1961B70E for ; Mon, 5 Jul 2021 09:18:11 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 416AB1B702 for ; Mon, 5 Jul 2021 09:18:10 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 1050B406D7 for ; Mon, 5 Jul 2021 09:18:10 +0200 (CEST) Date: Mon, 05 Jul 2021 09:18:01 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox Backup Server development discussion , Thomas Lamprecht References: <20210630155759.1894155-1-s.reiter@proxmox.com> <1625229020.brdiyhyox3.astroid@nora.none> In-Reply-To: MIME-Version: 1.0 User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid) Message-Id: <1625468662.buwudn2eyd.astroid@nora.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.424 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LOTSOFHASH 0.25 Emails with lots of hash-like gibberish SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH 0/5] Add LVM (thin) support for single file restore X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 07:18:11 -0000 On July 5, 2021 8:26 am, Thomas Lamprecht wrote: > On 02.07.21 14:39, Fabian Gr=C3=BCnbichler wrote: >> Reviewed-By: Fabian Gr=C3=BCnbichler >>=20 >> requires a Breaks on the old restore image (else the restore daemon=20 >> crashes because of missing lock/LVM support). >>=20 >> some potential for follow-ups/issues I encountered with a not-so-fast=20 >> PBS instance and bigger VMs: >> - increase VM start timeout (currently 12s?), as the initial scan of=20 >> disks seems to block responding to the startup status request, and=20 >> scanning all VGs/LVs can take quite a lot longer than that -=20 >> alternatively, fix this blockage and/or consider VM started sooner? >=20 > I do not think that is required, rather, did it work at all for you? yeah, but see below > As here it did not, and that was rather due to a recently introduced side= -effect by the > Memcom(pare) module[0], which is instantiated deep in the REST handler fo= r the cached user > handling and expects to be able to create a file in `/run/proxmox-backup`= . As that directory > did not exists in the restore image environment, all requests to the REST= server instance > there just got a 400 Bad Request: >=20 >> proxmox_backup::server::rest] GET /api2/json/status: 400 Bad Request: [c= lient 0.0.0.0:807] request failed >=20 > Which is stupid in two ways: > * in most cases this rather should be denoted as 50X HTTP error, as it's = produced > by a general error result in the server, user and param verification al= ready create > correct 40X errors, and API calls that want to denote fault on the user= request side > should be able to do so too, so remaining stuff is quite likely a 500 e= rror - making > it easier to search for. > * We did not set the log extension for the general error, only returned i= t as response > body, so the real error got lost in any access log. >=20 > The latter was fixed[1], but I did not care to much, nor had the time to = look into details > of the first one, just noting for the record. >=20 >=20 > Anyway, the symptoms of the earlier missing, now created[2], proxmox-back= up run-dir where > a "VM start timeout", as the any API call failed the status one used for = detecting a > finished start failed to -> timeout. >=20 > In any way, neither of you did actually test this on master, Stefan send = it the day > Dietmar applied the Memcom stuff, but it was still done so 9h before this= series was > send, meh... I did test it on master, ran into the issue including seeing the 400 log=20 entries, but incorrectly attributed them to "VM not yet ready". I then=20 tried with stock packages from the repo to rule out a general issue on=20 my system (worked) and then increased the timeout and the problem=20 disappeared, but the increased timeout was in a worktree that indeed had=20 Dietmar's openid patches reverted (it was the one I originally started=20 the review on before fixing the openid linkage issues, when this series and master were incompatible). in Stefan's defense - he did write that his series was developed with=20 the openid patches reverted as he was blocked on the linkage issue, so=20 in his testing the problem could/did not arise. so this one's on me -=20 sorry. > [0]: https://git.proxmox.com/?p=3Dproxmox-backup.git;a=3Dcommitdiff;h=3Df= da19dcc6f41b2537896b2e874be1c0cfe71d5ea;hp=3Dcd975e5787a3a9ace56e4aea0acc23= 64c283f438 > [1]: https://git.proxmox.com/?p=3Dproxmox-backup.git;a=3Dcommitdiff;h=3Df= 4d371d2d2c5d79352e9681b5d942627e26c6fd4;hp=3D835d0e5dd3d69886f0d6bfdd4192a4= a8c81af395 > [2]: https://git.proxmox.com/?p=3Dproxmox-backup.git;a=3Dcommitdiff;h=3D3= 3d7292f29bedfd7570bebcef2a5d6943ebe8cbb;hp=3Df4d371d2d2c5d79352e9681b5d9426= 27e26c6fd4 >=20 >> - filter out "known unreadable" LVs, like Ceph (bluestore) OSDs >> - investigate '--sysinit' option for lvchange/vgchange (seems=20 >> appropriate for what we are doing inside the VM ;)) >> - investigate '-K' to allow looking at "non-auto-activatable" LVs >>=20 >> and as discussed off-list, I think we should switch to=20 >> build-dependencies and copying binaries+needed libs in=20 >> build-initramfs.sh instead of downloading .deb packages and extracting=20 >> them only to remove part of their contents again - but that is firmly in= =20 >> "improve/cleanup build" territory, so lower priority than the above. >=20 >=20