From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8725A60202 for ; Tue, 1 Sep 2020 13:06:56 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 73CF9842C for ; Tue, 1 Sep 2020 13:06:56 +0200 (CEST) Received: from mailgate.itronic.at (mailgate.itronic.at [85.124.85.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 3E6AD841B for ; Tue, 1 Sep 2020 13:06:55 +0200 (CEST) Received: from [192.168.0.102] (ns1office.innonet.at [213.208.145.8]) (Authenticated sender: lord@itronic.at) by mailgate.itronic.at (Postfix) with ESMTPSA id 4BgkmG3V7Tzxx for ; Tue, 1 Sep 2020 13:06:54 +0200 (CEST) To: pbs-devel@lists.proxmox.com References: <82142373-8994-8d65-5cc8-6944873bb33e@itronic.at> <1598873753.xg9lm3vlx4.astroid@nora.none> <1598879286.etmszx6yka.astroid@nora.none> From: Harald Leithner Autocrypt: addr=leithner@itronic.at; keydata= mQGNBFq+KcwBDAD0GKdUdEMyHz/tT/bU7VB4roEu5UB5DkpwLqu8mTm7LOxuyv0q6Iyrstbf OaPaYqS94T4JDFnfjM+ZL3FqGO9TMWEjjVqOhEfdp/xy3DY6wA8ZrWTOGX4RUcy2LMm+1bNc cOHwPoVlSXrioIml91j4ugMKMvGP49zPf2z08YxuW0pujUIcwcifHgYfVJdzUOjTGtVMkb37 6sdxmsM1ScYv10faVoy/UwhgtkcEYBVFKLnfhhUndIzwNDBmYb1gV1V0loh9mtRAu35DOgX8 CeY4Aa8KbJS9DuTnzngTAJBNLuMDTVr1Zh0ENntnNd5bGG3Yddt9MJzeGHj6/Pte0t9AzJ8L s/t9YtI//CTR5FXcySuzVanEaNpG6i8jeCx5lQlZRYh740Yqy1FhmKD6x52kfOEt1rpL+vxv rCL7YCkUz3P4zUtXSSvnv/klvj7y1DmjenHOJNYpLbslkMU0JwSwxo+ygBMRdeKF+Jj+mIIc /bGxjt0vEEmqAdRBxG+rVaMAEQEAAbQlSGFyYWxkIExlaXRobmVyIDxsZWl0aG5lckBpdHJv bmljLmF0PokBzgQTAQgAOBYhBHPLS5Ya2+Gr3k3yYkc7k1xJ6hsIBQJavinMAhsjBQsJCAcC BhUKCQgLAgQWAgMBAh4BAheAAAoJEEc7k1xJ6hsIqVUMANx/9QEJTpS1Xxb8SJITeXX4+U2i T7WDINr9WrM52ifj9SX8AGp75VAhrTzHJ+wkQUtyUOEcz98meuWkQsYyx8/QqzZa5TSp+wbj 27LsMwT6fQMp3HXtuWW4baPCiZfSTd6vjwewO+lSFIkCzISzudUpw5RNAjfAdTxPGUw1ubv+ 84ZP19G+cru4cpiHcU2TyhKTJaGL6GwjSO3y5GYCREPl4vkW0I11DfHlMemjyLOON2I1J7u6 aDf0kkufiaQFfn4IrxAk+pI7m9Q6CfqksIFA5mhekJYfoFpxp4c9zlhjDzOh6meTTPHx2V0q OQiaIZtlbKKo0BmISdbXUBOQH9cfksyf8T/ngIdXZguDVNP+PkNcz+D2rWsAU1ttzDbqZFfS Bxe5maoGZ17xRRTjbGkkOLgz4QR1ya0x84Pn6oF4yMa2UC7dTGhRLDTlNeSHKKvogLn46NfS 4/FkQTltcGVlJBOZIZyaOhQqwHQRX4odrTMipT9Qsa4PaAyoX8MPHrkBjQRavinMAQwAqmzh A+EPb3L8aTl2sml3j0TOPLmiPiOOA52Q4pSJssC6rTmUoNRh/Wy4uR9VOmq2wxxAZJgJUXb2 se0EjJ8o0fNWamjCvOB/80wc2WU0qB90HHKBZU8CKTdyB1mu5YdVg7UX3Y8D0S+31PThkgrE 9TkeoDAG/yDKB3Z347sOtUV8i6ahVIj7vlim/fu1ide7OWV64r6ug2t0264iQlmIINGu8W4Z uK6AsvCRcqutN11MSaWy4jZXeMWGoBYzBol6n3kC7dSsMHROSm0+PW65UdCgfWMsOFymcwlL 9Syvft/yyyhL4WqWyyxFQvmgMcx3ZCJO+CPrvpQ1w/JtKr4NvvCxZvXPl/bUSTXodak2TVA3 9yYpHN0meK15sgSnasrckXvaMm1olHeNRZTgl76W4YuwZnmHIDXRtnqaJ5xkogKxrHaUp7rQ dItkv0BAXyjEXXXJi3FgaksWiO/Vvpk3yf/qVOldAWSEkWiJOKoZtY+g5su2B5r15gVED4RM k2DpABEBAAGJAbYEGAEIACAWIQRzy0uWGtvhq95N8mJHO5NcSeobCAUCWr4pzAIbDAAKCRBH O5NcSeobCP4yC/9dZFHvkkmoqNFUj/C2lv0cS+TdyFO39hE5cIPeG1HlEo5S3Ydo//ZmFQ7X RilkDVonvHmLLmTBi977qO9/k19ndbQsbfcgq5Aj7Olz4+5BnxHU5WlQstDPEfcDx9IK/LUJ bxDrmydeMpWmxCNTynwV8L/FiZFLiB9etZraN6QsUFjQXlvLSVtQuC2tB9+vtbAonKzIhuk7 48VswFKwESlag4SjW7fRtlaaEsdMvNrTTpj6ubok5A8RJV0UvvnlWnwlDAqroimmfTFyopB1 zJHx7PtTOysuiOqvRvp6g/pJHHDM9Bz0qCiPxEes6pVi8OmApG405R243D1iWsf0MnfCcL+X O6S/gVnpk5ZdNJfqOKBfxw1K2kofyi9lvOdBFPnAgS8GZGkxgE5pcmLO2ga1nzaJlxPr6inK 7+1eUm1RKEh1BZHvHCCunjv8qlKh0NAWxthlPbjJo/5miaaBczf2Dp1yS8I2GF2YXK/Vz8dS Zj0Wib+6xpu+4pK4VrpGkn4= Organization: ITronic Message-ID: <58a04d88-2846-1413-7c9a-b059aa516a2f@itronic.at> Date: Tue, 1 Sep 2020 13:06:54 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <1598879286.etmszx6yka.astroid@nora.none> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="QqbgjPYdVbQ7yN6YzjTgbgGziI6uO6hSD" X-SPAM-LEVEL: Spam detection results: 0 AWL 1.013 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -2.13 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com, itronic.at] Subject: Re: [pbs-devel] Some problems with fullstorage and cleanup X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Sep 2020 11:06:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --QqbgjPYdVbQ7yN6YzjTgbgGziI6uO6hSD Content-Type: multipart/mixed; boundary="PNR6froPc8YDxfdARqn617csT4lgWjvYG" --PNR6froPc8YDxfdARqn617csT4lgWjvYG Content-Type: text/plain; charset=utf-8 Content-Language: de-AT Content-Transfer-Encoding: quoted-printable Hi, after sometime the pbs was able to recover the diskspace of the deleted snapshots. I get some error reports after the backup but I think thats because the pve has not the latest backup client and tries root and not root@pam. Anyway thx for the help and I think a better "disk full handling" would be useful ;-) Harald On 2020-08-31 15:10, Fabian Gr=C3=BCnbichler wrote: > On August 31, 2020 2:32 pm, Harald Leithner wrote: >> >> >> On 2020-08-31 13:39, Fabian Gr=C3=BCnbichler wrote: >>> On August 31, 2020 1:14 pm, Harald Leithner wrote: >>>> Hi, >>>> >>>> my test stroage run out of diskspace. >>>> >>>> This happens at Version 0.8-11 >>>> >>>> I tried to forget the olders snapshots but this doesn't change the d= isk >>>> usage. After this I started a manual GC (I assume it's garbage >>>> collection). It failed after phase one with an error message. I didn= 't >>>> copied the error message... (I miss the log panel in the gui). >>>> >>>> I also configured GC and Prune Schedules at the same time and told t= he >>>> vm that backups to the pbs to keep only 20 copies. >>>> >>>> After a while I came back to the gui and has only 3 snapshots left a= nd >>>> one backup in progress (Thats maybe correct because of 1 yearly 1 >>>> monthly 1 daily and 2 last). >>>> The Statistics Tab still says 100% usage and "zfs list" lists the sa= me >>>> usage. >>>> >>>> Starting now a manual GC ends in the error message: >>>> >>>> unable to parse active worker status >>>> 'UPID:backup-erlach3-test:00000400:0000036A:00000000:5F4CD753:termpr= oxy::root:' >>>> - not a valid user id >>> >>> this was a known issue that should be fixed on upgrade to 0.8.11-1. c= an=20 >>> you run 'grep termproxy /var/log/apt/term.log' on the PBS server? >>> >> >> the only entry is "Fixing up termproxy user id in task log..." >> btw. my first version was 0.8.9-1 not 0.8.11 I upgraded later to this >> version >> >>> you can fixup the task index by running the sed command from=20 >>> >>> /var/lib/dpkg/info/proxmox-backup-server.postinst >>> >>> which replaces the invalid user 'root' with the correct 'root@pam' >>> >> >> ok after running the sed command manually the GC works again. >> >> but is complains about no diskspace: >> >> 2020-08-31T14:29:16+02:00: WARN: warning: unable to access chunk >> 135e565dc79f80d3a9980688bfe161409bf229fb4d11ab7290b5b1e58b27bc63, >> required by "/test3/vm/3011/2020-08-30T22:00:02Z/drive-scsi1.img.fidx"= - >> update atime failed for chunk >> "/test3/.chunks/135e/135e565dc79f80d3a9980688bfe161409bf229fb4d11ab729= 0b5b1e58b27bc63" >> - ENOSPC: No space left on device >=20 > if you don't have enough space to touch a chunk, that is rather bad..=20 > you can attempt to free up some more space by deleting backup metadata = > of snapshots you no longer needed, either by 'rm'-ing the directory tha= t=20 > represents them, or by using 'forget' on the GUI if that works.. >=20 > what does 'df -m /test3' report? and/or the equivalent command for=20 > whatever storage the datastore is on (e.g., zfs list -o space=20 > path/to/dataset). >=20 >=20 > _______________________________________________ > pbs-devel mailing list > pbs-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel >=20 --=20 ITronic Harald Leithner Wiedner Hauptstra=C3=9Fe 120/5.1, 1050 Wien, Austria Tel: +43-1-545 0 604 Mobil: +43-699-123 78 4 78 Mail: leithner@itronic.at | itronic.at --PNR6froPc8YDxfdARqn617csT4lgWjvYG-- --QqbgjPYdVbQ7yN6YzjTgbgGziI6uO6hSD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAEBCAAdFiEEc8tLlhrb4aveTfJiRzuTXEnqGwgFAl9OK04ACgkQRzuTXEnq GwgSpQwAsQ2h6pj9o+YF91Y8hyBAPVWCGg7A7dAHLQjsVKvSCSA9Z1GepfJFN80d NhFy8aLulDNckjsfjWJrZ7ugkHv/gK7OB2szTEBtpR2mjQ7T4uTasEI6Uolttj8r RuCLwBiBdAeRy6jtycwtXdQrYWs0qOwMLMGZcMvYv/soW89tfv218FBQmwY96oCQ gx+QWfxpYuQPUPcSEKWYVEkqBkNUSZl52L07vhzgQ0xBYZ3lisIpQPPf9+J3SreV tc8f0SJOOUwxrH5MZgm0mVMwXbQcLPsB2z27cO4oLgH/my0oG/ohviMgD2/w639U 1NLBD5ikToZQpUNsSY6FLyITaYpEDgPfDNZ2ogzMGC5q7EQeTtE3imZBF5JqOJkM Csprrva9noLkbfrdUoLx3vdg14aKkryCcPJBsCgP6mQb8BgYr/ObLZiZpDaftjaE MGu5XyUFrQCA4RH65p9O8R83dlJVqwFBfTjwK0reJnqGKgEGfsOk+6JT8iQaegsP /vHOBr2A =Psyi -----END PGP SIGNATURE----- --QqbgjPYdVbQ7yN6YzjTgbgGziI6uO6hSD--