From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id DBAF2879CF for ; Mon, 3 Jan 2022 13:53:29 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id C5BDD11C98 for ; Mon, 3 Jan 2022 13:52:59 +0100 (CET) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70059.outbound.protection.outlook.com [40.107.7.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id C1B1F11C8F for ; Mon, 3 Jan 2022 13:52:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inettGmbH.onmicrosoft.com; s=selector2-inettGmbH-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Alr2qaw4A8yvXxhtcDFdBpPFqrpB+PsMbuvL3jt5QnI=; b=SIk5Uqs+ESBADt+wA5FkFilrpl7/H38Y4DsmWdjexIoiSAASsNqF+4OAUZ3HNFig1WT9XJoKqcKeZv0S7EPg52byz/r5N9zyuUjhU2qVJQ+TtZrof9tR4KriHolxj46MoSkXMr7LVJh2fKLA4Ue+IeTq2F9HFB+/tksLdJpTtzo= Received: from DB6PR0501CA0012.eurprd05.prod.outlook.com (2603:10a6:4:8f::22) by PAXPR05MB8905.eurprd05.prod.outlook.com (2603:10a6:102:2bd::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.15; Mon, 3 Jan 2022 12:38:15 +0000 Received: from DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:8f:cafe::9d) by DB6PR0501CA0012.outlook.office365.com (2603:10a6:4:8f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.14 via Frontend Transport; Mon, 3 Jan 2022 12:38:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 20.93.157.195) smtp.mailfrom=inett.de; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=inett.de; Received-SPF: Pass (protection.outlook.com: domain of inett.de designates 20.93.157.195 as permitted sender) receiver=protection.outlook.com; client-ip=20.93.157.195; helo=westeu11-emailsignatures-cloud.codetwo.com; Received: from westeu11-emailsignatures-cloud.codetwo.com (20.93.157.195) by DB5EUR03FT019.mail.protection.outlook.com (10.152.20.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.14 via Frontend Transport; Mon, 3 Jan 2022 12:38:15 +0000 Received: from EUR02-HE1-obe.outbound.protection.outlook.com (104.47.5.57) by westeu11-emailsignatures-cloud.codetwo.com with CodeTwo SMTP Server (TLS12) via SMTP; Mon, 03 Jan 2022 12:38:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=giVoNNVGRNaItszWAxpC1hlrh/9hCLfBVleFJdp5K7LBOnCrwXP2SL61r8p4Xb9aVAo+c5289u/rPprcboKRm3Diwt+qZlZlGT/PG475Ta2ODusUs4CuKX+7jZ0x4XBiBXuAQjAhX0UNSrIL6wX5ez7QHA9bF+BdWgXviB++RQSsFr36LT9oh+FGmyw/FDvP8ahiozONxysp3USAwpiuxWz85rcP7d8Jhg9qoDBN+IeDhDs9A3VxPj4QYbvuEdyx6F4f/O4xNeT2rYuyF24Vg0KDWG+4Ql88wQIfsCOQedmlJgZ01G7oqhWFYVs1bLXKPqP5TyTSPPcrpON2UtlgLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bHhrUq6QWWoqroa5muz+OcZFWSo1DsVymIzVs/TCLsk=; b=lPiaN5XQZDtpBZu+f1gjcA0dOQ9w0LBD7vUI389UngdHiLpa/9ITXAe9X4XY1LPyEp2wpSpkGvcwFqbOjyoRDXtgXA3bUBsMB2eEfnF/l9Kdb0PTSnnTce1nkvb8hqHnDJ++dl4YjGseJhVq9CP6w2tLFE5IQz6kIhnOVi6imlKqimDWvfITcUHKHO4eqI4QsWpIhVdLybdShdZ9WudZaQZOu9wwKHvXYiuom5+ERVoDhKJV4AcypdfBbKYt8PXyLps1yr8LnqpnnoYQdwF0GyL05SEgZpaUaSIT6PxzRbX0Kaeip2c8vKwZMAvM/Ia+4b6IhQS3o6M3ZuKBbPD8Vg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=inett.de; dmarc=pass action=none header.from=inett.de; dkim=pass header.d=inett.de; arc=none Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=inett.de; Received: from AM0PR05MB5764.eurprd05.prod.outlook.com (2603:10a6:208:11c::31) by AM8PR05MB8082.eurprd05.prod.outlook.com (2603:10a6:20b:354::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.13; Mon, 3 Jan 2022 12:38:12 +0000 Received: from AM0PR05MB5764.eurprd05.prod.outlook.com ([fe80::25f8:894b:4716:a232]) by AM0PR05MB5764.eurprd05.prod.outlook.com ([fe80::25f8:894b:4716:a232%3]) with mapi id 15.20.4844.016; Mon, 3 Jan 2022 12:38:12 +0000 Date: Mon, 3 Jan 2022 13:38:10 +0100 From: Maximilian Hill To: Proxmox Backup Server development discussion Message-ID: References: <17A444C0-1F31-42AA-81FE-A95CE4B3291E@tuxis.nl> <718D0AF11703FA4C85B0535448A056100381A40015@SCOM4.directique.net> Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/2.1.4 (93ef06ae) (2021-12-11) X-ClientProxiedBy: AS9PR04CA0041.eurprd04.prod.outlook.com (2603:10a6:20b:46a::30) To AM0PR05MB5764.eurprd05.prod.outlook.com (2603:10a6:208:11c::31) MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 37886d76-8233-4faf-47b5-08d9ceb5e9d5 X-MS-TrafficTypeDiagnostic: AM8PR05MB8082:EE_|DB5EUR03FT019:EE_|PAXPR05MB8905:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:962;OLM:962; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: E607TWdOEzx9CJVuABIxgWNHi6mxL98NffqMQlBO9xiANJqwCUp3z4hJAGmMz0FSAX1gcnyjaakbKstSJzZKbGfWDUCq4SS5xhiFsa0rM7X/03IsXVn3qu5n4MjnDl7Hht7FZEouN8WAmcbJXz9V2y5zZnegepn7WmhUeba8cyG/TOoZHh00QIE/pIx0EhwcuoIcjX4hUQzCQ0GkA9j/Gdfuv9DJ11WuzdMecsFgbLfMZAGqN/Dz94vyNX+3QsjE7hbXCehSV1jL90NbhmbLbwRUx6k8oB7it5H8SFapY9qWyuhjTC7vkq+FB/uN5jOT+edWCZCjrL13XVXheK4WK1zHNWQ3rGApvWDkgV/hQAy5ts3/7A7z2MUNS4IGOzFZVICleNryICet/hAint5UIJ7ez/Lewt4PR0e0vo9o6aFC7B/Pis7s7oGuwpNGA7Bpe30IFP2n6+43yeh2jT+6RIAxCm4wgs7xUYar/qvHLb1txiuuihHWhpoyQpUVVfXjuuJmhvDj+bBCS1C/9ImBMuCEhjPHO6SKcG4QNVFWyasg2T03kGXf1dpiG9nOsG8Iu+ZIUdL/zrmZOvbHFdKYHiK03l4LtcIK/ttLoC5m0J1xrcPRT5sDNFrLeOnKlmzDq+x2YoI+2wky3jXHrIxMdjhnF3zRDTjAAZPdQ3Rq2goOIExy0TKC//SjsOXd4uYni7JxOML5wqxwWm+h49NajVy9N9OIxHIQPv9x70il5wA= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR05MB5764.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFS:(346002)(366004)(396003)(39830400003)(136003)(376002)(6916009)(2616005)(5660300002)(316002)(86362001)(6506007)(38100700002)(83380400001)(8676002)(66556008)(6512007)(186003)(53546011)(66476007)(36756003)(508600001)(6486002)(8936002)(66946007)(2906002)(52116002)(966005); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR05MB8082 X-CodeTwo-MessageID: 29bae8f1-05d6-4ff8-b839-4e2791a7c5d8.20220103123814@westeu11-emailsignatures-cloud.codetwo.com X-CodeTwoProcessed: true X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: aabebe68-220b-4b78-89d5-08d9ceb5e7c7 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: auF+stQThVBRvHKbbyT0gNVJVGvYxt6MwWHcw0gTNIF/wRNQy13ynZ/R7IOtoDZGjbFcyPppfyPCrGxZ7MFLTl5+YuzOGbLOvED4d5E/xb/94afCML1bqDUTQ/cGstK/+t20TuL64WITyDI3Oq7EYBk9RqnNKsCxSLjTl7nE/DMxNcYjpamU+gIYHiXjOQjMmFNHhqBsuAK2RasgkyFU6ahU7pamx07V0ZD+7FgUJUCwZrKfhH3gOocioOOmB4o2rx6W/FuYtScnqG4zAalYcox23ersPl+cyUBJQmsw7N7yyGu3q+SeU2me+2c5J5fZzggNC08uSasQm5huf10E2vtCu7vC9b2Galsi/mLZP+olCPPa3MCuCzhr2jv6D0xmfL/Si4FsGqQqMeawsbS0FPDu+/xmoXcpBB5lLY18kknzIcjAyckbY4NIoeHYy+7uvA5dDsvOJXYnnQvRaC8QNP205mNUpj4iq2wUSJ2luBdXGiolIHvqyC8slzyC+OLwEkc/PypFXloy4e9pnnYLWTVeznP+0pKZ5OPNVINZvyathAPT/a1jHS/5tx2ivzDX7gCvbJx4bzn+3p6GQ2JWPs41kXj8AJAhIzsQxbyrMyatJwYVmJFzVV3HaZ+nDEaKb3w31ISn3r37qXaH/TTRT7oZM5lrPbvw/ZtcHm2iqpv/DJ8VOMNFHEU8Ob8MGnAhZeUJE6YvTG7TA4kznldofnevYkHx5YSd8HBv23O2g/bWDtqheJ8lV9k2DyiQK3T2gfWP1xQKRwuVVRpoU8y4TcHpIXmMhDxIdGDzKkwsl0dA18usEnoiDqay++H0WeIE X-Forefront-Antispam-Report: CIP:20.93.157.195; CTRY:NL; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:westeu11-emailsignatures-cloud.codetwo.com; PTR:westeu11-emailsignatures-cloud.codetwo.com; CAT:NONE; SFS:(39830400003)(346002)(136003)(376002)(396003)(46966006)(36840700001)(2906002)(356005)(966005)(336012)(8676002)(2616005)(186003)(508600001)(82310400004)(8936002)(47076005)(316002)(36860700001)(70586007)(86362001)(36756003)(6512007)(6486002)(40480700001)(6506007)(5660300002)(6916009)(66574015)(26005)(15974865002)(7596003)(70206006)(7636003)(53546011)(83380400001); DIR:OUT; SFP:1101; X-OriginatorOrg: inett.de X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2022 12:38:15.6423 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 37886d76-8233-4faf-47b5-08d9ceb5e9d5 X-MS-Exchange-CrossTenant-Id: fbb4c88a-8c06-44c8-b416-5376f93313e0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fbb4c88a-8c06-44c8-b416-5376f93313e0; Ip=[20.93.157.195]; Helo=[westeu11-emailsignatures-cloud.codetwo.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR05MB8905 X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature KAM_ASCII_DIVIDERS 0.8 Spam that uses ascii formatting tricks RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust RCVD_IN_MSPIKE_H2 -0.001 Average reputation (+2) SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] Poor client backup performance [Bug#3811] X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2022 12:53:29 -0000 Hi, we ourselves don't have this problem, because we mostly use PBS to back up QEMU VMs, but whenever we use containers, this becomes a problem for us as well. For containers this could be solved using rbd/zfs diffs of snapshots and QEMU backups would benefit as well, but this problem would stay a problem for host backups. It's way too easy to push Proxmox Bakup (Server) to it's limits right now. On Thu, Dec 30, 2021 at 05:30:24PM +0100, Mark Schouten wrote: > Hi Lubomir, >=20 > I think it=E2=80=99s a bit more nuanced, but indeed, the current situatio= n is quite unusable. I=E2=80=99ll have to move my file-based backups back t= o my previous situation. >=20 > My bug has rightfully been closed as a duplicate, but maybe if you have a= ny ideas you could share them here: https://bugzilla.proxmox.com/show_bug.c= gi?id=3D3174 >=20 > I=E2=80=99d like to see this fixed too, and maybe with all brains combine= d, we actually can :) > =20 > =E2=80=94=20 > Mark Schouten, CTO > Tuxis B.V. > mark@tuxis.nl >=20 >=20 >=20 >=20 inett GmbH=0A= >> Ihr IT Systemhaus in Saarbr=C3=BCcken=0A= =0A= Maximilian Hill=0A= =0A= =0A= t: 0681-410993-0=0A= e: mhill@inett.de=0A= www.inett.de=0A= =0A= Technische Fragen:=0A= Support E-Mail: support@inett.de=0A= Support Hotline: 0681-410993-42=0A= =0A= =0A= inett GmbH | Gesch=C3=A4ftsf=C3=BChrung: Marco Gabriel | Amtsgericht Saarbr= =C3=BCcken HRB 16588=0A= > On 30 Dec 2021, at 11:49, Lubomir Apostolov wrote: > >=20 > > Hello,=20 > >=20 > > One year ago I talked about this here without success.=20 > >=20 > > Proxmox Backup has no differential read algorithm and no plans to do it= . It handles only changes during current backup, but has to read every time= all backup data. If you have 20Tb to backup every 3 hours, your storage mu= st be able to read at at least 2Gb/s 24/7 and even more if you dare use you= r VMs at the same time. Complete design failure except for some dev/lab set= ups. > >=20 > > It could use snapshots from the underlying storage like ZFS or Ceph, bu= t no, devs seem happy without it. A simple script for differential snapshot= copy does a better job. > >=20 > > Best regards, > > Lubomir > >=20 > >=20 > > -------- Message d'origine -------- > > De : Mark Schouten > > Date : 30/12/2021 10:07 (GMT+01:00) > > =C3=80 : pbs-devel > > Objet : [pbs-devel] Poor client backup performance [Bug#3811] > >=20 > > Hi! > >=20 > > I switched a few servers to Proxmox Backup Server this week (a slow wee= k :)), but am somewhat disappointed in the performance of the client. I cre= ated bug 3811 for it, and would like to help and assist in debugging this i= ssue. > >=20 > > The mentioned server in the bug should backup about three times per day= . Currently, the machine is working on it almost fulltime, because the back= up takes more than 7 hours. > >=20 > > I think the client reads all files I every run, and I would expect stat= =E2=80=99ing the files should be enough to determine whether a file should = be re-read, but I=E2=80=99ve not spent a single second reading the code, so= I do not have any more than suspicions. > >=20 > > As said, I=E2=80=99d like to help improving this situation, please let = me know what you need from us. > >=20 > > https://bugzilla.proxmox.com/show_bug.cgi?id=3D3811 > >=20 > > Kind regards, > >=20 > > =E2=80=94=20 > > Mark Schouten, CTO > > Tuxis B.V. > > mark@tuxis.nl ___________________________________= ____________ > > pbs-devel mailing list > > pbs-devel@lists.proxmox.com > > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel >=20 > _______________________________________________ > pbs-devel mailing list > pbs-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel