From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 10CF2872CD for ; Thu, 30 Dec 2021 11:53:09 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 04C1B18ABB for ; Thu, 30 Dec 2021 11:53:09 +0100 (CET) Received: from mail.directique.com (mail.directique.com [176.31.91.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 3D2C518AA4 for ; Thu, 30 Dec 2021 11:53:07 +0100 (CET) Received: from SCOM4.directique.net ([::1]) by SCOM4.directique.net ([::1]) with mapi id 14.03.0513.000; Thu, 30 Dec 2021 11:49:55 +0100 From: Lubomir Apostolov To: Proxmox Backup Server development discussion Thread-Topic: [pbs-devel] Poor client backup performance [Bug#3811] Thread-Index: AQHX/Vyo8eg6RVLGXUiE/7+4JS5M6axK2zxY Date: Thu, 30 Dec 2021 10:49:54 +0000 Message-ID: <718D0AF11703FA4C85B0535448A056100381A40015@SCOM4.directique.net> References: <17A444C0-1F31-42AA-81FE-A95CE4B3291E@tuxis.nl> In-Reply-To: <17A444C0-1F31-42AA-81FE-A95CE4B3291E@tuxis.nl> Accept-Language: fr-FR, en-US Content-Language: fr-FR X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_718D0AF11703FA4C85B0535448A056100381A40015SCOM4directiq_" MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% HTML_MESSAGE 0.001 HTML included in message KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] Poor client backup performance [Bug#3811] X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Dec 2021 10:53:09 -0000 --_000_718D0AF11703FA4C85B0535448A056100381A40015SCOM4directiq_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hello, One year ago I talked about this here without success. Proxmox Backup has no differential read algorithm and no plans to do it. It= handles only changes during current backup, but has to read every time all= backup data. If you have 20Tb to backup every 3 hours, your storage must b= e able to read at at least 2Gb/s 24/7 and even more if you dare use your VM= s at the same time. Complete design failure except for some dev/lab setups. It could use snapshots from the underlying storage like ZFS or Ceph, but no= , devs seem happy without it. A simple script for differential snapshot cop= y does a better job. Best regards, Lubomir -------- Message d'origine -------- De : Mark Schouten Date : 30/12/2021 10:07 (GMT+01:00) =C0 : pbs-devel Objet : [pbs-devel] Poor client backup performance [Bug#3811] Hi! I switched a few servers to Proxmox Backup Server this week (a slow week :)= ), but am somewhat disappointed in the performance of the client. I created= bug 3811 for it, and would like to help and assist in debugging this issue= . The mentioned server in the bug should backup about three times per day. Cu= rrently, the machine is working on it almost fulltime, because the backup t= akes more than 7 hours. I think the client reads all files I every run, and I would expect stat=92i= ng the files should be enough to determine whether a file should be re-read= , but I=92ve not spent a single second reading the code, so I do not have a= ny more than suspicions. As said, I=92d like to help improving this situation, please let me know wh= at you need from us. https://bugzilla.proxmox.com/show_bug.cgi?id=3D3811 Kind regards, =97 Mark Schouten, CTO Tuxis B.V. mark@tuxis.nl --_000_718D0AF11703FA4C85B0535448A056100381A40015SCOM4directiq_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hello, 

One year ago I talked about this here without success.&nb= sp;

Proxmox Backup has no differential read algorithm and no = plans to do it. It handles only changes during current backup, but has to r= ead every time all backup data. If you have 20Tb to backup every 3 hours, y= our storage must be able to read at at least 2Gb/s 24/7 and even more if you dare use your VMs at the same tim= e. Complete design failure except for some dev/lab setups.

It could use snapshots from the underlying storage like Z= FS or Ceph, but no, devs seem happy without it. A simple script for differe= ntial snapshot copy does a better job.

Best regards,
Lubomir


-------- Message d'origine --------
De : Mark Schouten <mark@tuxis.nl>
Date : 30/12/2021 10:07 (GMT+01:00)
=C0 : pbs-devel <pbs-devel@lists.proxmox.com>
Objet : [pbs-devel] Poor client backup performance [Bug#3811]

Hi!

I switched a few servers to Proxmox Backup Server this week= (a slow week :)), but am somewhat disappointed in the performance of the c= lient. I created bug 3811 for it, and would like to help and assist in debu= gging this issue.

The mentioned server in the bug should backup about three t= imes per day. Currently, the machine is working on it almost fulltime, beca= use the backup takes more than 7 hours.

I think the client reads all files I every run, and I would= expect stat=92ing the files should be enough to determine whether a file s= hould be re-read, but I=92ve not spent a single second reading the code, so= I do not have any more than suspicions.

As said, I=92d like to help improving this situation, pleas= e let me know what you need from us.

https://bugzilla.proxmox.com/show_bug.cgi?id=3D3811

Kind regards,

=97 
Mark Schouten, CTO
Tuxis B.V.
mark@tuxis.nl
--_000_718D0AF11703FA4C85B0535448A056100381A40015SCOM4directiq_--