From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id EA0C0654DF for ; Thu, 23 Jul 2020 08:43:14 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DFF0725985 for ; Thu, 23 Jul 2020 08:43:14 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id EAC372597B for ; Thu, 23 Jul 2020 08:43:13 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id B807E43309 for ; Thu, 23 Jul 2020 08:43:13 +0200 (CEST) Date: Thu, 23 Jul 2020 08:43:06 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox VE user list References: <1110267368.76036.1595436034847.JavaMail.zimbra@fws.fr> In-Reply-To: <1110267368.76036.1595436034847.JavaMail.zimbra@fws.fr> MIME-Version: 1.0 User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid) Message-Id: <1595486387.pi9zv7y79a.astroid@nora.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.068 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [PVE-User] PBS : is dirty-bitmap really accurate ? X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jul 2020 06:43:15 -0000 On July 22, 2020 6:40 pm, Daniel Berteaud wrote: > Hi=20 >=20 > I've started playing with PBS on some VM. So far, it's looking really pro= mizing.=20 > There's one strange thing though : the percent of the dirty data. For exa= mple, I backup one VM every 2 or 3 days. It's a moderately busy server, mai= nly serving a MariaDB database (zabbix server + mariadb). On each backup, I= get similar dirty values :=20 >=20 > INFO: using fast incremental mode (dirty-bitmap), 492.7 GiB dirty of 590.= 0 GiB total=20 >=20 > While I'm sur not even 10% of this has really been written.=20 >=20 > Get more or less the same problem on other VM. One which I know just slee= p all day (my personnal OnlyOffice document server), and which I backup dai= ly, and get values like :=20 >=20 > INFO: using fast incremental mode (dirty-bitmap), 5.0 GiB dirty of 10.0 G= iB total=20 >=20 > Or another small one (personnal samba DC controler) :=20 >=20 > INFO: using fast incremental mode (dirty-bitmap), 13.0 GiB dirty of 20.0 = GiB total=20 >=20 > The only write activity for those 2 are just a few KB or maybe MB of log = lines. Respectivly 5 and 13GB of dirty blocks seems unreal.=20 >=20 > Am I the only one seeing this ? Could the dirty-bitmap mark dirty blocks = without write activity somehow ?=20 possibly you haven't upgraded to pve-qemu-kvm 5.0-11 (or your VM hasn't=20 been restarted yet since the upgrade): https://git.proxmox.com/?p=3Dpve-qemu.git;a=3Dcommit;h=3Df257cc05f4fbf772c= ad3231021b3ce7587127a1b the bitmap has a granularity of 4MB, so depending on the activity inside=20 you can see quite a bit of amplification. also writing and then=20 zeroing/reverting again to the old content would leave a mark in the=20 bitmap without permanently changing the contents. =