From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pve@junkyard.4t2.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8721263E3F for <pve-user@lists.proxmox.com>; Fri, 17 Jul 2020 15:24:32 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 7CA061C4E6 for <pve-user@lists.proxmox.com>; Fri, 17 Jul 2020 15:24:02 +0200 (CEST) Received: from mxf.4t2.com (mxf.4t2.com [78.47.65.59]) by firstgate.proxmox.com (Proxmox) with ESMTP id AF1DF1C4D7 for <pve-user@lists.proxmox.com>; Fri, 17 Jul 2020 15:23:59 +0200 (CEST) X-Spam-Status: No X-4t2Systems-MailScanner-Watermark: 1595597018.00331@Aos2pLpq86hW17q34aqsAQ X-4t2Systems-MailScanner-From: pve@junkyard.4t2.com X-4t2Systems-MailScanner: Found to be clean X-4t2Systems-MailScanner-ID: DE05E4138A.AA59F X-4t2Systems-MailScanner-Information: processed at mxf.4t2.com Received: from mailrelay.abyss.4t2.com (mailrelay.abyss.4t2.com [192.168.1.11]) by mxf.4t2.com (Postfix) with ESMTP id DE05E4138A for <pve-user@lists.proxmox.com>; Fri, 17 Jul 2020 15:23:37 +0200 (CEST) Received: from mailserv.abyss.4t2.com (mailserv.abyss.4t2.com [192.168.1.12]) by mailrelay.abyss.4t2.com (Postfix) with ESMTP id C70542A7 for <pve-user@lists.proxmox.com>; Fri, 17 Jul 2020 15:23:37 +0200 (CEST) Received: from [10.8.1.101] (zerberus.abyss.4t2.com [192.168.1.254]) (Authenticated sender: x) by mailserv.abyss.4t2.com (Postfix) with ESMTPSA id ADC752063F for <pve-user@lists.proxmox.com>; Fri, 17 Jul 2020 15:23:37 +0200 (CEST) Message-ID: <1edc84ae5a9b082c5744149ce7d3e0dfdc32a2ae.camel@junkyard.4t2.com> From: Tom Weber <pve@junkyard.4t2.com> To: pve-user@lists.proxmox.com Date: Fri, 17 Jul 2020 15:23:37 +0200 In-Reply-To: <1594969118.mb1771vpca.astroid@nora.none> References: <c84ac772-d577-27fd-710c-293d8a4baffe@proxmox.com> <7d1bd7a4-f47b-4a1a-9278-ff1889508c33@gmail.com> <1768587204.461.1594383204281@webmail.proxmox.com> <450971473.487.1594395714078@webmail.proxmox.com> <aa8534d4-174b-5069-e27b-160b8fb92d72@merit.unu.edu> <c51e8cf4-1c6d-051b-f91d-5d600a566c5a@proxmox.com> <c522fc9b-6b3f-7770-d32a-c1bb47961884@merit.unu.edu> <1514606299.525.1594478405167@webmail.proxmox.com> <16057806.272035.1594737045788.JavaMail.zimbra@odiso.com> <0852a3fa-ab39-d551-5a01-0264687d4b56@proxmox.com> <4b7e3a23d9b1f41ef51e6363373072d265797380.camel@junkyard.4t2.com> <1594969118.mb1771vpca.astroid@nora.none> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.2 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [PVE-User] Proxmox Backup Server (beta) X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list <pve-user.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-user/> List-Post: <mailto:pve-user@lists.proxmox.com> List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Fri, 17 Jul 2020 13:24:32 -0000 Am Freitag, den 17.07.2020, 09:31 +0200 schrieb Fabian Grünbichler: > On July 16, 2020 3:03 pm, Tom Weber wrote: > > Am Dienstag, den 14.07.2020, 17:52 +0200 schrieb Thomas Lamprecht: > > > > > > Proxmox Backup Server effectively does that too, but independent > > > from > > > the > > > source storage. We always get the last backup index and only > > > upload > > > the chunks > > > which changed. For running VMs dirty-bitmap is on to improve this > > > (avoids > > > reading of unchanged blocks) but it's only an optimization - the > > > backup is > > > incremental either way. > > > > So there is exactly one dirty-bitmap that get's nulled after a > > backup? > > > > I'm asking because I have Backup setups with 2 Backup Servers at > > different Locations, backing up (file-level, incremental) on odd > > days > > to server1 on even days to server2. > > > > Such a setup wouldn't work with the block level incremental backup > > and > > the dirty-bitmap for pve vms + pbs, right? > > > > Regards, > > Tom > > right now, this would not work since for each backup, the bitmap > would > be invalidated since the last backup returned by the server does not > match the locally stored value. theoretically we could track > multiple > backup storages, but bitmaps are not free and the handling would > quickly > become unwieldy. > > probably you are better off backing up to one server and syncing > that to your second one - you can define both as storage on the PVE > side > and switch over the backup job targets if the primary one fails. > > theoretically[1] > > 1.) backup to A > 2.) sync A->B > 3.) backup to B > 4.) sync B->A > 5.) repeat > > works as well and keeps the bitmap valid, but you carefully need to > lock-step backup and sync jobs, so it's probably less robust than: > > 1.) backup to A > 2.) sync A->B > > where missing a sync is not ideal, but does not invalidate the > bitmap. > > note that your backup will still be incremental in any case w.r.t. > client <-> server traffic, the client just has to re-read all disks > to > decide whether it has to upload those chunks or not if the bitmap is > not > valid or does not exist. > > 1: theoretically, as you probably run into > https://bugzilla.proxmox.com/show_bug.cgi?id=2864 unless you do your > backups as 'backup@pam', which is not recommended ;) > thanks for the very detailed answer :) I was already thinking that this wouldn't work like my current setup. Once the bitmap on the source side of the backup gets corrupted for whatever reason, incremental wouldn't work and break. Is there some way that the system would notify such a "corrupted" bitmap? I'm thinking of a manual / test / accidential backup run to a different backup server which would / could ruin all further regular incremental backups undetected. about my setup scenario - a bit off topic - backing up to 2 different locations every other day basically doubles my backup space and reduces the risk of one failing backup server - of course by taking a 50:50 chance of needing to go back 2 days in a worst case scenario. Syncing the backup servers would require twice the space capacity (and additional bw). For now I'm just trying to understand the features and limits of pbs - which really looks nice so far! Regards, Tom