From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 3A88D1FF0A7 for ; Wed, 02 Sep 2026 15:56:20 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id E3B7421693; Wed, 02 Sep 2026 15:55:47 +0200 (CEST) From: Fiona Ebner To: pve-devel@lists.proxmox.com Subject: [RFC qemu/qemu-server/proxmox-backup-qemu 0/9] close #6169: backup: pbs: per-target dirty bitmaps Date: Wed, 2 Sep 2026 15:54:31 +0200 Message-ID: <20260902135536.525194-1-f.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1788357335746 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.753 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 56OLWZX3D54LPUWY5KH3UFG3UP7XJXH6 X-Message-ID-Hash: 56OLWZX3D54LPUWY5KH3UFG3UP7XJXH6 X-MailFrom: f.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: To decide whether an incremental backup can be done, the checksum of the latest backup on the target is compared against the checksum from the previous backup the QEMU client library did. This was previously tracked per device as the PVE backup code in QEMU also only used a single bitmap per device. So backing up to target A, dirtying the disk and backing up to target B would mean that the checksum for target A is overwritten and a subsequent backup to target A needs to discard the dirty bitmap. Allow for tracking multiple targets with different bitmaps, by saving the checksums per device+target combination, rather than just per device. Whether the new behavior or the old behavior is used is controlled by QEMU, optionally passing along the target ID, and a flag for the export and import functions for migration. These are machine version guarded on the QEMU/qemu-server side to avoid a mismatch. This is an ABI change for the backup library, so a versioned breaks and a versioned depends in the other direction is needed. QUESTION: with old qemu-server, target ID is not provided, but the new machine version might be in use. In this case, the dirty bitmap state is not migrated, in case of a new -> old -> new migration, which can lead to stale info! Would it be okay to add a Breaks for old qemu-server or should I try to resolve it differently somehow? QUESTION: How to best avoid 'orphaned' bitmaps? Auto-remove for storages no longer in configuration? Or have some knob/limit for how many to keep? The clippy fixes are independent. proxmox-backup-qemu: Fiona Ebner (6): clippy: restore: fix needless borrows clippy: commands: avoid manual implementation of ok() cargo: add dependency for serde close #6169: track checksums for incremental backups per target update current-api.h d/control: bump versioned breaks for pve-qemu-kvm Cargo.toml | 1 + current-api.h | 9 +++-- debian/control | 2 +- src/backup.rs | 27 ++++++++++---- src/commands.rs | 97 +++++++++++++++++++++++++++++++++++++++---------- src/lib.rs | 25 +++++++++---- src/restore.rs | 12 ++---- 7 files changed, 126 insertions(+), 47 deletions(-) qemu: Fiona Ebner (2): PVE backup: properly track if snapshot access was set up PVE backup: track per-target dirty bitmaps block/monitor/block-hmp-cmds.c | 1 + hw/core/machine.c | 2 ++ migration/pbs-state.c | 42 ++++++++++++++++++++++++++++------ migration/pbs-state.h | 32 ++++++++++++++++++++++++++ proxmox-backup-client.c | 12 +++++++++- proxmox-backup-client.h | 1 + pve-backup.c | 29 +++++++++++++++++------ qapi/block-core.json | 7 +++++- 8 files changed, 110 insertions(+), 16 deletions(-) create mode 100644 migration/pbs-state.h qemu-server: Fiona Ebner (1): close #6169: backup: pbs: specify target ID so QEMU keeps track of per-target dirty bitmap src/PVE/VZDump/QemuServer.pm | 6 ++++++ 1 file changed, 6 insertions(+) Summary over all repositories: 16 files changed, 242 insertions(+), 63 deletions(-) -- Generated by git-murpp 0.5.0