From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 0D668B99BB for ; Fri, 15 Mar 2024 11:25:47 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 93D6319E92 for ; Fri, 15 Mar 2024 11:25:14 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 15 Mar 2024 11:25:09 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 9C1EA48A48 for ; Fri, 15 Mar 2024 11:25:08 +0100 (CET) From: Fiona Ebner To: pve-devel@lists.proxmox.com Date: Fri, 15 Mar 2024 11:24:45 +0100 Message-Id: <20240315102502.84163-5-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240315102502.84163-1-f.ebner@proxmox.com> References: <20240315102502.84163-1-f.ebner@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.069 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pve-devel] [PATCH qemu v2 04/21] qapi: blockdev-backup: add discard-source parameter X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2024 10:25:47 -0000 From: Vladimir Sementsov-Ogievskiy Add a parameter that enables discard-after-copy. That is mostly useful in "push backup with fleecing" scheme, when source is snapshot-access format driver node, based on copy-before-write filter snapshot-access API: [guest] [snapshot-access] ~~ blockdev-backup ~~> [backup target] | | | root | file v v [copy-before-write] | | | file | target v v [active disk] [temp.img] In this case discard-after-copy does two things: - discard data in temp.img to save disk space - avoid further copy-before-write operation in discarded area Note that we have to declare WRITE permission on source in copy-before-write filter, for discard to work. Still we can't take it unconditionally, as it will break normal backup from RO source. So, we have to add a parameter and pass it thorough bdrv_open flags. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner --- Changes in v2: * update to latest upstream version block/backup.c | 5 +++-- block/block-copy.c | 9 +++++++++ block/copy-before-write.c | 15 +++++++++++++-- block/copy-before-write.h | 1 + block/replication.c | 4 ++-- blockdev.c | 2 +- include/block/block-common.h | 2 ++ include/block/block-copy.h | 1 + include/block/block_int-global-state.h | 2 +- qapi/block-core.json | 4 ++++ 10 files changed, 37 insertions(+), 8 deletions(-) diff --git a/block/backup.c b/block/backup.c index af87fa6aa9..3dc955f625 100644 --- a/block/backup.c +++ b/block/backup.c @@ -332,7 +332,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, BitmapSyncMode bitmap_mode, - bool compress, + bool compress, bool discard_source, const char *filter_node_name, BackupPerf *perf, BlockdevOnError on_source_error, @@ -429,7 +429,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, goto error; } - cbw = bdrv_cbw_append(bs, target, filter_node_name, &bcs, errp); + cbw = bdrv_cbw_append(bs, target, filter_node_name, discard_source, + &bcs, errp); if (!cbw) { goto error; } diff --git a/block/block-copy.c b/block/block-copy.c index b61685f1a2..3c61e52bae 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -137,6 +137,7 @@ typedef struct BlockCopyState { CoMutex lock; int64_t in_flight_bytes; BlockCopyMethod method; + bool discard_source; BlockReqList reqs; QLIST_HEAD(, BlockCopyCallState) calls; /* @@ -348,6 +349,7 @@ static int64_t block_copy_calculate_cluster_size(BlockDriverState *target, BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, BlockDriverState *copy_bitmap_bs, const BdrvDirtyBitmap *bitmap, + bool discard_source, Error **errp) { ERRP_GUARD(); @@ -409,6 +411,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, cluster_size), }; + s->discard_source = discard_source; block_copy_set_copy_opts(s, false, false); ratelimit_init(&s->rate_limit); @@ -580,6 +583,12 @@ static coroutine_fn int block_copy_task_entry(AioTask *task) co_put_to_shres(s->mem, t->req.bytes); block_copy_task_end(t, ret); + if (s->discard_source && ret == 0) { + int64_t nbytes = + MIN(t->req.offset + t->req.bytes, s->len) - t->req.offset; + bdrv_co_pdiscard(s->source, t->req.offset, nbytes); + } + return ret; } diff --git a/block/copy-before-write.c b/block/copy-before-write.c index d3b95bd600..3503702d71 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -44,6 +44,7 @@ typedef struct BDRVCopyBeforeWriteState { BdrvChild *target; OnCbwError on_cbw_error; uint32_t cbw_timeout_ns; + bool discard_source; /* * @lock: protects access to @access_bitmap, @done_bitmap and @@ -357,6 +358,8 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, uint64_t perm, uint64_t shared, uint64_t *nperm, uint64_t *nshared) { + BDRVCopyBeforeWriteState *s = bs->opaque; + if (!(role & BDRV_CHILD_FILTERED)) { /* * Target child @@ -381,6 +384,10 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, * start */ *nperm = *nperm | BLK_PERM_CONSISTENT_READ; + if (s->discard_source) { + *nperm = *nperm | BLK_PERM_WRITE; + } + *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); } } @@ -470,7 +477,9 @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags, ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags); - s->bcs = block_copy_state_new(bs->file, s->target, bs, bitmap, errp); + s->discard_source = flags & BDRV_O_CBW_DISCARD_SOURCE; + s->bcs = block_copy_state_new(bs->file, s->target, bs, bitmap, + flags & BDRV_O_CBW_DISCARD_SOURCE, errp); if (!s->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); ret = -EINVAL; @@ -544,12 +553,14 @@ BlockDriver bdrv_cbw_filter = { BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, + bool discard_source, BlockCopyState **bcs, Error **errp) { BDRVCopyBeforeWriteState *state; BlockDriverState *top; QDict *opts; + int flags = BDRV_O_RDWR | (discard_source ? BDRV_O_CBW_DISCARD_SOURCE : 0); assert(source->total_sectors == target->total_sectors); GLOBAL_STATE_CODE(); @@ -562,7 +573,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, qdict_put_str(opts, "file", bdrv_get_node_name(source)); qdict_put_str(opts, "target", bdrv_get_node_name(target)); - top = bdrv_insert_node(source, opts, BDRV_O_RDWR, errp); + top = bdrv_insert_node(source, opts, flags, errp); if (!top) { return NULL; } diff --git a/block/copy-before-write.h b/block/copy-before-write.h index 6e72bb25e9..01af0cd3c4 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -39,6 +39,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, + bool discard_source, BlockCopyState **bcs, Error **errp); void bdrv_cbw_drop(BlockDriverState *bs); diff --git a/block/replication.c b/block/replication.c index ea4bf1aa80..39ad78cf98 100644 --- a/block/replication.c +++ b/block/replication.c @@ -579,8 +579,8 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode, s->backup_job = backup_job_create( NULL, s->secondary_disk->bs, s->hidden_disk->bs, - 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, NULL, - &perf, + 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, false, + NULL, &perf, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, backup_job_completed, bs, NULL, &local_err); diff --git a/blockdev.c b/blockdev.c index 7793143d76..ce3fef924c 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2802,7 +2802,7 @@ static BlockJob *do_backup_common(BackupCommon *backup, job = backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, bmap, backup->bitmap_mode, - backup->compress, + backup->compress, backup->discard_source, backup->filter_node_name, &perf, backup->on_source_error, diff --git a/include/block/block-common.h b/include/block/block-common.h index e15395f2cb..913a8b259c 100644 --- a/include/block/block-common.h +++ b/include/block/block-common.h @@ -234,6 +234,8 @@ typedef enum { read-write fails */ #define BDRV_O_IO_URING 0x40000 /* use io_uring instead of the thread pool */ +#define BDRV_O_CBW_DISCARD_SOURCE 0x80000 /* for copy-before-write filter */ + #define BDRV_O_CACHE_MASK (BDRV_O_NOCACHE | BDRV_O_NO_FLUSH) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 8b41643bfa..bdc703bacd 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -27,6 +27,7 @@ typedef struct BlockCopyCallState BlockCopyCallState; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, BlockDriverState *copy_bitmap_bs, const BdrvDirtyBitmap *bitmap, + bool discard_source, Error **errp); /* Function should be called prior any actual copy request */ diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h index 32f0f9858a..546f2b5532 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -189,7 +189,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, BitmapSyncMode bitmap_mode, - bool compress, + bool compress, bool discard_source, const char *filter_node_name, BackupPerf *perf, BlockdevOnError on_source_error, diff --git a/qapi/block-core.json b/qapi/block-core.json index 09de550c95..4297e5beda 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1816,6 +1816,9 @@ # node specified by @drive. If this option is not given, a node # name is autogenerated. (Since: 4.2) # +# @discard-source: Discard blocks on source which are already copied +# to the target. (Since 9.0) +# # @x-perf: Performance options. (Since 6.0) # # Features: @@ -1837,6 +1840,7 @@ '*on-target-error': 'BlockdevOnError', '*auto-finalize': 'bool', '*auto-dismiss': 'bool', '*filter-node-name': 'str', + '*discard-source': 'bool', '*x-perf': { 'type': 'BackupPerf', 'features': [ 'unstable' ] } } } -- 2.39.2