From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <f.ebner@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id 2F7E89B225
 for <pve-devel@lists.proxmox.com>; Tue, 17 Oct 2023 14:10:21 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id 0F59632B03
 for <pve-devel@lists.proxmox.com>; Tue, 17 Oct 2023 14:10:21 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [94.136.29.106])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS
 for <pve-devel@lists.proxmox.com>; Tue, 17 Oct 2023 14:10:17 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 42EF641B38
 for <pve-devel@lists.proxmox.com>; Tue, 17 Oct 2023 14:10:17 +0200 (CEST)
From: Fiona Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Tue, 17 Oct 2023 14:10:05 +0200
Message-Id: <20231017121012.132636-1-f.ebner@proxmox.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results:  0
 AWL -0.337 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DMARC_MISSING             0.1 Missing DMARC policy
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
 information. [test.py, meson.build, argparse.py]
Subject: [pve-devel] [PATCH-SERIES v3 qemu] update to QEMU 8.1.2
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Tue, 17 Oct 2023 12:10:21 -0000

Patch changes:

For backup, opening the backup dump block driver needed to be adapted,
because of coroutine context changes.

Block graph locking was disabled, because of deadlocks.

Snapshot code has a huge performance regression which required a
workaround.


Meta-changes:

Use --disable-download options to avoid automatic downloads during
build, but require the user to do so once themselves. Also done when
initializing the submodule in the Makefile

Switch back to using QEMU's keycodemapdb instead of splitting it out
in our build-dir. Wasn't updated since QEMU 6.0 anymore and the reason
for the split is not known. If anything pops up, we can re-do it and
document the reason this time.

Versioned Breaks for qemu-server required, because an upstream change
would prevent VM boot with most configurations (including default
one).


Changes in v3:
    * Rebase on QEMU 8.1.2 which includes a few more TCG and
      migration-related fixes and...
    * ...remove previously picked-up patches that are included there.

Changes in v2:
    * Add stable fix for guest-triggerable SCSI crash.
    * Make reverting graph locking complete.


Fiona Ebner (7):
  d/rules: use disable-download option instead of git-submodules=ignore
  buildsys: fixup submodule target
  buildsys: use QEMU's keycodemapdb again
  update submodule and patches to QEMU 8.1.2
  add patch to disable graph locking
  add patch to avoid huge snapshot performance regression
  d/control: add versioned Breaks for qemu-server <= 8.0.6

 Makefile                                      |   20 +-
 debian/control                                |    1 +
 ...d-support-for-sync-bitmap-mode-never.patch |  124 +-
 ...-support-for-conditional-and-always-.patch |   10 +-
 ...check-for-bitmap-mode-without-bitmap.patch |    4 +-
 ...-to-bdrv_dirty_bitmap_merge_internal.patch |    6 +-
 .../0006-mirror-move-some-checks-to-qmp.patch |    8 +-
 ...race-with-clients-disconnecting-earl.patch |   22 +-
 ...as-Internal-cdbs-have-16-byte-length.patch |   10 +-
 ...ial-deadlock-when-draining-during-tr.patch |   10 +-
 ...irty-bitmap-fix-loading-bitmap-when.patch} |    2 +-
 ...hen-getting-cursor-without-a-console.patch |   36 -
 ...el-async-DMA-operation-before-reset.patch} |    4 +-
 ...-memory-prevent-dma-reentracy-issues.patch |  130 -
 ...t-graph-lock-Disable-locking-for-now.patch |  140 +
 ...le-reentrancy-detection-for-script-R.patch |   39 -
 ...-disable-reentrancy-detection-for-io.patch |   37 -
 ...-workaround-snapshot-performance-reg.patch |   57 +
 ...sable-reentrancy-detection-for-iomem.patch |   35 -
 ...le-reentrancy-detection-for-apic-msi.patch |   36 -
 .../extra/0011-vhost-fix-the-fd-leak.patch    |   29 -
 ...k-file-change-locking-default-to-off.patch |    6 +-
 ...he-CPU-model-to-kvm64-32-instead-of-.patch |    4 +-
 ...erfs-no-default-logfile-if-daemonize.patch |    4 +-
 ...PVE-Up-glusterfs-allow-partial-reads.patch |    2 +-
 ...return-success-on-info-without-snaps.patch |    4 +-
 ...dd-add-osize-and-read-from-to-stdin-.patch |   12 +-
 ...E-Up-qemu-img-dd-add-isize-parameter.patch |   14 +-
 ...PVE-Up-qemu-img-dd-add-n-skip_create.patch |   10 +-
 ...-add-l-option-for-loading-a-snapshot.patch |   14 +-
 ...virtio-balloon-improve-query-balloon.patch |   12 +-
 .../0014-PVE-qapi-modify-query-machines.patch |   12 +-
 .../0015-PVE-qapi-modify-spice-query.patch    |    4 +-
 ...nnel-implementation-for-savevm-async.patch |    8 +-
 ...async-for-background-state-snapshots.patch |   58 +-
 ...add-optional-buffer-size-to-QEMUFile.patch |   44 +-
 ...add-the-zeroinit-block-driver-filter.patch |   10 +-
 ...-Add-dummy-id-command-line-parameter.patch |   10 +-
 ...le-posix-make-locking-optiono-on-cre.patch |   18 +-
 ...3-PVE-monitor-disable-oob-capability.patch |    4 +-
 ...sed-balloon-qemu-4-0-config-size-fal.patch |    4 +-
 ...E-Allow-version-code-in-machine-type.patch |   22 +-
 ...VE-Backup-add-vma-backup-format-code.patch |   22 +-
 ...-Backup-add-backup-dump-block-driver.patch |    4 +-
 ...ckup-Proxmox-backup-patches-for-QEMU.patch |  127 +-
 ...estore-new-command-to-restore-from-p.patch |    4 +-
 ...k-driver-to-map-backup-archives-into.patch |   54 +-
 ...ct-stderr-to-journal-when-daemonized.patch |   12 +-
 ...igrate-dirty-bitmap-state-via-savevm.patch |   23 +-
 ...dirty-bitmap-migrate-other-bitmaps-e.patch |    4 +-
 ...all-back-to-open-iscsi-initiatorname.patch |    4 +-
 ...PVE-block-stream-increase-chunk-size.patch |    2 +-
 ...accept-NULL-qiov-in-bdrv_pad_request.patch |   14 +-
 .../0039-block-add-alloc-track-driver.patch   |    2 +-
 ...apshots-hold-the-BQL-during-setup-ca.patch |   24 +-
 ...vm-async-don-t-hold-BQL-during-setup.patch |    4 +-
 debian/patches/series                         |   13 +-
 debian/rules                                  |    2 +-
 keycodemapdb/LICENSE.BSD                      |   27 -
 keycodemapdb/LICENSE.GPL2                     |  339 ---
 keycodemapdb/README                           |  114 -
 keycodemapdb/data/README                      |   89 -
 keycodemapdb/data/keymaps.csv                 |  539 ----
 keycodemapdb/meson.build                      |    1 -
 keycodemapdb/tests/.gitignore                 |   11 -
 keycodemapdb/tests/Makefile                   |  150 --
 keycodemapdb/tests/javascript                 |   53 -
 keycodemapdb/tests/python2                    |    3 -
 keycodemapdb/tests/python3                    |    3 -
 keycodemapdb/tests/stdc++.cc                  |   40 -
 keycodemapdb/tests/stdc.c                     |   64 -
 keycodemapdb/tests/test.py                    |   30 -
 keycodemapdb/thirdparty/LICENSE-argparse.txt  |   20 -
 keycodemapdb/thirdparty/__init__.py           |    0
 keycodemapdb/thirdparty/argparse.py           | 2392 -----------------
 keycodemapdb/tools/keymap-gen                 | 1147 --------
 qemu                                          |    2 +-
 77 files changed, 617 insertions(+), 5758 deletions(-)
 rename debian/patches/extra/{0010-migration-block-dirty-bitmap-fix-loading-bitmap-when.patch => 0004-migration-block-dirty-bitmap-fix-loading-bitmap-when.patch} (98%)
 delete mode 100644 debian/patches/extra/0004-ui-return-NULL-when-getting-cursor-without-a-console.patch
 rename debian/patches/extra/{0012-hw-ide-reset-cancel-async-DMA-operation-before-reset.patch => 0005-hw-ide-reset-cancel-async-DMA-operation-before-reset.patch} (97%)
 delete mode 100644 debian/patches/extra/0005-memory-prevent-dma-reentracy-issues.patch
 create mode 100644 debian/patches/extra/0006-Revert-Revert-graph-lock-Disable-locking-for-now.patch
 delete mode 100644 debian/patches/extra/0006-lsi53c895a-disable-reentrancy-detection-for-script-R.patch
 delete mode 100644 debian/patches/extra/0007-bcm2835_property-disable-reentrancy-detection-for-io.patch
 create mode 100644 debian/patches/extra/0007-migration-states-workaround-snapshot-performance-reg.patch
 delete mode 100644 debian/patches/extra/0008-raven-disable-reentrancy-detection-for-iomem.patch
 delete mode 100644 debian/patches/extra/0009-apic-disable-reentrancy-detection-for-apic-msi.patch
 delete mode 100644 debian/patches/extra/0011-vhost-fix-the-fd-leak.patch
 delete mode 100644 keycodemapdb/LICENSE.BSD
 delete mode 100644 keycodemapdb/LICENSE.GPL2
 delete mode 100644 keycodemapdb/README
 delete mode 100644 keycodemapdb/data/README
 delete mode 100644 keycodemapdb/data/keymaps.csv
 delete mode 100644 keycodemapdb/meson.build
 delete mode 100644 keycodemapdb/tests/.gitignore
 delete mode 100644 keycodemapdb/tests/Makefile
 delete mode 100755 keycodemapdb/tests/javascript
 delete mode 100755 keycodemapdb/tests/python2
 delete mode 100755 keycodemapdb/tests/python3
 delete mode 100644 keycodemapdb/tests/stdc++.cc
 delete mode 100644 keycodemapdb/tests/stdc.c
 delete mode 100644 keycodemapdb/tests/test.py
 delete mode 100644 keycodemapdb/thirdparty/LICENSE-argparse.txt
 delete mode 100644 keycodemapdb/thirdparty/__init__.py
 delete mode 100644 keycodemapdb/thirdparty/argparse.py
 delete mode 100755 keycodemapdb/tools/keymap-gen

-- 
2.39.2