From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 2090A1FF16E
for <inbox@lore.proxmox.com>; Mon, 17 Mar 2025 15:18:24 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 37EBE54EF;
Mon, 17 Mar 2025 15:17:33 +0100 (CET)
From: Christoph Heiss <c.heiss@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Mon, 17 Mar 2025 15:11:37 +0100
Message-ID: <20250317141152.1247324-1-c.heiss@proxmox.com>
X-Mailer: git-send-email 2.48.1
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.027 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH many 00/14] fix #5180: migrate conntrack state
on live migration
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
Fixes #5180 [0].
This implements migration of per-VM conntrack state on live-migration.
The core of the implementation are in patch #7 & #8. See there for more
details.
Patch #1 - #3 implement CONNMARK'ing any VM traffic with their unique
VMID. This is needed later on to filter conntrack entries for the
migration. These three patches can be applied independently,
CONNMARK'ing traffic does not have any visible impact.
Patch #13 & #14 are marked RFC, as I'm not sure if we need/should
implement that. But it's working well and cleanup of old resources is
always good IMHO.
Currently, remote/inter-cluster migration is not supported and indicated
to the user with a warning. See also patch #8 for a bit more in-depth
explanation.
Needed dependency bumps between packages are indicated in the notes
appropriately.
[0] https://bugzilla.proxmox.com/show_bug.cgi?id=5180
Testing
=======
I've primarily tested intra-cluster live-migrations, with both the
iptables-based and nftables-based firewall), using the reproducer as
described in #5180. I further verified that the D-Bus servers get
started as expected and are _always_ stopped, even in the case of some
migration error.
Finally, I also checked using `conntrack -L -m <vmid>` tool that the
conntrack entries are
a) added/updated on the target node and
b) removed from the source node (w/ patch #13/#14 applied).
Also tested was the migration from/to an "old" (unpatched) node, which
results in the issue as per #5180 & appropriate warnings in the UI.
For remote migrations, only tested that the warning is logged as
expected.
Diffstat
========
pve-firewall:
Christoph Heiss (2):
firewall: add connmark rule with VMID to all guest chains
firewall: helpers: add sub for flushing conntrack entries by mark
debian/control | 3 ++-
src/PVE/Firewall.pm | 7 +++++--
src/PVE/Firewall/Helpers.pm | 11 +++++++++++
3 files changed, 18 insertions(+), 3 deletions(-)
proxmox-firewall:
Christoph Heiss (1):
firewall: add connmark rule with VMID to all guest chains
proxmox-firewall/src/firewall.rs | 14 ++-
.../integration_tests__firewall.snap | 85 ++++++++++++++++++-
proxmox-nftables/src/expression.rs | 9 ++
proxmox-nftables/src/statement.rs | 10 ++-
4 files changed, 114 insertions(+), 4 deletions(-)
proxmox-ve-rs:
Christoph Heiss (1):
config: guest: allow access to raw Vmid value
proxmox-ve-config/src/guest/types.rs | 4 ++++
1 file changed, 4 insertions(+)
qemu-server:
Christoph Heiss (5):
qmp helpers: allow passing structured args via qemu_objectadd()
api2: qemu: add module exposing node migration capabilities
fix #5180: libexec: add QEMU dbus-vmstate daemon for migrating
conntrack
fix #5180: migrate: integrate helper for live-migrating conntrack info
migrate: flush old VM conntrack entries after successful migration
Makefile | 3 +
PVE/API2/Qemu.pm | 72 +++++++++++++++
PVE/API2/Qemu/Makefile | 2 +-
PVE/API2/Qemu/Migration.pm | 46 ++++++++++
PVE/CLI/qm.pm | 5 ++
PVE/QemuMigrate.pm | 69 ++++++++++++++
PVE/QemuServer.pm | 6 ++
PVE/QemuServer/DBusVMState.pm | 124 +++++++++++++++++++++++++
PVE/QemuServer/Makefile | 1 +
PVE/QemuServer/QMPHelpers.pm | 4 +-
debian/control | 7 +-
libexec/dbus-vmstate | 164 ++++++++++++++++++++++++++++++++++
org.qemu.VMState1.conf | 11 +++
13 files changed, 510 insertions(+), 4 deletions(-)
create mode 100644 PVE/API2/Qemu/Migration.pm
create mode 100644 PVE/QemuServer/DBusVMState.pm
create mode 100755 libexec/dbus-vmstate
create mode 100644 org.qemu.VMState1.conf
pve-common:
Christoph Heiss (1):
tools: add run_fork_detached() for spawning daemons
src/PVE/Tools.pm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
pve-manager:
Christoph Heiss (4):
api2: capabilities: explicitly import CPU capabilities module
api2: capabilities: proxy index endpoints to respective nodes
api2: capabilities: expose new qemu/migration endpoint
ui: window: Migrate: add checkbox for migrating VM conntrack state
PVE/API2/Capabilities.pm | 9 +++++
www/manager6/window/Migrate.js | 73 ++++++++++++++++++++++++++++++++--
2 files changed, 78 insertions(+), 4 deletions(-)
--
2.47.1
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel