From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5A76A735CE for ; Thu, 15 Apr 2021 16:33:20 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 46A181B995 for ; Thu, 15 Apr 2021 16:32:50 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 20F211B98B for ; Thu, 15 Apr 2021 16:32:49 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id E236345AE5; Thu, 15 Apr 2021 16:32:48 +0200 (CEST) Date: Thu, 15 Apr 2021 16:32:41 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: alexandre derumier , Proxmox VE development discussion References: <20210413121640.3602975-1-f.gruenbichler@proxmox.com> In-Reply-To: MIME-Version: 1.0 User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid) Message-Id: <1618496842.5t56y2jruz.astroid@nora.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.026 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [pve-devel] [RFC qemu-server++ 0/22] remote migration X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2021 14:33:20 -0000 On April 15, 2021 4:04 pm, alexandre derumier wrote: > Hi, >=20 > thanks for working on this ! >=20 > I'll be able to test it soon as I'll need to migrate 200-300 vms between=20 > 2 datacenter soon. looking forward to feedback :) you'll need to put the=20 proxmox-websocket-tunnel binary into $PATH of pveproxy/qm, after=20 building it with 'cargo build'. if your inter-DC link is fast enough, you'll likely be hit by the=20 pveproxy bottleneck. it would still be interesting to get some=20 real-world numbers, I haven't tested with baremetal and fast storage=20 yet. please be aware that this is very much experimental code still! > I think it could be great to add optionnal "tag" option to targetbridge,=20 > as it could be different on target cluster. hmm, we could have another (optional) map for VLAN tags? since tags and=20 bridges are not one entity (you can have on interface on bridge A with=20 tag X, and another interface on bridge A with tag Y, and those need to=20 be mapped to bridge B with tag P and bridge B with tag Q, for example). > Also, we should transfert vm firewall config. yes, that's definitely true. another source of potential=20 mismatches/things to check before migrating (security groups/aliases!) >=20 > On 13/04/2021 14:16, Fabian Gr=C3=BCnbichler wrote: >> this series adds remote migration for VMs. there's still plenty of >> TODOs/FIXMEs/stuff that requires discussion, hence the RFC. live >> migration with NBD and storage-migrated disks should work already. >> >> the performance bottle neck (~190MB/s on loopback) for the websocket >> connection seems to be in pveproxy at the moment - the rust code should >> manage about 700MB/s. >> >> overview over affected repos and changes, see individual patches for >> more details. >> >> proxmox: >> >> some compatible changes to make websocket code usable for client-side >> connections, required by proxmox-websocket-tunnel >> >> proxmox-websocket-tunnel: >> >> new tunnel helper tool for forwarding commands and data over websocket >> connections, required by qemu-server on source side >> TODO: better error handling >> TODO: fingerprint checking/valid certs/.. >> TODO: WS key generation >> TODO: decide on mask? >> TODO: investigate performance bottlenecks once PVE api server gets >> faster >> >> pve-access-control: >> >> new ticket type, required by qemu-server on target side >> >> pve-cluster: >> >> new remote.cfg and related helpers, required by qemu-server on source >> side >> TODO: ACLs, CLI, API for managing config >> TODO: handling of discovered nodes with valid certificates >> TODO: add additional information like default bwlimits, storage/bridge >> mappings >> >> pve-common: >> >> bridgepair format akin to storage pair, pve-bridge-id option, required >> by qemu-server >> TODO: adapt pve-container >> >> pve-guest-common: >> >> handle remote migration (no SSH) in AbstractMigrate, >> required by qemu-server >> >> pve-manager: >> >> new 'addr' endpoint for retrieving remote node IPs, required on target >> node >> >> pve-storage: >> >> extend 'pvesm import' to allow import from UNIX socket, required on >> target node by qemu-server >> >> qemu-server: >> >> some refactoring, new mtunnel endpoints, new remote_migration endpoints >> TODO: check remote ACLs >> TODO: handle pending changes and snapshots >> TODO: CLI for remote migration >> potential TODO: expose remote info via additional endpoints (resources? = vmids? >> permissions? ...) >> >> as usual, some of the patches are best viewed with '-w', especially in >> qemu-server.. >> >> >> _______________________________________________ >> pve-devel mailing list >> pve-devel@lists.proxmox.com >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >> >=20 >=20 =