From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 00AF260FA8 for ; Wed, 2 Dec 2020 09:47:57 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id BFA2918AED for ; Wed, 2 Dec 2020 09:47:27 +0100 (CET) Received: from smtp12.dentaku.gol.com (smtp12.dentaku.gol.com [203.216.5.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id A5BB118AE3 for ; Wed, 2 Dec 2020 09:47:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gol.com; s=mail; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Subject:To:From:Date; bh=d2VZcb7/stveeoe7/h7mj7H/BqAieZXRnTTNk2sNp94=; b=aJ/y+pi/ZqGrn5soQuJiifOWVVlPIF6x35uZ0Wz4AVNdcKP5HB4cqtahZVUQNT2+Cn1voSJW2wmvlrVXbcj1G/rGD80QHyJccY6ABPIciwPNg2iYxyCxz34x9DeAeq7aQEKFpL6sUbdEIA0n+My8ulKsby2PMhUCtdEOXuuIb0fn2i1frVWAA0TLekZLaNART4zYMC0EAZV6vCGKU4cfRma/L+bCPgOm1EPyQcwn0xghFLDHZXfU/p/3oVAxWQp0OKZnSxB/AdCltmrjSHoPqajc9cggQICxuyo0BmhrkEUVn1BOAVgpS4hroLOYODrAwRov9+FjQ+JbS/aeaHcTRw==; Received: from batzmaru.gol.ad.jp ([203.216.0.80]) by smtp12.dentaku.gol.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Dentaku) (envelope-from ) id 1kkNnF-0003JO-7A for pve-user@lists.proxmox.com; Wed, 02 Dec 2020 17:47:24 +0900 Date: Wed, 2 Dec 2020 17:47:18 +0900 From: Christian Balzer To: Proxmox VE user list Message-ID: <20201202174718.0bdf3912@batzmaru.gol.ad.jp> In-Reply-To: <20758633.47066.1606897064143.JavaMail.zimbra@fws.fr> References: <20201202171102.03c7f2e2@batzmaru.gol.ad.jp> <20758633.47066.1606897064143.JavaMail.zimbra@fws.fr> Organization: Rakuten Communications X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: ClamAV GOL (outbound) X-GOL-Outbound-Spam-Score: -1.9 X-Abuse-Complaints: abuse@gol.com X-SPAM-LEVEL: Spam detection results: 0 DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust RCVD_IN_MSPIKE_H3 0.001 Good reputation (+3) RCVD_IN_MSPIKE_WL 0.001 Mailspike good senders SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [fws.fr, gol.com, firewall-services.com, proxmox.com] Subject: Re: [PVE-User] Live VM migration 6.3 to 6.2 fails X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 08:47:58 -0000 On Wed, 2 Dec 2020 09:17:44 +0100 (CET) Daniel Berteaud wrote: > ----- Le 2 D=C3=A9c 20, =C3=A0 9:11, Christian Balzer chibi@gol.com a =C3= =A9crit : >=20 > > Hello, > >=20 > > yes I know that migrating back to an older version is not guaranteed but > > I'm rather confused as to why this happens. > >=20 > > Firstly this, more details below: > > --- > > 2020-12-02 16:34:08 start migrate command to tcp:10.0.0.11:60000 > > 2020-12-02 16:34:09 migration status error: failed > > 2020-12-02 16:34:09 ERROR: online migrate failure - aborting > > 2020-12-02 16:34:09 aborting phase 2 - cleanup resources > > 2020-12-02 16:34:09 migrate_cancel > > 2020-12-02 16:34:11 ERROR: migration finished with problems (duration 0= 0:00:08) > > TASK ERROR: migration problems > > --- > >=20 > > That's all I get, where would one find the exact issue? > >=20 > > Now for the details, in the previous minor upgrade we saw a similar iss= ue, > > but that only affected VMs which had been created on an upgraded node, = old > > ones were fine to move in both ways. > > OK, that was because of new features int he VM definition not present on > > the old version, understandable. > >=20 > > With this one on a (more frequently updated) test cluster 6.3 to 6.2 > > migrations also works, the test cluster old nodes are at > > pve-manager (6.2-12), the one with issues are pve-manager (6.2-6). > > I however see nothing in the changelogs which would explain this differ= ence. > >=20 > > In general being able to live migrate back to a node with the previous > > version is a very desirable situation, since if there are issues/bugs (= on > > the VM level) with the new version there no longer is an impact free way > > of reverting if this functionality is not present. > > =20 >=20 >=20 > That's most likely the QEMU version 5.1 on the 6.3 node vs QEMU 5.0 on th= e 6.2 one. > There's no migration support from newer to older, only from older to newe= r. >=20 If that is really the case (qemu migrations between minor versions have worked in the past, at least sometimes), then this is actually worse. As I wrote, some 6.2 nodes don't have that issue and indeed they have qemu 5.1 which was brought in with 6.2-13.=20 So no version upgrade flashing warning lights going off in your brain and if one had not upgraded all nodes, the same would have happened in a cluster that was nominally still 6.2? That's rather... scary. Christian > Cheers, > Daniel >=20 > --=20 > [ https://www.firewall-services.com/ ] =09 > Daniel Berteaud=20 > FIREWALL-SERVICES SAS, La s=C3=A9curit=C3=A9 des r=C3=A9seaux=20 > Soci=C3=A9t=C3=A9 de Services en Logiciels Libres=20 > T=C3=A9l : +33.5 56 64 15 32=20 > Matrix: @dani:fws.fr=20 > [ https://www.firewall-services.com/ | https://www.firewall-services.com ] >=20 >=20 > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user --=20 Christian Balzer Network/Systems Engineer =20 chibi@gol.com Rakuten Mobile Inc.