From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id B4B5960A4A for ; Thu, 10 Sep 2020 10:59:22 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A12191A3D2 for ; Thu, 10 Sep 2020 10:58:52 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id B0E211A3C8 for ; Thu, 10 Sep 2020 10:58:51 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 7706F403CE for ; Thu, 10 Sep 2020 10:58:51 +0200 (CEST) To: pve-user@lists.proxmox.com References: <65499872-efcc-6f73-4b73-4039728c2a4e@matrixscience.com> <93d6fc90-f902-e33a-d2ee-b09daebc1d1b@proxmox.com> <6f14f3f9-87d2-f685-030a-a1b1a61de003@matrixscience.com> From: Fabian Ebner Message-ID: <0f1b6ce7-7983-6866-dcdc-1c16328646a6@proxmox.com> Date: Thu, 10 Sep 2020 10:58:45 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <6f14f3f9-87d2-f685-030a-a1b1a61de003@matrixscience.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 1.339 Adjusted score from AWL reputation of From: address KAM_ASCII_DIVIDERS 0.8 Spam that uses ascii formatting tricks KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -3.576 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [PVE-User] CT replication error X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Sep 2020 08:59:22 -0000 Hi, glad you were able to resolve the issue. Did you use 'purge' to remove the container? Doing that does not (yet) clean up the replicated volumes on the remote nodes. We're probably going to change that. Am 09.09.20 um 20:11 schrieb Adam Weremczuk: > Hi Fabian, > > Yes, all replication use the same shared zfs storage. > > Are you referring to /var/log/pve/replicate/102-0 file? > > It seems to only hold information about the last run. > > Anyway, my problem turned out to be the node2 still holding > zfs-pool/subvol-102-disk-0 of the previous container. > > I had deleted the old container from the web GUI before creating a new > one in its place (id 102). > > For some reason node2 still had the old disk. Once I rm'ed it from the > shell of node2 replication started working for CT-102. > > Regards, > Adam > > On 09/09/2020 08:51, Fabian Ebner wrote: >> Hi, >> could you check the replication log itself? There might be more >> information there. Do the working replications use the same storages >> as the failing one? >> >> Am 03.09.20 um 14:56 schrieb Adam Weremczuk: >>> Hi all, >>> >>> I have a dual host set up, PVE 6.2-6. >>> >>> All containers replicate fine except for 102 giving the following: >>> >>> Sep  3 13:49:00 node1 systemd[1]: Starting Proxmox VE replication >>> runner... >>> Sep  3 13:49:02 node1 zed: eid=7290 class=history_event >>> pool_guid=0x33A69221E174DDE9 >>> Sep  3 13:49:03 node1 pvesr[6852]: send/receive failed, cleaning up >>> snapshot(s).. >>> Sep  3 13:49:03 node1 pvesr[6852]: 102-0: got unexpected replication >>> job error - command 'set -o pipefail && pvesm export >>> zfs-pool:subvol-102-disk-0 zfs - -with-snapshots 1 -snapshot >>> __replicate_102-0_1599137341__ | /usr/bin/ssh -e none -o >>> 'BatchMode=yes' -o 'HostKeyAlias=node2' root@192.168.100.2 -- pvesm >>> import zfs-pool:subvol-102-disk-0 zfs - -with-snapshots 1 >>> -allow-rename 0' failed: exit code 255 >>> Sep  3 13:49:03 node1 zed: eid=7291 class=history_event >>> pool_guid=0x33A69221E174DDE9 >>> >>> Any idea what the problem is and how to fix it? >>> >>> Regards, >>> Adam >>> >>> >>> _______________________________________________ >>> pve-user mailing list >>> pve-user@lists.proxmox.com >>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> >> >> _______________________________________________ >> pve-user mailing list >> pve-user@lists.proxmox.com >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user