From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <martin.konold@konsec.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 1D55793FAD for <pve-user@lists.proxmox.com>; Mon, 9 Jan 2023 13:56:05 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DD90E2D965 for <pve-user@lists.proxmox.com>; Mon, 9 Jan 2023 13:56:04 +0100 (CET) Received: from mx0.konsec.net (hbase53.h.konsec.com [88.99.94.78]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for <pve-user@lists.proxmox.com>; Mon, 9 Jan 2023 13:56:02 +0100 (CET) Received: from kolab.konsec.com (unknown [10.21.1.190]) by mx0.konsec.net (Postfix) with ESMTPS id D7DF285C112 for <pve-user@lists.proxmox.com>; Mon, 9 Jan 2023 13:46:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsec.com; s=201809; t=1673268370; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7mpLWkqFWxkMJKrtyA/vdWeJGT+Cq8559GQIbufVXPY=; b=vl6QVhTlhC0+e1MjYRjRyt4JeUYUzCp+OdGkLaQRfNVVdkyai2UuS2jizYGze+tsvCt/tf vVYYlYUPAmqQD3wMq3yiMPMKZOXMfEY0fQrJaIlTxbzv6VSE6GXR5X2xipJanArsJPA62B vc7/T2mI4H/Pfq6jyMSdOUh+jJhoombZCGzzDPpd7V0TjJq5jBwJaF/68FK0lRfLWEUmD+ MUsgj6nI1DFQkbUArvemGFbcXe9H7j2MOKnDn9PZfFlm54haXE/JrelgDO9eG6q2otqXP2 LANm/bIapDOP0WIy2f3+IgBQhLlymJia+jJ9JXTqr5yZOvRoX5JjJxtvveyhuQ== Sender: martin.konold@konsec.com MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Date: Mon, 09 Jan 2023 13:46:09 +0100 From: "Konold, Martin" <martin.konold@konsec.com> To: Proxmox VE user list <pve-user@lists.proxmox.com> Reply-To: martin.konold@konsec.com In-Reply-To: <1683aac607c0ddcd12a54ceb06446c3b@shelldog.de> References: <1683aac607c0ddcd12a54ceb06446c3b@shelldog.de> Message-ID: <15cd6766a06066f1f6047d753192d788@konsec.com> X-Sender: martin.konold@konsec.com Organization: KONSEC GmbH Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.486 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain JMQ_SPF_NEUTRAL 0.5 SPF set to ?all SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [PVE-User] Failed backups of stopped machines X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list <pve-user.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-user/> List-Post: <mailto:pve-user@lists.proxmox.com> List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Mon, 09 Jan 2023 12:56:05 -0000 Hi Sven, how many nodes are in your pve cluster? I am experiencing the same as all nodes seem to sequentially backup=20 their vms but the nodes run in parallel. This can overwhelm the PBS. --- Regards ppa. Martin Konold -- Martin Konold - Prokurist, CTO KONSEC GmbH -=E2=81=A0 make things real Amtsgericht Stuttgart, HRB 23690 Gesch=C3=A4ftsf=C3=BChrer: Andreas Mack Im K=C3=B6ller 3, 70794 Filderstadt, Germany On 2023-01-09 13:08, Sven wrote: > Hello, >=20 > in the last few days, there have been some backup errors. The log is > always the same: >=20 >> INFO: Starting Backup of VM 147 (qemu) >> INFO: Backup started at 2023-01-06 02:55:37 >> INFO: status =3D stopped >> INFO: backup mode: stop >> INFO: ionice priority: 7 >> INFO: VM Name: <censored> >> INFO: include disk 'virtio0' 'ceph_disks:vm-147-disk-0' 10G >> INFO: include disk 'virtio1' 'ceph_disks:vm-147-disk-1' 20G >> INFO: include disk 'virtio2' 'ceph_disks:vm-147-disk-2' 150G >> INFO: include disk 'virtio3' 'ceph_disks:vm-147-disk-3' 4G >> INFO: creating Proxmox Backup Server archive=20 >> 'vm/147/2023-01-06T01:55:37Z' >> INFO: starting kvm to execute backup task >> ERROR: VM 147 qmp command 'backup' failed - got timeout >> INFO: aborting backup job >> INFO: stopping kvm after backup task >> ERROR: Backup of VM 147 failed - VM 147 qmp command 'backup' failed -=20 >> got timeout >> INFO: Failed at 2023-01-06 02:58:20 >=20 > Status: and config: >=20 >> $ qm status 147 --verbose >> cpus: 4 >> disk: 0 >> diskread: 0 >> diskwrite: 0 >> maxdisk: 10737418240 >> maxmem: 8589934592 >> mem: 0 >> name: <censored> >> netin: 0 >> netout: 0 >> qmpstatus: stopped >> status: stopped >> uptime: 0 >> vmid: 147 >=20 >> $ qm config 147 >> boot: order=3Dnet0;virtio0 >> cores: 4 >> ide2: none,media=3Dcdrom >> memory: 8192 >> name: <censored> >> net0: virtio=3DF2:F3:75:8D:92:9E,bridge=3Dvmbr0,tag=3D1 >> numa: 0 >> ostype: l26 >> scsihw: virtio-scsi-pci >> smbios1: uuid=3D743f558d-823a-4213-baa3-558a663aa273 >> sockets: 1 >> virtio0: ceph_disks:vm-147-disk-0,size=3D10G >> virtio1: ceph_disks:vm-147-disk-1,size=3D20G >> virtio2: ceph_disks:vm-147-disk-2,size=3D150G >> virtio3: ceph_disks:vm-147-disk-3,size=3D4G >> vmgenid: 90e25752-16b4-431d-92fe-8d71b364fb63 >=20 > Proxmox-Version: 7.2-1 > Version pve-qemu-kvm: 7.0.0-4 > Version of proxmox-backup-server: 2.2.5-1 >=20 > There are several VMs failing each night, not always the same. There > is an accumulation of failed backups of stopped VMs. >=20 > Does anybody has an idea? What timed out (connection to backup server,=20 > ...)? >=20 > Thanks! > Regards, > Sven >=20 > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user