From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 07020CCAE for ; Mon, 10 Jul 2023 17:58:21 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DB7641A5F5 for ; Mon, 10 Jul 2023 17:58:20 +0200 (CEST) Received: from mail.velder.link (mail.velder.link [IPv6:2a06:8280:0:5::29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 10 Jul 2023 17:58:20 +0200 (CEST) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 914BB1A672E for ; Mon, 10 Jul 2023 17:58:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=velder.li; s=dkim; t=1689004699; h=from:subject:date:message-id:to:mime-version:content-type: content-language:in-reply-to:references; bh=H41Fie4hFChSn1dfomTA/Zppbel9dRQlFcsubCqkpu0=; b=E4fyM+bma9+cIh5C9wVq2ITNamVIp7peC7feuLs2SdAsV3IKwgxMNAqcH87aYOqI8uiBxT 4jrJ4VRk/B4mqI0hVLedNMF1KIL9j2kiIdJOoNfS+2WANCDiVsVemtqVsrsrqElxL3QQYT 0qkGEoif/71TJkIqayElSxRKmLQb7KhuZZpwN6rrX5dn43fLPuuug9SjRpfbdQBh35Nlq/ Nk14T7FFgowk4WeEN/u+d2YmE9TuEZ4msaNx0aRE47oUs5viCrDx7T2goNsJmTSZhNa6zj 4AmwrAOdFYD0pCiqvfASmRElV/zW84wUmRBeBwW1pv3LRIzrs7yEVietwSA8/g== Message-ID: Date: Mon, 10 Jul 2023 17:58:16 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 From: Patrick Velder To: pve-user@lists.proxmox.com References: Content-Language: en-US In-Reply-To: X-Last-TLS-Session-Version: TLSv1.3 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.045 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_MISSING 0.1 Missing DMARC policy HTML_MESSAGE 0.001 HTML included in message KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years NICE_REPLY_A -0.09 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [PVE-User] PVE Firewall IPset+Alias broken in v8 X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Jul 2023 15:58:21 -0000 Update: Upon further investigation, I discovered that the error message "value does not look like a valid IP address or CIDR network" also occurs on functioning PVE 7.xx systems. It appears that these messages are unrelated to the current issue. However, they can cause confusion when troubleshooting firewall-related problems and should also be addressed. The actual problem lies in the fact that when a global IP set is defined at the datacenter level, which includes aliases with the prefixes "dc/" or "guest/", the rules fail to work, also resulting in the following error messages: > no such alias 'xxx' > no such alias 'yyy' Best regards Patrick On 7/9/23 21:11, Patrick Velder wrote: > Hello, > > Since the upgrade to PVE 8, there appears to be a problem with the > combination of ipset and alias. When checking the firewall status > using the command "pve-firewall status," I receive the error message > "value does not look like a valid IP address or CIDR network" repeated > multiple times. Despite attempting to downgrade to > pve-firewall_4.3-2_amd64.deb, the issue remains unresolved. > > To further investigate and find a potential solution, I recommend > checking the following forum threads: > > * https://forum.proxmox.com/threads/pve-8-pve-firewall-status-no-such-alias.130202/ > * https://forum.proxmox.com/threads/ipset-not-working-for-accepting-cluster-traffic.129599/ > > Is that a known issue and is there maybe a workaround, since many > rules stopped working? > > Thanks and best regards > Patrick > >From elacunza@binovo.es Wed Jul 12 08:32:36 2023 Return-Path: X-Original-To: pve-user@lists.proxmox.com Delivered-To: pve-user@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id CEC5BCF9A for ; Wed, 12 Jul 2023 08:32:36 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id AC66729E28 for ; Wed, 12 Jul 2023 08:32:06 +0200 (CEST) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 12 Jul 2023 08:32:02 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-3159adb7cb5so719635f8f.0 for ; Tue, 11 Jul 2023 23:32:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=binovo.es; s=google; t=1689143515; x=1691735515; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=iAHh1ktA1SawSwBVC+nhIebtSt0CDaa0IWfS328cgCQ=; b=aBA9MxkqDyAhGdYynso4FiXNgzENYLToBaHG37fT4Zx3+z2Jv0Ldt226vSTEvm79Yr LzdehlrkP6fG2yIeCRYx3uJhiK8lJxIy86Hq22RJq9I7aqshbRKYOuo1ohABSZ9dEruk /R8kC79E00PZKQdtKUGYgEZLOCv3eYByUbi33IE/ILpiuOnTQ15RGGhCmEwfi2m4c2p0 0qQVCfJn2Sd3tykmEHkN+vyPNcnMbRyMqEnwjcC2910zFfKh5M8dOlUxM/aoZBwMBLGP y3z934wbCyCHdWMVnoFaxHBOdqIGZJ9D3lhNGLN6BnqVC+g/9ijMC6drzumtrd2im8Dn RlTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689143515; x=1691735515; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=iAHh1ktA1SawSwBVC+nhIebtSt0CDaa0IWfS328cgCQ=; b=ZydHvtTLHUsQVm7lgsc4OX6ezoP9G12QqrkwNwF+isjPDqKqB46ejzCwTQuI3iunkj FydgSqa0RvKef/uW5OnL+ivBciLdGa96gVv4uiGRXibBJB3h1SNrCtvoKjID7mhdCSTD oYI4hHTpCuCaLfn1r2WFLGCtTtdju05pxIrJYsif4x58JQp7ZZnZFF+I5IXZB+KUHLJu tZ7SiN4lpbkI20CyhYYM08L0DnK8xoYL/uHGHxxfBefHJ8KcTrWiLM3bQrOseYa9ymDx tfB1GD9diOk6EVgnr/NOW6AcOtrTJ6YR+oT1edtoCgxbWavVlPcZiH50yGM7eME/bmYZ 6wVg== X-Gm-Message-State: ABy/qLb2OdIEHtZTUKMkGMsrLRrgOca7R1u94QvMKiQ+YNATyZx9UUrt X2GOtUH6n9w3rkMiZU60ZYdMSU1skbPCRe7kyX4= X-Google-Smtp-Source: APBJJlG1QREfSvbTlVWU3vS18FZJIKSrnRZmoC+L42I3N72nobYUd9yzZq0w24k/SLFiYDPlj46F2w== X-Received: by 2002:a5d:5643:0:b0:314:c86:bee5 with SMTP id j3-20020a5d5643000000b003140c86bee5mr16774753wrw.2.1689143515354; Tue, 11 Jul 2023 23:31:55 -0700 (PDT) Received: from [192.168.11.42] (192.red-88-9-88.dynamicip.rima-tde.net. [88.9.88.192]) by smtp.gmail.com with ESMTPSA id f4-20020a5d4dc4000000b0031424f4ef1dsm4114002wru.19.2023.07.11.23.31.54 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 11 Jul 2023 23:31:54 -0700 (PDT) Message-ID: Date: Wed, 12 Jul 2023 08:31:53 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 To: pve-user@lists.proxmox.com Content-Language: es-CO From: Eneko Lacunza Subject: PVE 7.4 Windows VM hang Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - X-List-Received-Date: Wed, 12 Jul 2023 06:32:36 -0000 Hi all, We have been experiencing Windows VM hangs during the last weeks, in previously stable cluster/VMs. So far we have seen a Windows 7 (yes I know!) and a Windows 2016 Std guest crash with 100% multi-cpu use. A hard stop and start leaves guest working again. - Windows 7 guest has crashed several times:  * June, 18th  * June, 25th  * July 4th - Windows 2016 guest has crashed once:  * June 23th There are more guests in the cluster; 5 linux and 1 Windows 2016; none have crashed (all were running in other nodes). Both guest were running on the same physical node, a Dell T350 server with E-2356G and 32GB RAM. We have moved both to another node (with E3-1220 v6 CPU) just to see if the issue is with the node; no crashes since July, 5th but it's too soon to be sure. Storage is Ceph v16.2.13, backed by HDDs with SSD partition for journal/db (there is a mixture of filestore and bluestore OSDs). Originally node was running since April, 20th: * pve-kernel 6.2.9-1-pve * qemu-server 7.4-3 * pve-qemu-kvm 7.2.0-8 on June 26th we tried upgrading: * pve-kernel 6.2.11-2-pve * qemu-server 7.4-4 VM conf: boot: order=ide0;ide2 cores: 4 ide0: rbd:vm-104-disk-0,cache=writeback,size=30G ide2: nas-backups2:iso/virtio-win-0.1.126.iso,media=cdrom,size=152204K machine: pc-i440fx-6.2 memory: 12288 meta: creation-qemu=6.2.0,ctime=1651156825 name: Windows7 net0: virtio=7E:0E:12:3D:0A:89,bridge=vmbr0,firewall=1 numa: 0 onboot: 1 ostype: win10 parent: Snapshot_02_05_2022 scsi1: rbd:vm-104-disk-2,cache=writeback,size=40G scsihw: virtio-scsi-pci smbios1: uuid=1006a846-b056-4ce3-bd6f-7d16cfb5f573 sockets: 1 vmgenid: 27764cc2-dcab-405f-be80-bd6630ba75ba --- boot: cdn bootdisk: scsi0 cores: 4 ide2: none,media=cdrom ide3: none,media=cdrom memory: 8192 name: Windows2016 net0: virtio=3A:F9:3C:9F:70:FE,bridge=vmbr0 numa: 0 onboot: 1 ostype: win10 scsi0: rbd:vm-110-disk-1,cache=writeback,discard=on,size=110G scsi1: rbd:vm-110-disk-2,cache=writeback,discard=on,size=30G scsihw: virtio-scsi-pci smbios1: uuid=3e82d96a-464a-483d-b178-2359d24333db sockets: 1 Currently: # pveversion -v proxmox-ve: 7.4-1 (running kernel: 6.2.11-2-pve) pve-manager: 7.4-15 (running version: 7.4-15/a5d2a31e) pve-kernel-5.15: 7.4-4 pve-kernel-6.2: 7.4-3 pve-kernel-5.13: 7.1-9 pve-kernel-6.2.11-2-pve: 6.2.11-2 pve-kernel-6.2.9-1-pve: 6.2.9-1 pve-kernel-5.15.108-1-pve: 5.15.108-1 pve-kernel-5.15.104-1-pve: 5.15.104-2 pve-kernel-5.15.30-2-pve: 5.15.30-3 pve-kernel-5.13.19-6-pve: 5.13.19-15 ceph: 16.2.13-pve1 ceph-fuse: 16.2.13-pve1 corosync: 3.1.7-pve1 criu: 3.15-1+pve-1 glusterfs-client: 9.2-1 ifupdown2: 3.1.0-1+pmx4 ksm-control-daemon: 1.4-1 libjs-extjs: 7.0.0-1 libknet1: 1.24-pve2 libproxmox-acme-perl: 1.4.4 libproxmox-backup-qemu0: 1.3.1-1 libproxmox-rs-perl: 0.2.1 libpve-access-control: 7.4.1 libpve-apiclient-perl: 3.2-1 libpve-common-perl: 7.4-2 libpve-guest-common-perl: 4.2-4 libpve-http-server-perl: 4.2-3 libpve-rs-perl: 0.7.7 libpve-storage-perl: 7.4-3 libspice-server1: 0.14.3-2.1 lvm2: 2.03.11-2.1 lxc-pve: 5.0.2-2 lxcfs: 5.0.3-pve1 novnc-pve: 1.4.0-1 proxmox-backup-client: 2.4.2-1 proxmox-backup-file-restore: 2.4.2-1 proxmox-kernel-helper: 7.4-1 proxmox-mail-forward: 0.1.1-1 proxmox-mini-journalreader: 1.3-1 proxmox-offline-mirror-helper: 0.5.2 proxmox-widget-toolkit: 3.7.3 pve-cluster: 7.3-3 pve-container: 4.4-6 pve-docs: 7.4-2 pve-edk2-firmware: 3.20230228-4~bpo11+1 pve-firewall: 4.3-4 pve-firmware: 3.6-5 pve-ha-manager: 3.6.1 pve-i18n: 2.12-1 pve-qemu-kvm: 7.2.0-8 pve-xtermjs: 4.16.0-2 qemu-server: 7.4-4 smartmontools: 7.2-pve3 spiceterm: 3.2-2 swtpm: 0.8.0~bpo11+3 vncterm: 1.7-1 zfsutils-linux: 2.1.11-pve1 Any ideas? Thanks Eneko Lacunza Zuzendari teknikoa | Director técnico Binovo IT Human Project Tel. +34 943 569 206 | https://www.binovo.es Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun https://www.youtube.com/user/CANALBINOVO https://www.linkedin.com/company/37269706/