From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 6ED4B7B32B for ; Fri, 8 Jul 2022 11:50:04 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 64A93271B8 for ; Fri, 8 Jul 2022 11:50:04 +0200 (CEST) Received: from mail.corep.it (mail.corep.it [93.186.252.128]) by firstgate.proxmox.com (Proxmox) with ESMTP for ; Fri, 8 Jul 2022 11:50:02 +0200 (CEST) Message-ID: Date: Fri, 8 Jul 2022 11:40:40 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.0.1 To: pbs-devel@lists.proxmox.com References: <94e2e388-2d57-dd62-1063-585e7af2ab38@gmail.com> From: dea In-Reply-To: <94e2e388-2d57-dd62-1063-585e7af2ab38@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_ASCII_DIVIDERS 0.8 Spam that uses ascii formatting tricks KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: Re: [pbs-devel] Scheduler causing connectivity issues? X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2022 09:50:04 -0000 Yes, same behavior for me too. release 2.2-3 is problematic (NFS storage connected to PBS) Previous releases work fine Thanks Luca Il 08/07/22 11:36, Jorge Boncompte ha scritto: > El 8/7/22 a las 9:09, Thomas Lamprecht escribió: >> Hi, >> >> On 07/07/2022 17:49, Mark Schouten wrote: >>> We’re getting complaints that one of our PBS’es is periodically unreachable. After investigation if the network might be at fault (even though it’s handling about 5.5Gbit at night), we found that PBS is piling up waiting connections every minute, on the minute, as you can see below. You see the output of `date`, combined with `ss -np | grep -c 8007`, the number of active connections. >>> >>> At first I thought that pvestatd was ddossing PBS, but pvestatd seems to run more often than once in a minute. >>> >>> So stracing the API process, I found that that process is also just waiting for something; must be the proxy-process. >>> >>> grepping for ‘minute’ in the code, I stumbled upon the function `next_minute` in ./src/bin/proxmox-backup-proxy.rs. I’m not quite sure if I understand it correctly, but it seems that every minute, the scheduler is going to try and find out if it should be doing something. >>> >>> Drilling down on that in my strace-foo, I think I see quite some read/write/rename actions on jobstate-files. Which leads me to conclude that the proxy process is waiting for the scheduler.. >>> >>> This is just guess-work, but you guys can surely find out better what’s going on than me. >>> >>> This PBS is running with 45 users and 67 datastores. >>> >>> Hope you guys can find something.. If I need to debug anything, let me know! >> Thanks for the info, this already helps quite a bit. We'll look into it and re-check >> with you if we need more info. > Hi, We've been having a problem that resembles this one with several > proxmox-backup-server 2.2.x versions. Our PBS stopped accepting backups > jobs sometimes, but if we retried manully they started fine. The only > message I could find was: > > backup failed: could not activate storage 'XXXXXXX': XXXXXX: error > fetching datastores - 500 Can't connect to XXXXXXXX:8007 > > Restarting the proxy seemed to help to get it working one or two days > more. We have reverted proxmox-backup-server to 2.1.8-1 and every is > fine again. > > Regards. > >> cheers, >> Thomas >> >> >> _______________________________________________ >> pbs-devel mailing list >> pbs-devel@lists.proxmox.com >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel > > _______________________________________________ > pbs-devel mailing list > pbs-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel