From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 924721FF15E for ; Fri, 9 Aug 2024 14:52:49 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 878323E02E; Fri, 9 Aug 2024 14:53:00 +0200 (CEST) Message-ID: Date: Fri, 9 Aug 2024 14:52:56 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta To: Proxmox Backup Server development discussion , Max Carrara References: Content-Language: en-US From: Dominik Csapak In-Reply-To: X-SPAM-LEVEL: Spam detection results: 0 AWL 0.016 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] RFC: Scheduler for PBS X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" Hi, great to see that you tackle this! I read through the overview, which sounds fine, but I think that it should more reflect the actual issues, namely limitations in memory, threads, disk io and network. The actual reason people want to schedule things is to not overload the system (because of timeouts, hangs, etc.) so any scheduling system should consider not only the amount of jobs, but how much resources the the job will/can utilize. E.g. when I tried to introduce multi-threaded tape backup (configurable threads per tape job), Thomas rightfully said that it's probably not a good idea, since making multiple parallel tape backup job increases the load by much more than before. I generally like the approach, but I personally would like to see some work with resource constraints, for example one could imagine a configurable amount of available threads and (configurable?) used thread by job type so i can set my available to e.g. 10 and if my tape backup jobs then get 4, i can start 2 in parallel but not more Such a system does not have to be included from the beginning IMO, but the architecture should be prepared for such things Does that make sense? _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel