From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <pbs-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
	by lore.proxmox.com (Postfix) with ESMTPS id 924721FF15E
	for <inbox@lore.proxmox.com>; Fri,  9 Aug 2024 14:52:49 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 878323E02E;
	Fri,  9 Aug 2024 14:53:00 +0200 (CEST)
Message-ID: <ff31232c-7f29-4859-9204-6c579b2ef7a5@proxmox.com>
Date: Fri, 9 Aug 2024 14:52:56 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird Beta
To: Proxmox Backup Server development discussion
 <pbs-devel@lists.proxmox.com>, Max Carrara <m.carrara@proxmox.com>
References: <D3B9YIIHKRP2.2EX6MKWE3C0NP@proxmox.com>
Content-Language: en-US
From: Dominik Csapak <d.csapak@proxmox.com>
In-Reply-To: <D3B9YIIHKRP2.2EX6MKWE3C0NP@proxmox.com>
X-SPAM-LEVEL: Spam detection results:  0
 AWL 0.016 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DMARC_MISSING             0.1 Missing DMARC policy
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
 Validity was blocked. See
 https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
 information.
 RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
 Validity was blocked. See
 https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
 information.
 RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
 Validity was blocked. See
 https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
 information.
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
Subject: Re: [pbs-devel] RFC: Scheduler for PBS
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
 <pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, 
 <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox Backup Server development discussion
 <pbs-devel@lists.proxmox.com>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: pbs-devel-bounces@lists.proxmox.com
Sender: "pbs-devel" <pbs-devel-bounces@lists.proxmox.com>

Hi,

great to see that you tackle this!

I read through the overview, which sounds fine, but I think that it
should more reflect the actual issues, namely limitations in memory,
threads, disk io and network.

The actual reason people want to schedule things is to not overload the system
(because of timeouts, hangs, etc.) so any scheduling system should consider
not only the amount of jobs, but how much resources the the job will/can
utilize.

E.g. when I tried to introduce multi-threaded tape backup (configurable threads
per tape job), Thomas rightfully said that it's probably not a good idea, since
making multiple parallel tape backup job increases the load by much more than before.

I generally like the approach, but I personally would like to see some
work with resource constraints, for example one could imagine a configurable
amount of available threads and (configurable?) used thread by job type

so i can set my available to e.g. 10 and if my tape backup jobs then get
4, i can start 2 in parallel but not more

Such a system does not have to be included from the beginning IMO, but the
architecture should be prepared for such things

Does that make sense?


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel