From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 3E27A1FF15D for ; Thu, 19 Sep 2024 08:56:28 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 03378E80B; Thu, 19 Sep 2024 08:56:21 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Tue, 17 Sep 2024 07:50:20 +0200 In-Reply-To: <20240917055020.10507-1-sascha.westermann@hl-services.de> References: <20240917055020.10507-1-sascha.westermann@hl-services.de> X-Mailman-Approved-At: Thu, 19 Sep 2024 08:56:19 +0200 MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Sascha Westermann via pve-devel Precedence: list Cc: Sascha Westermann X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH qemu-server 3/3] Fix #5708: Add CPU raw counters Content-Type: multipart/mixed; boundary="===============3909298754889539854==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============3909298754889539854== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 6E2E1C36C6 for ; Tue, 17 Sep 2024 08:06:34 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5884710CC0 for ; Tue, 17 Sep 2024 08:06:34 +0200 (CEST) Received: from EUR03-VI1-obe.outbound.protection.outlook.com (mail-vi1eur03on20724.outbound.protection.outlook.com [IPv6:2a01:111:f403:260c::724]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 17 Sep 2024 08:06:33 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FggeQJMCd28f4HOjua9FG2TpONHGFZK4Vg8+kVSFxMOh8aGBnmSXm2KjD0graOQ3raday88fn9NTvolKnKBQlEfM/Y/HIWoMJn7PjL5kNqSkgaMTl29muPWtW8Ged5Mjzd52a8Dx+8Nzx9dCVODMiRpFjaRmWQfbwfjaJihnUSe1Y+OStFrynDIGU9WYd8kguK8y9+1JmNX2Htykd3h7HWcyFSvma2/d1WXtTQNPt+PLbVgPP8PjTYGNDuHcY1vioCHypzPs6aDqLwcPQYt0+Y4nzrWTkt9oX2rss+5JrB83pn3PmWwzhiTX2ZzvsLNXhrnyf1wYsZFg3BMj/TFJrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ngNAZ0dunao3aeK79blpdwyyGDHGWsVl4dftOBd+PXw=; b=RXe3hdo0yGrEEGdXj2WKKPEwl1Xjo+qKKjz2tA2FESPqFKjJYPDMec1Z4VAJ7iNpVsvczBsxpN06kBRdvFlwwVkSxyLXAng2/M4YoRdg0EGh6pxLH9/Qo7/AxmAaVlsSlkgKG5s2Q4/+luvLKIG9cqxSeI787RwNPOJfAkp29kng56KMy58qvUR+2tWdu2jn/nNyYR9zHj4A3nn/plSSwijZmZsvhgA7AyB8qmOjtGXeq2A1TmC/lz5+mVF4V38KCFM5UVXXtrvoBdSa1q2JOAByVHaXU5Paqz0FWD8TRoQQdvJ0OpEIpXdXu/UrcmW8hH6WJR3yVjqvcq1+OhYKUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=hl-services.de; dmarc=pass action=none header.from=hl-services.de; dkim=pass header.d=hl-services.de; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hl-services.de; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ngNAZ0dunao3aeK79blpdwyyGDHGWsVl4dftOBd+PXw=; b=c7Tc8bAgj9aFvGt3RFM5JTWf2GmI6Msd7do0gcyRL1EzHA3wikHgIC7AuJAPgYe9No4KBVIC/KwcV9eJ7AtQInc+zoJ+9Zu+y1RhOsFs0umWaDlVmd3OwKpACzD6unJUNJmrEr0FpGISlgmd55LAplorSb0bCnKHKmpvTz7qzsf511RBQUUwL098tYKnigKIVBNZCujLtwo8bmp+lU3bqa+tVTPCSMqvhabHZ/xr0HACjCKx55UQOwbmoHC3c80cGs3/CGM5Z+UOOsdKVSYs+bih4wbE1sred8mVT2zEOMUN8A7ySLeNway+umiXaAM39I3zcNqVgRxEkUbeUbfPDA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=hl-services.de; Received: from AM9PR10MB4885.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:419::5) by GV2PR10MB7031.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:150:d3::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.24; Tue, 17 Sep 2024 05:50:33 +0000 Received: from AM9PR10MB4885.EURPRD10.PROD.OUTLOOK.COM ([fe80::69f0:b468:92e7:7210]) by AM9PR10MB4885.EURPRD10.PROD.OUTLOOK.COM ([fe80::69f0:b468:92e7:7210%6]) with mapi id 15.20.7962.022; Tue, 17 Sep 2024 05:50:33 +0000 From: Sascha Westermann To: pve-devel@lists.proxmox.com Cc: Sascha Westermann Subject: [PATCH qemu-server 3/3] Fix #5708: Add CPU raw counters Date: Tue, 17 Sep 2024 07:50:20 +0200 Message-ID: <20240917055020.10507-4-sascha.westermann@hl-services.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240917055020.10507-1-sascha.westermann@hl-services.de> References: <20240917055020.10507-1-sascha.westermann@hl-services.de> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: AM0PR03CA0045.eurprd03.prod.outlook.com (2603:10a6:208::22) To AM9PR10MB4885.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:419::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4885:EE_|GV2PR10MB7031:EE_ X-MS-Office365-Filtering-Correlation-Id: 0805cba0-adae-435c-c670-08dcd6dca4d9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|10070799003|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?9T9IEQ7Zy79oHgkYaGNODTrwiLLHD4lkLTdyvcl5SBNHtGGjwn7MO57G1Rqr?= =?us-ascii?Q?z/nOcEf0F5hnoUFLOpXA9yZbMByUSgGjzhZ7VTPZpDQyrGlWZ09mPY5wHTp4?= =?us-ascii?Q?JRDc49SR7+6bGfxzCwvS9vZz8O6FVl880LJpkYypYe08QzmxQ1kr7edX+Sar?= =?us-ascii?Q?QJL4SggU6gXqds/GnbY7LUalGps9pxjvf3tAnRIhBJV+wv+bWdkLxav3OHlV?= =?us-ascii?Q?Ve6Hhg+iJ6WeMA/PsMcijYRvU2ca3sY9sPrnrNhdZjDQg2MUGYAdAJQW2n0R?= =?us-ascii?Q?EMTu/hIv52pHppKtTP6V0BoYacKFODGongfsI/IRjrfuo6n+fzl/lJKN44LN?= =?us-ascii?Q?7nQeOoIB4iQRinaCs6TdTS4lw2iqXweTiuAXqFcd8dApjy/u9Aq7GQNA1FHm?= =?us-ascii?Q?BgUZ+RRT02xyxGr+mr3xAAqPTcs4s9Ngd3aD0FAR6b3FURavJo+ZZyRQksrz?= =?us-ascii?Q?zCiF6ggcd++kUUe88jD+zFWI1hLZGDrrwFRi6DtQBvvjq7GpsqPQupUIhFRg?= =?us-ascii?Q?a+Wr5Tm+C2MoKF6LUKxAL6ntHMbz8IgO7HR8GXxGyo1YQXee7NJP+OMHc43Z?= =?us-ascii?Q?eB4tiN4tlNT/eGeVLP6GAcpLFEX/CdNVK30Tdmv9PFcG/OOzUublSjYNmdRE?= =?us-ascii?Q?/sSKsdSwrqCv/r0Ua0xB19TSFQPGn3Buh/4EhMLaWGgxUFOmhYi8bDJ4w4z0?= =?us-ascii?Q?ATuOVa3oXv+GDVMjsgkPTnb8V1TaIZNUqWJDE6aw9Kh04y3feOrp7FMgz+BF?= =?us-ascii?Q?zFUZtRFPAktDhixWFXDCGNxzrnxeQXpwYMixFsomr4OF5v4Q6DlMqSpO21Wn?= =?us-ascii?Q?ZjlA7KISFfwzepMMCsxviC0gnlJqiMHBGkkAl5niFdgic2awg56RxafMsv0E?= =?us-ascii?Q?YoyQ8br0yPS6ajXAXNl22m5BjaKqxrbqNfDPpHG5Cj9rPU6ypPFyNWdyP9fW?= =?us-ascii?Q?BXXlFjDOe5vFyFfoc5DeNPne/p7CTr7xLsvM8pDUclj9h+oEnNlVWmoAMEnL?= =?us-ascii?Q?t/sBGbpQ/XYRIIRfa4VOpUjzxhQQEKjuNLa2MACAlNfk+9lYwtlDbZyVNcsW?= =?us-ascii?Q?gdAxrDd8Azu9kVdCrDLOBcaotzHDi/kAhdKPJd0koCLXafXOPrMmT0mFRKdZ?= =?us-ascii?Q?mvFTYNyI0A8m1B3Uj3rpB0yGmLuwAOI82FRDAATsKU9NEsXnwiV6+PH2IGuH?= =?us-ascii?Q?KA7l+PXIIt5jYQFqpUkFGcpJaI95qe+Ly2cgfFw/W76t+lUmhkmONdNYeT6e?= =?us-ascii?Q?S+JBFRf4HkvaKgI9+3pGkVqX0Ex2y5i85qu2vdrC933itJsSzuDjTA03Wn2E?= =?us-ascii?Q?iN/0YaKD94RkKT1r9WlB/xivjEpth6I41DJnrEDkL284/A=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4885.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(10070799003)(376014)(1800799024)(366016);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?PDgU1terg86J3rB039tERRQIxlwlP5GjZ66hCiJRO2w2NLFVSBRcyL+tkzXy?= =?us-ascii?Q?oJgWoQPJ+Qc8OlMYTS5qoBGu3GhY6YFu3Msj5jPS8Iqbq3Y91gfdBXr/vrn1?= =?us-ascii?Q?8EtXu7uyOKYprBB92fhWxJUs8vSyxGFoVDuZAqpIVR0+Jb4aF3W7qoxQKmK0?= =?us-ascii?Q?7FwyI2a6Pd+qouGeoIWjKlhTHV8xfyGJenHIX6msGxdgRRFmLDzCuluJttKP?= =?us-ascii?Q?Yz84a8DCZuNh4sk20POzCjYm87FFLR95oSdSKmnyFesN+buAwE40iuDv5GB7?= =?us-ascii?Q?pp9Mjl4k3qHY02hSvpXVYbGq8dsPvtrKQ7qVQs8HNdP9WntRwCAyKff0Mrtk?= =?us-ascii?Q?8RWzGilKhj5EWFi5cagwmPmuWmQa6IXeuu7rpT7ueKldtdGVoRF0xkiCz+Hd?= =?us-ascii?Q?yHFjV8KE4OmoTkhDlz4IsuYVID9nvcCSCsiCcnJiHDpfaZm5ax125VnfAb7b?= =?us-ascii?Q?+XRndaa3vCYvg1BLbaHkAlNc3ebp6ZG6hY+N0AcSOkUJjZtlz3FKO8ZMps5x?= =?us-ascii?Q?cAWWjnPaIkaWYCKO7r4Be0nn8N5+vdyGvEM+pNuVmkAd4TAwUsTsxljI9MK0?= =?us-ascii?Q?qr5Nq2Dq0tvklADsGh+B4mDwIzAGuVVERgUoqkzF81S3Q+kUsXNTDWj89QRg?= =?us-ascii?Q?95H9U+CdB7iirlzHpBgyDv3GnAIa0BFCPVGk+yLguF0ZYNL6mCXO8SyYVc+Z?= =?us-ascii?Q?5kYP4Rm6RJ/kN0FY79NJ0JNQPlNH0GTymdHWI5rNLjULIbpYi6kwQ3ucovBS?= =?us-ascii?Q?MfosFknLBzuy2zR+c8514jT0f0Xvx90D1m2H3lHEKpYbK58zErFOmbyu/xP+?= =?us-ascii?Q?fcYUIzq6PF1lpp2R8I9pP7B/zJfSgiNT8UTlJte89Xuzy4bwmh4ZC/ef4OTg?= =?us-ascii?Q?Vnq03vYIWpqSGl6GKU5oxwU3CEEERT/ZgCCGl11tfKdRQgBbemvJx/Qatpab?= =?us-ascii?Q?rEE0VIpci15ErpR1jVUuP7+p9Kz6yFbWNJ10EvzSkKHzhbJbgSjrwykvYhQ+?= =?us-ascii?Q?mgk8DJC4bZruq5PTTc6gTe7x52WjhMEf6tkx3CBuKLzKEXJjNzbQF4Jul7TL?= =?us-ascii?Q?WE7K/wr8CbHv+yObwKGj1QJC9r8P90lTtULWLjDZQjoUK5PnQiuk6D1SLC38?= =?us-ascii?Q?qL2IelzICTQ409Ukncgnaq2jXp8mY9uwDOrKqyvM0HD4DYmHy6PtgyuuEzQ6?= =?us-ascii?Q?T7uJvCiEncM4qWUvoTTYQQIQ4xNfAHak+T7uPAwBqX9NBOxtn7qvRxa4lMBY?= =?us-ascii?Q?gjlVebTnKsiLn/WDk9ATmn1SxtEFaRLdAy34+jW4I503D2Sqhrj22E7fx+CB?= =?us-ascii?Q?3V3RUgs9po/pK766+U4g4wMSy/gKvCwZ/ANqRe63B/RLTfBOj2P2BVQESpgJ?= =?us-ascii?Q?SNkRO0t+aRjfqaQRMzbLMJcCr71hoVj3pcwVGhjX4vdDiYhG5313eZwXKPt4?= =?us-ascii?Q?IkObjPYyM+h4WN/8mm1LD8AjHLiEFRcULA3QA5aH/qvuEbvjJ/V3uTUI3h6I?= =?us-ascii?Q?2lG8MjnOQv4To/w9rpZXtglRdaVAKLSH/WOYSZJbUdMV5N35HcQHa3HYV85o?= =?us-ascii?Q?05Lz/yvs3/oq3K5Aacck4txSTWrCCMWSwzssg1mA9XgXeU1unHuASJoQI/a2?= =?us-ascii?Q?SytNoVq+Jznwh4TnK3fTDqJM49hY7kHrGmSFqrxwaatJoGL3iF+0qqM0ZVFs?= =?us-ascii?Q?JGPIIUpNk5geyXfvjhfdjrERTIQ=3D?= X-OriginatorOrg: hl-services.de X-MS-Exchange-CrossTenant-Network-Message-Id: 0805cba0-adae-435c-c670-08dcd6dca4d9 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4885.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2024 05:50:32.8048 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 996ee4f0-343c-4038-abc8-fd1e1094eeb5 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4gOPpKBQ1SKn/ey3WfD5vBUytGsJ6jiZH+CdallUsZgVUFGejkDzy4EeZ9i+5y6XFQeX9lJ/kdeJRklbh3Hqj3YKkUXqwJAwpuomNuzoX1o9w5Z8KpZBpnEZ1R1FrWQa X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR10MB7031 X-SPAM-LEVEL: Spam detection results: 0 ARC_SIGNED 0.001 Message has a ARC signature ARC_VALID 0.001 Message has a valid ARC signature AWL 0.001 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record X-Mailman-Approved-At: Thu, 19 Sep 2024 08:56:19 +0200 Add a map containing raw values from /proc//stat (utime, stime and guest_time), "uptime_ticks" and "user_hz" (from cpuinfo) to calcuate physical CPU usage from two samples. In addition, virtual CPU statistics based on /proc//task//schedstat ( for virtual cores) are added - based on this data, the CPU usage can be calculated from the perspective of the virtual machine. The total usage corresponds to "cpu_ns + runqueue_ns", "cpu_ns" should roughly reflect the physical CPU usage (without I/O-threads and emulators) and "runqueue_ns" corresponds to the value of %steal, i.e. the same as "CPU ready" for VMware or "Wait for dispatch" for Hyper-V. To calculate the difference value, uptime_ticks and user_hz would be converted to nanoseconds - the value was determined immediately after utime, stime and guest_time were determined from /proc//stat, i.e. before /proc//task//schedstat was determined. The time value is therefore not exact, but should be sufficiently close to the time of determination so that the values determined should be relatively accurate. Signed-off-by: Sascha Westermann --- PVE/QemuServer.pm | 55 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 53 insertions(+), 2 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index b26da505..39830709 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -2814,6 +2814,40 @@ our $vmstatus_return_properties = { my $last_proc_pid_stat; +sub get_vcpu_to_thread_id { + my ($pid) = @_; + my @cpu_to_thread_id; + my $task_dir = "/proc/$pid/task"; + + if (! -d $task_dir) { + return @cpu_to_thread_id; + } + + opendir(my $dh, $task_dir); + if (!$dh) { + return @cpu_to_thread_id; + } + while (my $tid = readdir($dh)) { + next if $tid =~ /^\./; + my $comm_file = "$task_dir/$tid/comm"; + next unless -f $comm_file; + + open(my $fh, '<', $comm_file) or next; + my $comm = <$fh>; + close($fh); + + chomp $comm; + + if ($comm =~ /^CPU\s+(\d+)\/KVM$/) { + my $vcpu = $1; + push @cpu_to_thread_id, { tid => $tid, vcpu => $vcpu }; + } + } + closedir($dh); + + return @cpu_to_thread_id; +} + # get VM status information # This must be fast and should not block ($full == false) # We only query KVM using QMP if $full == true (this can be slow) @@ -2827,8 +2861,6 @@ sub vmstatus { my $list = vzlist(); my $defaults = load_defaults(); - my ($uptime) = PVE::ProcFSTools::read_proc_uptime(1); - my $cpucount = $cpuinfo->{cpus} || 1; foreach my $vmid (keys %$list) { @@ -2911,6 +2943,25 @@ sub vmstatus { my $pstat = PVE::ProcFSTools::read_proc_pid_stat($pid); next if !$pstat; # not running + my ($uptime) = PVE::ProcFSTools::read_proc_uptime(1); + my $process_uptime_ticks = $uptime - $pstat->{starttime}; + + $d->{cpustat}->{guest_time} = int($pstat->{guest_time}); + $d->{cpustat}->{process_uptime_ticks} = $process_uptime_ticks; + $d->{cpustat}->{stime} = int($pstat->{stime}); + $d->{cpustat}->{user_hz} = $cpuinfo->{user_hz}; + $d->{cpustat}->{utime} = int($pstat->{utime}); + + my @vcpu_to_thread_id = get_vcpu_to_thread_id($pid); + if (@vcpu_to_thread_id) { + foreach my $entry (@vcpu_to_thread_id) { + my $statstr = PVE::Tools::file_read_firstline("/proc/$pid/task/$entry->{tid}/schedstat") or next; + if ($statstr && $statstr =~ m/^(\d+) (\d+) \d/) { + $d->{cpustat}->{"vcpu" . $entry->{vcpu}}->{cpu_ns} = int($1); + $d->{cpustat}->{"vcpu" . $entry->{vcpu}}->{runqueue_ns} = int($2); + }; + } + } my $used = $pstat->{utime} + $pstat->{stime}; -- 2.46.0 --===============3909298754889539854== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============3909298754889539854==--