From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id CEBBD1FF2A8 for ; Tue, 2 Jul 2024 12:37:40 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 6267B1B824; Tue, 2 Jul 2024 12:37:53 +0200 (CEST) To: Fiona Ebner , "pve-devel@lists.proxmox.com" , "f.gruenbichler@proxmox.com" Date: Tue, 2 Jul 2024 10:37:41 +0000 MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Rovshan Pashayev via pve-devel Precedence: list Cc: Rovshan Pashayev , Pavel Tide X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: Re: [pve-devel] =?windows-1252?q?Strange_Behavior_=96_ZFS_=28and_oth?= =?windows-1252?q?er=29_Storage_Configuration_in_PVE_=28plain_text=29?= Content-Type: multipart/mixed; boundary="===============7358229867838168892==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============7358229867838168892== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8BD0CBBB48 for ; Tue, 2 Jul 2024 12:37:52 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 656C01B7C1 for ; Tue, 2 Jul 2024 12:37:52 +0200 (CEST) Received: from mx1.veeam.com (mx1.veeam.com [216.253.77.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 2 Jul 2024 12:37:50 +0200 (CEST) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02lp2041.outbound.protection.outlook.com [104.47.57.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.veeam.com (Postfix) with ESMTPS id 16FFC802FC; Tue, 2 Jul 2024 06:37:44 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=veeam.com; s=mx1-2022; t=1719916666; bh=AQN5KNO7lghMqwdsXkMK3vEbS3otTGztrzw+bxa78xA=; h=From:To:CC:Subject:Date:From; b=IgDIKIlvasfIM0AHzURSfsItvdkgmY84lvJsn4PyruxPxJlYrmAVVzk5d1nDJi3Jc gg+PAtVD0o4EhGyBlYAV3qXtqg8l7YahAzaFWE7y7V+n9TgSnXQLWku+zt1+E0f7Ob W8a+/brtqcwjwt6WQm+ASu1nfgGyGrjQQT9SoTOxi+xCnhdWZmAhkmEMgNXyQ/wetE 4Qu2EIj3P9Qwsbb9TdtPCNt/VCHH7pqe1iOaPGaJdOlPCHPBDdjqXjSRpq5frDfjEt l3vtHTpfhQww9JeFqFJ5D8pPXRihmG5l7qyUDhPsfQ6LwSPZSIJx8xxZsCwN1Cu+gQ 9Jx9rl32zdhAA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kqDyv6sHD9FqE/1zVQzEdgYYhgdYqqzD2DAszhgkUqJgOKVJD9AJMs3Y6ZsAyFL0xBelvL5Ky/tSiHL0iYbYtXOZeZImpqS2T/blm29Es+9OdMfoPXf9WYDnT6nBQgwFadg3pThM7SvnUZYs0oXADqj1RHYQ+V2SIFAi3yy5uFpTXrmKiLcN97r55ChCtAViS8Bt+32ax5E3kO3+3suAdUFxuP8JQWrMEnkA5v+SiCdY7Xuz22twHr4i9Jf6oXLA53gSx6iWEPWIwiU/emz1u3dAZ+HhHjBSTb5lk240Ws2FqHHDHfrz/zeH1AHLPw6ru+Nr9UIrLdCbeKTTyEV0hQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AQN5KNO7lghMqwdsXkMK3vEbS3otTGztrzw+bxa78xA=; b=WGPLHcyJzEGl+BZ2YfhwYcIpq2RVrqmNU0oqqwMEitX9s/p6NGu43E9a9BK1gy6uOpPpLuDe75tFl2OmhEqetvOZAHLAlJG9RhrajlNXwfCS7cQg9UJ8md6tFJCoP/E7duqOf00H9aCvw5quiudouFxxN+Kcp15SI+yZMrnJWWX7ktMVA9XLJVgDF0w8WkVOCSf7Z5JIrURzy/BUmPEsi4jp6xabXXFF2M2qrsLvAQ1DRwsF5Svb2BeL4kpUlaQ50t7B4EDvcYM/9PLThNFL7ch6FGv8qIz0dKyaibIZ5fSvKpZFkhizu+tjERg1S/eSRLZUqL2bjyfiyL7vCSA/bw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=veeam.com; dmarc=pass action=none header.from=veeam.com; dkim=pass header.d=veeam.com; arc=none Received: from PH0PR14MB4735.namprd14.prod.outlook.com (2603:10b6:510:85::9) by PH7PR14MB6453.namprd14.prod.outlook.com (2603:10b6:510:270::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.18; Tue, 2 Jul 2024 10:37:41 +0000 Received: from PH0PR14MB4735.namprd14.prod.outlook.com ([fe80::f4fe:8794:4784:c224]) by PH0PR14MB4735.namprd14.prod.outlook.com ([fe80::f4fe:8794:4784:c224%5]) with mapi id 15.20.7741.017; Tue, 2 Jul 2024 10:37:41 +0000 From: Rovshan Pashayev To: Fiona Ebner , "pve-devel@lists.proxmox.com" , "f.gruenbichler@proxmox.com" CC: Pavel Tide Subject: =?Windows-1252?Q?Re:_Strange_Behavior_=96_ZFS_(and_other)_Storage_Configu?= =?Windows-1252?Q?ration_in_PVE_(plain_text)?= Thread-Topic: =?Windows-1252?Q?Strange_Behavior_=96_ZFS_(and_other)_Storage_Configurati?= =?Windows-1252?Q?on_in_PVE_(plain_text)?= Thread-Index: AQHazGvdJcvxIqzXnkO37+lCPMuvAw== Date: Tue, 2 Jul 2024 10:37:41 +0000 Message-ID: Accept-Language: en-US, cs-CZ, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=veeam.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH0PR14MB4735:EE_|PH7PR14MB6453:EE_ x-ms-office365-filtering-correlation-id: 95217180-601a-42fc-3926-08dc9a830056 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|1800799024|376014|366016|38070700018; x-microsoft-antispam-message-info: =?Windows-1252?Q?khH4fNvNueyWUSkAh5NF/BPw3aQY0rNpn8LcoBnwjbTokQHmfoJ2cL24?= =?Windows-1252?Q?ZmpyuauxMDFlTe6S3WEAvXWZXMk7GpD8R/Cg9LHKw4uNS3pi9k9V32l6?= =?Windows-1252?Q?oQaa3WA3e1ZY5yMMDodPRpsM8ZCQIsNs2Z8bfLT5mx12rIURDRTyB6OX?= =?Windows-1252?Q?w7EAOL1P2Hqwsr6DkeLG8CM7dzpuVd6TrqjWFYsOxQZ6VvtLcwhP5WMz?= =?Windows-1252?Q?LFEttfHNgJJMRf+SgdsaCdKLDYn7//dLxs3SBfcHtHEySBVDKaUW8RBH?= =?Windows-1252?Q?zDJ4jVhcmIBNaBScnE8jd0OuyPHRB2luRPs++7JmvN82IL9Eb7Smi7aX?= =?Windows-1252?Q?Tw0o9yg1DKU0a8yDDx+GpLisB+fZKnPlDILrL+kKs6lSsA2klIRNkULd?= =?Windows-1252?Q?JgC338isvcjT8l1Fcjhg8Vw0HtudQB0neAXu0jRRDXRUP3sQRWNyp7nP?= =?Windows-1252?Q?1NubAy5aO+JJAaqsqvmYERtxE2hJq9sYv6z9UXE9xHN0b1UuhRALutMi?= =?Windows-1252?Q?Xoc0Ph4UwQRtJ3YS+XqEEsqnX5cJEnboULP7pEL9K2uI/r9jBVrahjid?= =?Windows-1252?Q?SThj5jOQkbyVI9QqB/AMXc/rNqlZaWOY3/vIzmlsRy9hKwWJ6NEQTqXe?= =?Windows-1252?Q?VAQRHnHh6iGqcdRKimJUQIQW869cjPM+/gAsEn7NBDSzMDvjKrwrZmqd?= =?Windows-1252?Q?LEIGsDKFPP+UcRRWm165r/oe4uaYIEKClg+NEKiYsXyqYQPG7gPaoEUv?= =?Windows-1252?Q?/c4zEywq2A02rTz0ck9e0T3w1g0NdHIcwhKXmsUt6PE4lW5dCVINUy2L?= =?Windows-1252?Q?wBtKMBr84JTyhDyNuIwhYw7Y+yTWDDMhplze79ilo91kAIBahu5FUxZ3?= =?Windows-1252?Q?e0xUhRuJzyuKiPBXQ9LMTscVYVLX51xY4TjAHcz4l/SHy/qcMMccAjLq?= =?Windows-1252?Q?BoAIUmn4p5zEOiBRXJZvaDNDtF+k5q4x/59myzu50FFk96v7wHjEJjgn?= =?Windows-1252?Q?Q96rhAvfD+Ho5pzAjdtIBBhjofrn+pc9RGo8uq2lM/ZEd+EVL3EaMprB?= =?Windows-1252?Q?0jvJEiSc5qctDhoWBx5NXTEcqQEa4sdsxPXLAymMbxJrfRIu6y7yeg3B?= =?Windows-1252?Q?3VuK8HLEBrsnpo+hJL4YfX1cJ/e/V63UzolW/NvjfQBYc9dg+i5C6EAt?= =?Windows-1252?Q?1yRdYcycP9qSkm+caRhzTQmmkFgRM/ObNp9LhbV2M8pCv8PGldt9auGT?= =?Windows-1252?Q?ncA8+xLRVEHShY3TQUn30He1+cDE5NTJJcDKNlK1/UI00jBydLS9j3A0?= =?Windows-1252?Q?Srw0J5OBIEZJv9CfOAqya1GvAdNpY+gKZnfNKzI94sYsL3ZRJCK8iROd?= =?Windows-1252?Q?F9+N2gKQ5drPO794G8z0vdT31Ljk0vCe/2oNNGPYZCMiNxsEURe4dr5L?= =?Windows-1252?Q?YcK1nhBexpCejDMYz7gw/bw54gPgvw2jRmOFkycoQ04=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR14MB4735.namprd14.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016)(38070700018);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?vXxbSVnCMM0sF2TDEaSo++xgz4qhq8NybhJXX41Z/FH/BYA7OSO2grae?= =?Windows-1252?Q?BZ/TxU7p8VD/bwFEbB3o6kz01z5UJjidizpV/w7R1RmcCEnAxM/qIJ4a?= =?Windows-1252?Q?MmGg0Gnav3rnQo1a7MqC/L6bBw8EQYVX8PebrSz+qKeOFCmOCJ8bGF+F?= =?Windows-1252?Q?vNodw8BRmsbm5/iQPouwmGMEd1zGNiBKy6Mbk74SepMFpLVvTvf1H2a3?= =?Windows-1252?Q?XzwhYdWNIRGGFOVnqVKMD+vy7aN1bbLD/I2gUMAqMuJDaNLa6sPUK9kS?= =?Windows-1252?Q?t0X5UTccNils3PbjgdVrwvRdxLlNfq5g8MtrJ/+CO7BIzO+ZNxqz61Ov?= =?Windows-1252?Q?YdRrGNCkc74ftfA5YlaQ+V+ORetKwZ7nJGokoePT+BnowGMCMXrwECCt?= =?Windows-1252?Q?xJTlKvAbei31lPrL7jNwaCZCgkPyrDuBSShgm2H072Hkm6b4F9BPWoio?= =?Windows-1252?Q?QQjSCvYfJrcaQgRyRkYi33vzDWm70+QfaSexQ+J+ZmNx+oP4xDNxJYKE?= =?Windows-1252?Q?KKd0DLBG65MXXjPQ9qOWYEyDKXQ0EZOzcgPpzK8M5UA0Q3l29Gas2Z1I?= =?Windows-1252?Q?Kd4y4tKWhc8qQWbZN01l1Kqr9ZJukdE8PZyrc2indZwaHavBozousGeK?= =?Windows-1252?Q?YKwg/OxBVgS1v3yzObqcwTyMUvbPyb3VBfm8CpN10hmCFB2+UsKMkkNi?= =?Windows-1252?Q?thGXo/4bk2+VVreDe4qu5aYO+h9VuOe/CWeT2IMXOt+Xnz9LIungyS2Y?= =?Windows-1252?Q?ODjLRDKcSMj3YuIi1RHkLbKn/BRua/HfnCSE/hMEJgYhCp80V7N4+I/+?= =?Windows-1252?Q?uulMtNI+rdjJ11tauFwmT/llsVnH/yfLK0WcqXrsQMK9yUUR63GjZKrq?= =?Windows-1252?Q?UL7XytUv0zoMARZRsaLRXl4KJDAMBj12WntTJ1uW5akMBVms0VSxGXG0?= =?Windows-1252?Q?1+r+x8GUGWO+8W+M70rm/E/SIf824/W/lDUcq96y303WlZKOYYwAqpkZ?= =?Windows-1252?Q?LiNPCZ3+T5E8xvJejX81hfAznlnUSkVlYJzErFYcdgzLIfhWaZHjBSBG?= =?Windows-1252?Q?n/692d9uFwuWssg2q4k4YPFK+dQcfY2whlH45aHf0aUmLAIWsvBVxJni?= =?Windows-1252?Q?H1hF1F+b1kvIPxR78XnIgQc6N8k5L3Q/s5ZuJC/h1rDbejjD3zQsvl5P?= =?Windows-1252?Q?Lwi9VA3CITcXhqfMCkSuUiq1c3qcf9vDOka4g0t2gs3KOmeZ9LHZfF9n?= =?Windows-1252?Q?mleXaHFU7VMO3RrbymaRkBO84Bdf7fZB1NG9WfZTc5xIyDjSkUknvhBF?= =?Windows-1252?Q?p/6sc6bHeaZPUG+PBc2wTmsTEkN5y//WZDm6HmAmLYEcykIO6k66uKLf?= =?Windows-1252?Q?MGJduiRof87Y3SJmIXCMCidwN9t7sVBre1uiFkRfwljyCNXH7yq+tQc4?= =?Windows-1252?Q?7CIokJ5Kip4uZcezFXKRw9yjOC2qR5jheu1t20i6emqlKBL1uVGKkI/E?= =?Windows-1252?Q?85ons5oDWb4JmYT+k8UHZbWoJhxsXF3r6sBx5Cp7J9SwtDbjKlb2LtN5?= =?Windows-1252?Q?GSp0n/ncA6ARkV4FAARW/40GRqpMHakC70//oxFz8brLqBP2uY/h+u04?= =?Windows-1252?Q?wipFlAuS69FZWlHEV15+jdJxwccWyJXFgtPqeIFcqBhv1z/eyJMZe7fX?= =?Windows-1252?Q?vba8zGtAaEJz0YPZbEPdd8UT0tmWjwNj?= Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: veeam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR14MB4735.namprd14.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 95217180-601a-42fc-3926-08dc9a830056 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Jul 2024 10:37:41.6621 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: ba07baab-431b-49ed-add7-cbc3542f5140 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Lzz5xJ4YgSdyIQD/GZvzGYmLivjqj+dYcgLPeO5bQK1cME981AJddOQQajDbYj9Ap2Xfpr45AUeBxVwJYFmCHWFkORp9/9KxSIrncnc7dqM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR14MB6453 X-SPAM-LEVEL: Spam detection results: 0 ARC_SIGNED 0.001 Message has a ARC signature ARC_VALID 0.001 Message has a valid ARC signature AWL 0.406 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIMWL_WL_HIGH -0.001 DKIMwl.org - High trust sender DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_MSPIKE_H3 0.001 Good reputation (+3) RCVD_IN_MSPIKE_WL 0.001 Mailspike good senders SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [veeam.com,proxmox.com] Hi Fiaona and Fabian, I am merging your questions and answering them in this one reply (I got rep= ly from both of you at the same time :) ) > We would like to report a strange behavior that we have encountered in Pr= oxmox, version 8.2.2. The issue is as follows: > We noticed that ZFS storage configuration is missing mountpoint line rand= omly. And workaround is manually adding missing line. Fabina G =3D=3D=3D=3D what exactly do you mean by this? the entry in /etc/pve/storage.cfg? that one should only change if you call the corresponding API endpoint (or CLI command). please double-check the logs that no such invocation is happening. if you mean something else, please clearly describe how your system is configured and what the unexpected thing that is happening is. VEEAM's REPLY =3D=3D=3D=3D We have requested info about storage using the next command: # pvesh get st= orage/{storageName} --output json As we understand, this command just parse /etc/pve/storage.cfg and return r= esults. So yes, field path and mountpoint are missed in the response of mentioned c= ommand and in the file. But all VM=92s that have disks on this storage are working normally and can= be re-started. Fiona E =3D=3D=3D=3D does the option get lost for already existing storages or does it affect newly added ones? How do you edit/add the ZFS storages, via GUI or CLI, please share the exact commands in the latter case. Please also note that the storage configuration 'mountpoint' is optional. If it is not set, it will be assumed to be the default mount point, i.e. the 'pool' configuration option prepended by '/'. VEEAM's REPLY =3D=3D=3D=3D=3D Could you please confirm that if =91mountpoint=92 is missing in configurati= on it implicitly means that mountpoint is =93/{pool_name}=94? Best Regards, Rovshan Pashayev, Analyst, Veeam Agent for Linux, Mac, AIX & Solaris ________________________________________ From: Fiona Ebner Sent: Tuesday, July 2, 2024 11:00 To: Rovshan Pashayev ; pve-devel@lists.proxmox.= com Cc: Pavel Tide Subject: Re: Strange Behavior =96 ZFS (and other) Storage Configuration in = PVE (plain text) Hi, Am 02.07.24 um 10:48 schrieb Rovshan Pashayev: > Dear Proxmox Developers, > > We would like to report a strange behavior that we have encountered in Pr= oxmox, version 8.2.2. The issue is as follows: > We noticed that ZFS storage configuration is missing mountpoint line rand= omly. And workaround is manually adding missing line. does the option get lost for already existing storages or does it affect newly added ones? How do you edit/add the ZFS storages, via GUI or CLI, please share the exact commands in the latter case. Please also note that the storage configuration 'mountpoint' is optional. If it is not set, it will be assumed to be the default mount point, i.e. the 'pool' configuration option prepended by '/'. > > We also can see same behavior was reported in your forum: > NFS: [SOLVED] - Proxmox 3.1 Lost Mount Point in Cluster | Proxmox Support= Forum (https://forum.proxmox.com/threads/proxmox-3-1-lost-mount-point-in-c= luster.16664/) I don't see how this is related. There is no 'mountpoint' storage config option for NFS storages, just a 'path'. The issue here seems to have been wrong mount options. > ZFS: mountpoint missing after restoring a backup | Proxmox Support Forum = (https://forum.proxmox.com/threads/mountpoint-missing-after-restoring-a-bac= kup.110012/) The issue here is about a missing container mount point volume after restoring a backup (which Fabian pointed out is not restored for unprivileged users). So again, I don't see how this is related. > > We kindly request clarification on whether this behavior is expected and = if you are already aware of this issue. > We are interested to know if there are plans to address and resolve this = in future updates of Proxmox. > > If you require any further information or clarification, please do not he= sitate to ask. > Thank you for your prompt attention to this matter. I look forward to hea= ring from you soon. > > Best Regards, > Rovshan Pashayev > Analyst > Veeam Agent for Linux, Mac, AIX & Solaris > Best Regards, Fiona --===============7358229867838168892== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============7358229867838168892==--