From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id F067D1FF2A8 for ; Tue, 2 Jul 2024 13:55:51 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4B7231D134; Tue, 2 Jul 2024 13:56:06 +0200 (CEST) To: Fiona Ebner , "pve-devel@lists.proxmox.com" , "f.gruenbichler@proxmox.com" Date: Tue, 2 Jul 2024 11:55:24 +0000 References: <1ef60445-0225-4d2d-951b-761b329fe070@proxmox.com> In-Reply-To: <1ef60445-0225-4d2d-951b-761b329fe070@proxmox.com> MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Rovshan Pashayev via pve-devel Precedence: list Cc: Rovshan Pashayev , Pavel Tide X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: Re: [pve-devel] =?windows-1252?q?Strange_Behavior_=96_ZFS_=28and_oth?= =?windows-1252?q?er=29_Storage_Configuration_in_PVE_=28plain_text=29?= Content-Type: multipart/mixed; boundary="===============8056472226466090159==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============8056472226466090159== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 03E09BBC83 for ; Tue, 2 Jul 2024 13:56:05 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id CFEE51D0DE for ; Tue, 2 Jul 2024 13:55:34 +0200 (CEST) Received: from mx2.veeam.com (mx2.veeam.com [38.133.70.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 2 Jul 2024 13:55:33 +0200 (CEST) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2041.outbound.protection.outlook.com [104.47.74.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx2.veeam.com (Postfix) with ESMTPS id F3FB08051C; Tue, 2 Jul 2024 07:55:28 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=veeam.com; s=mx2-2022; t=1719921330; bh=Y8ADb29lKmucNJs3got9+th9CxBnIjNPLafgrgJUJqU=; h=From:To:CC:Subject:Date:References:In-Reply-To:From; b=cUb+0XBsqBdft07ct7siC8ubrzCp+7UxeAq0yD4crVM294banh18lF90Udqaf1hwy w1QRrKJhbiQjox38yT9YgY9S6JjW2jYnn2ybwVXFUkrU4x+VbSflMDjKo4zbhUBO85 0ixWialbs/9DV3VGXmbKthWHchLAmMJq7Lsx14udtG7zBnhnwWZCvtkGZUY7oVsBiu 9V+xRaZKQbscUJvzBawWs3PUq/5AUkNSRUAl6LDOEz1fNbLBf6FwE53Nj0la+Zfl35 zAFNoga02JDlb+zMrDgbAHSt+IMwW62snU3Z8gOqZB4K2pHJUWTUcd3ihACTdQdwp+ avR4ETXIoHUKA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n5Ic6ExBFrXZ1JgDnJvzf0ifWXQjxqzyvJRcoC4wmjRTjE7379KuaXbLahYsE09rG0HS4AGE6hkD855fR1j0J01V54z4an28JhBpwHrplZqkQrMgLO43cKTz08GwNJqdO4sBjOSxbrRzk0q4H0EIixugPZsybhJrRk6Purq3chroKE6zoT0JwAeRJnKY4znDv95PbiMHfnyc/cEjqJQBc/5wHq1etW0xIdJcRVYtdj5dl6SWfrKIJ/TDQ3/kEXhRNujbecfeZzNtsZNWasHL9c5OY535F8kCSq352ycSE5rTCHLfIGKXEjFngmPftBhMAUdcAzt4FVqwe7wlemHt2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y8ADb29lKmucNJs3got9+th9CxBnIjNPLafgrgJUJqU=; b=CFMzuOQHM2Og8v3XUoRQ3YtDRa7r2dVsHGEhcJAn56B5QSyI4G2cHctz4XPR1Gw641wdiRnP9j2JW7fQEO8xjzFcx+ebVHViadT0u4GQPnhOuowcWniSLqpJbH4q0P11bKl2hrcfEb1yRmmLPI6P0nyK29I47m01fpiD2u6hNANvw9s8RHwTDZknx3dSX/UpwZZ/1VYx2K4p+3qrqdzb6ncBp9h7VP/nOaV8weNJmWMQJnP3gcUorHBSoLOyYiXXnSa6FqB5o/fSyAYVCtZbZov79RgJLK6qWR4SZJaaChUlFanLMlA0SzhpMrzxpqrHuId9j4g2vW0Sv4o2gTiOUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=veeam.com; dmarc=pass action=none header.from=veeam.com; dkim=pass header.d=veeam.com; arc=none Received: from PH0PR14MB4735.namprd14.prod.outlook.com (2603:10b6:510:85::9) by BY5PR14MB3512.namprd14.prod.outlook.com (2603:10b6:a03:1ef::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.23; Tue, 2 Jul 2024 11:55:24 +0000 Received: from PH0PR14MB4735.namprd14.prod.outlook.com ([fe80::f4fe:8794:4784:c224]) by PH0PR14MB4735.namprd14.prod.outlook.com ([fe80::f4fe:8794:4784:c224%5]) with mapi id 15.20.7741.017; Tue, 2 Jul 2024 11:55:24 +0000 From: Rovshan Pashayev To: Fiona Ebner , "pve-devel@lists.proxmox.com" , "f.gruenbichler@proxmox.com" CC: Pavel Tide Subject: =?Windows-1252?Q?Re:_Strange_Behavior_=96_ZFS_(and_other)_Storage_Configu?= =?Windows-1252?Q?ration_in_PVE_(plain_text)?= Thread-Topic: =?Windows-1252?Q?Strange_Behavior_=96_ZFS_(and_other)_Storage_Configurati?= =?Windows-1252?Q?on_in_PVE_(plain_text)?= Thread-Index: AQHazGvdJcvxIqzXnkO37+lCPMuvA7HjSQYAgAALTEc= Date: Tue, 2 Jul 2024 11:55:24 +0000 Message-ID: References: <1ef60445-0225-4d2d-951b-761b329fe070@proxmox.com> In-Reply-To: <1ef60445-0225-4d2d-951b-761b329fe070@proxmox.com> Accept-Language: en-US, cs-CZ, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=veeam.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH0PR14MB4735:EE_|BY5PR14MB3512:EE_ x-ms-office365-filtering-correlation-id: b2fc4225-7c6a-4c63-6b8e-08dc9a8ddbac x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|1800799024|376014|366016|38070700018; x-microsoft-antispam-message-info: =?Windows-1252?Q?FI7zlM5iCdzVaPWxji+je6tK2NTkhMjpjuWdmKe+Eixw07vFR2jKC4ob?= =?Windows-1252?Q?C8WjssQgJ7kjVBtG6lB/ciJo8KMckBtm8t3oQOlh80zY8l/qc+2gZO58?= =?Windows-1252?Q?dxp9q8sDKYr2ydGZ+8dG+zDngnVUU8QFgquQTirgdbnd6hK4tUS1EADt?= =?Windows-1252?Q?1a86omBhtSSEqR0aHV0HEErPNAgTk8B2a+IFUBmIa53Zix0cZ/kX3LUx?= =?Windows-1252?Q?muonPSx94Fr0sGXU86vDPbVu08BI6YPztBr8RgiMoJAFvPmegyXe3XMZ?= =?Windows-1252?Q?u1Ame22fdmHaD5A8uSbhI4IQN2RICH9pZcHihSaPwl3meQMPxLAfDELu?= =?Windows-1252?Q?FNwAkstjZIkpec7HrLw2kCz4xTBp8CXiHeFaUvhVYHNY/GiuNUoe/Jew?= =?Windows-1252?Q?wsSN23TphkgRGkbvHbg8mdEAXke/eAiDLia2NTLvquocwgyewhdNrKhQ?= =?Windows-1252?Q?C0vpXXOAVqeE2TOz/IeWhaOCbkBk1fIHhwUkYW/oiXx6qAqYewWe0nDx?= =?Windows-1252?Q?bmIdtw4KtCstHNLZcE8//fAdyksIwkWjsMP5bfGKK9JVfe2sJRnr2QI9?= =?Windows-1252?Q?0bzCTHPUnYmLzDT8zMiy7EHANdyu/l0s7jdCIxu2Nlq1GTt02Pg9hEKc?= =?Windows-1252?Q?W3eBlJLRjkdUSwt3y1YWXeygyKh6NXKoYLbCukr9eVeSrbtu9OR5lRyd?= =?Windows-1252?Q?vBewM9TouyNZD8tqBtj2djpPc0hvocfDJW0j5rS3CNaiNAiN9O5TJCnf?= =?Windows-1252?Q?aoCLUVTS2qQJy0X1L1qDk0ldLgs4CuArEzHdeCOuMWfz95vDPo7WnsKi?= =?Windows-1252?Q?y2CG44xKqj8ylRBQyXO4CLKJIJ6dNmFFBTVWHXzrr6dfkTnyWPb0vqzV?= =?Windows-1252?Q?Usjeo+6VtExFyPiaF8PLsiMgNKJpp6ICgcuL/4+S9NYdY/Cttw4F3iLZ?= =?Windows-1252?Q?CmEYGtMwsbfVH+kqNru3y/Jet/ViRYmI5UzWJnxaYuIrztlNJi4sC908?= =?Windows-1252?Q?f2A/w9GsZ4OX3WFdC1k+zIRh6mLOFKIsrzFnsvZy+dn2w9IPbD9qlgby?= =?Windows-1252?Q?KxEu+mM3j66M8DwRMJwURKTc2AKgw0wOUC5tQORqZa0YaWwEAGju1oPF?= =?Windows-1252?Q?9xNNfaSH5huT/u0U0g73hAGYzT+OQTSNKQdClXEU4n1+UFTMy07DZ3os?= =?Windows-1252?Q?BcdGoo3YSutgQUDiDEVAs2mpKmTYEThImzpRJiG4hVnxU+H8acI7qqWq?= =?Windows-1252?Q?DG4tWFHoo7zKv6m8W5oja/7rO2vFKu2FgyQV092Ohj8Cp+PqSlkU+VyW?= =?Windows-1252?Q?o0zwuef3OpCVUhTg/Pp9CkdK48RSmT0k7IHL78XMf4wm6hwCuuRPi27y?= =?Windows-1252?Q?BaOPbS/XfuRT8tiG25UhKSrMXU/8VXlAaqnQ5VaqbiPkFxyCVK/yu2Hg?= =?Windows-1252?Q?MRFLfO5i9TIRQm/6kIjWDA=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR14MB4735.namprd14.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016)(38070700018);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?KXHvjE+IOf72cCOwgAPZmEw8IRMG/fwlXaJplWPjbh/LoOTKnutfkDaq?= =?Windows-1252?Q?pMwupDu/hjv32cXkRDw+60HNUsinF1JZz0gOiWY07ycjTQZ4aSG/dsKc?= =?Windows-1252?Q?4aTWENOFIWs5pSpw0Zy+RSVK6TEHAoAabQ9KDZSf08cVQShEP1Ngp38k?= =?Windows-1252?Q?cc/3S7K5CFn5cYpzW2Nf6z9KrjJPAr1NsHDk7VGkwImevoQIWnqv9e6s?= =?Windows-1252?Q?PmuilSWf+cBy92qYVXqjKxM/f63Wyx7D5VybSVEVmU25X7ODKH99LgsI?= =?Windows-1252?Q?+oEyZHK6dFps27uS5+Ue+6Ydf/FaRmfgPOjNcBA16VOtvAXt0oAV17rB?= =?Windows-1252?Q?boz3JVNhikC0gKS0HQAHHBCl8q3LbNQzEr9R7YCYto3CTaNS0XINZcZG?= =?Windows-1252?Q?KAsKjzyq00zpbcZXzi7FwkavQ+VrArgZnx2r0IemLjD3qcOv25rsKAw6?= =?Windows-1252?Q?nolZSxkBrYDVLuFcUS4nxYH9dnflgQZ8MaEWBcoTESmGt5fxh7DSG8yc?= =?Windows-1252?Q?P1w008khGqmlqtH/nm8N7DRLD0DLgqzsU+/aaOTbEOwqwHiHy2wUcgGu?= =?Windows-1252?Q?X5jL+8UQ8t705SzcJnC1nhCb8z526NvhG4qq3HM9J5u6Pb2RaB32rKo9?= =?Windows-1252?Q?yqrvDHtIcxLemHEsynYSDgA+wBrl1Qan0os6XAPhdU5z03Fao38A05hZ?= =?Windows-1252?Q?GeBPqBh72tHdkUjqqdz/I2R+8zpijVChbvCJUv6qyZu7S4bKkzOUBTiU?= =?Windows-1252?Q?F0zpm8j5piBt0LNJl0LDNBcA0DBZmwUn/4wc0vQysVIsetpeSeMdKLxd?= =?Windows-1252?Q?5gKSIqbb+g1RaxgPD+ktCNCghhbtnV0LS9iDnm+rWBkxREEbQ3tpJJFV?= =?Windows-1252?Q?/A2Ayqgi0vwsF+u3YWdnudLPnPWzYli8s988DNu5KsZiG6vppjjpG1KL?= =?Windows-1252?Q?tUcy6+y/HGw2TrRex0cpkUuOnf5RNDUfBRIa1zHHCPo7ulQnb4WtqBX8?= =?Windows-1252?Q?YPmekL41znUHhUKsMoEEjTkfgjMKF0eRm52u9Sdx02GNuMjMzNuinFYe?= =?Windows-1252?Q?y5xMcFbSg1wblVGSVaCgUl7UuLzIn0Yl3LYu6ScG2Fkyts8Hip4oOzdv?= =?Windows-1252?Q?4xMjtuxqqMPysXYgcQNb4Y+qd44FTC/oxN0TihzGr0hZ2E7KJN5Ynlu9?= =?Windows-1252?Q?59Gv4IJ3MLz7/Bj6Q8mc/jGtzMuC4tA3JxV/t/pktm39rFv/9vQkrrhW?= =?Windows-1252?Q?m0XNF2SwJWEfMFpIgXFhxdflDx4oNj0IR9/vs4hyc19QqL3fDH6nQlPP?= =?Windows-1252?Q?cwRF4HrHijNljkjg7QcTPCvGt2SH8y7pUposHwF3+88/mmDHm7Zp47R0?= =?Windows-1252?Q?WZnI96glSEZe1mjCgwxfoqTd9SeXooQSAkOMz2YFN3wsRE2ns8pXCPAB?= =?Windows-1252?Q?HdyX3UmlX6UpLz8bEWKJ8uLXOu2Nrwv2p08SUy7k5HpPz3lWSvbeWUba?= =?Windows-1252?Q?qvUvtmfcy+RrSGnIi21n3NmFABO/ScRxpcAWPJbxKG5ves+a/K++bUBf?= =?Windows-1252?Q?P3sYbzLTWPJjGLwUAAKsd3qNwiWnC8/zT5dSqYCuPS2krQOwQ/LvFDVR?= =?Windows-1252?Q?Cp5TiN3bRsQYZrGJbjjkx+SYwTeai7TLPCoIcTz790f4IQjU/p+HLQUo?= =?Windows-1252?Q?zg9QoGhyPBmtkj4vr+jMnpqhAY81PSk2?= Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: veeam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR14MB4735.namprd14.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b2fc4225-7c6a-4c63-6b8e-08dc9a8ddbac X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Jul 2024 11:55:24.5990 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: ba07baab-431b-49ed-add7-cbc3542f5140 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: morRwrBieSZvHypo06TBfwsBqvapzgIba8TpQw/J+e6IexwqsQocZO+PMrUSWYbBOT1IoOwUgunyJvA2FdHPF27sJOA0NE3J+KVST/Pn3kU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR14MB3512 X-SPAM-LEVEL: Spam detection results: 0 ARC_SIGNED 0.001 Message has a ARC signature ARC_VALID 0.001 Message has a valid ARC signature AWL 0.204 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIMWL_WL_HIGH -0.001 DKIMwl.org - High trust sender DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [veeam.com,proxmox.com] Hello Fiona, Thank you for replies, they are very helpful. I will update thread if anything needed more. ________________________________________ From: Fiona Ebner Sent: Tuesday, July 2, 2024 13:13 To: Rovshan Pashayev ; pve-devel@lists.proxmox.= com ; f.gruenbichler@proxmox.com Cc: Pavel Tide Subject: Re: Strange Behavior =96 ZFS (and other) Storage Configuration in = PVE (plain text) Hi, Am 02.07.24 um 12:37 schrieb Rovshan Pashayev: > Hi Fiaona and Fabian, > > I am merging your questions and answering them in this one reply (I got r= eply from both of you at the same time :) ) > >> We would like to report a strange behavior that we have encountered in P= roxmox, version 8.2.2. The issue is as follows: >> We noticed that ZFS storage configuration is missing mountpoint line ran= domly. And workaround is manually adding missing line. > > Fabina G =3D=3D=3D=3D > what exactly do you mean by this? the entry in /etc/pve/storage.cfg? > that one should only change if you call the corresponding API endpoint > (or CLI command). please double-check the logs that no such invocation > is happening. > > if you mean something else, please clearly describe how your system is > configured and what the unexpected thing that is happening is. > > VEEAM's REPLY =3D=3D=3D=3D > We have requested info about storage using the next command: # pvesh get = storage/{storageName} --output json > As we understand, this command just parse /etc/pve/storage.cfg and return= results. > So yes, field path and mountpoint are missed in the response of mentioned= command and in the file. > But all VM=92s that have disks on this storage are working normally and c= an be re-started. > There is no 'path' property for ZFS storages in Proxmox VE. There are 'pool' and optionally 'mountpoint', see [0]. > Fiona E =3D=3D=3D=3D > > does the option get lost for already existing storages or does it affect > newly added ones? How do you edit/add the ZFS storages, via GUI or CLI, > please share the exact commands in the latter case. Please also note > that the storage configuration 'mountpoint' is optional. If it is not > set, it will be assumed to be the default mount point, i.e. the 'pool' > configuration option prepended by '/'. > > VEEAM's REPLY =3D=3D=3D=3D=3D > Could you please confirm that if =91mountpoint=92 is missing in configura= tion it implicitly means that mountpoint is =93/{pool_name}=94? > Yes, see the code below (from [1]). If the 'mountpoint' option is not set, the pool 'option' with a prepended '/' will be used as a fallback, as that is the default mountpoint for a ZFS dataset: sub path { my ($class, $scfg, $volname, $storeid, $snapname) =3D @_; my ($vtype, $name, $vmid) =3D $class->parse_volname($volname); my $path =3D ''; my $mountpoint =3D $scfg->{mountpoint} // "/$scfg->{pool}"; [0]: https://git.proxmox.com/?p=3Dpve-storage.git;a=3Dblob;f=3Dsrc/PVE/Storage/Z= FSPoolPlugin.pm;h=3D3669fe152b62df7ebeda572cdf56ef19be872d56;hb=3D13a81873b= a22f37ecc0613de3f3ef994b064e3b9#l45 [1]: https://git.proxmox.com/?p=3Dpve-storage.git;a=3Dblob;f=3Dsrc/PVE/Storage/Z= FSPoolPlugin.pm;h=3D3669fe152b62df7ebeda572cdf56ef19be872d56;hb=3D13a81873b= a22f37ecc0613de3f3ef994b064e3b9#l141 Best Regards, Fiona --===============8056472226466090159== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============8056472226466090159==--