From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id A5D1D60243 for ; Wed, 12 Jan 2022 15:07:04 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 2D8A82AB1 for ; Wed, 12 Jan 2022 15:06:34 +0100 (CET) Received: from smtp-relay-out-s1.neusta.de (smtp-s1.neusta.de [82.198.213.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 628282AA2 for ; Wed, 12 Jan 2022 15:06:32 +0100 (CET) Received: from lxvm163-62.prod-s1-nsd.neusta.de (localhost.localdomain [127.0.0.1]) by smtp-relay-out-s1.neusta.de (Proxmox) with ESMTP id 2A9F720070 for ; Wed, 12 Jan 2022 14:57:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=neusta.de; h=cc :content-transfer-encoding:content-type:content-type:date:from :from:message-id:mime-version:reply-to:subject:subject:to:to; s= 20200720; bh=rYhkWPdyqFAP8x43em3jgzbMAPs5zVsX87dPkEt3E6A=; b=YfR nbeinzl7c+tfNUnpaQgjPgnd6PC17lM+UgZ/0zHJHWGHXc1cIwkR88Ee67p9fsk8 FPI3dtzPiCFlBkeFWe/Hzv7UAOlkFz0vWGX3o9ciJPHyH0D3d4KVS4mjnWueHOs/ /R+JBZCP2mInnEGxRcbMR12wDlXQvOqWRJ1Q/teNw32QFHbT0182w/LdVsF2cCmo EQW8On/fJnWo5+8o8grQl7pNMP/S32CfJafl85xZvKKCAqDqel5C0+WXk2FbHH1/ ayxY8NEl+dFRWCU8kLgxB/WmBfHdKgpZkh3AsxZB8Rb4AT4mgYmvGwKe1Y2IVY4p 5Z6Xhy4ZznuJKfyORjQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z8RA7kepSr0Y3OkI0ZhA0wfHGykM8ngX+nPSZnXSWPEnBLLgs2heZya5NmGvB4RDWX4ybZlwp1An+Dkt2xpe3Apn+wkQP+Ni4LNmzvOvuM4S5of/YT7qfelvpmIjtGmushdS403MHpHlvv/RLkGbU719mc5x43i55CpBdgHEJuYtrxSVQq3mbltpAIQceXuOHMSNATfE7rZZJ4fO5RtgQ0lv6ifEzxbrh1bOt+eonq1AAstJWHb/6EXuzx6GlguKrXVeR26DgX53ENR3C/lmZ+Hv9XkKY+ZAVfxt9OmL4LdYVRQn65HMSFOsLSFhSQqkkrHlCNGC5A5luyfY2lKzTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rYhkWPdyqFAP8x43em3jgzbMAPs5zVsX87dPkEt3E6A=; b=U0sAZodcqWxk4KIzK8spGR2JWasLDDpKFO0I1DZvroGIU8+HK3A+9PzsnZHsUbblPOjw6sx1EiHc2nF0Eu4my76LPM6WXvnT43LndTEaaUFsmQ1z58QJs5tEjUxun7YmNqok+TWR647k5QAUfsNNaV3x6tuPPC1slupiXWo3jPtB4uHNbLotFoJDgfHLksnZ9yPrDSm+Kgls3SHc+fTbNY3+Wf8Eih2KFI4s/HzsJCX0pM96vig+8kFS6DWcIPm/iE2gGQSdgJMsegvIWCb1EJZAcnK6KtozaqhUZ/7DwKtgHGZ4oUxm2+dvz81+xBew6LN05SOVXBYcGXKRZoFHbA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=neusta.de; dmarc=pass action=none header.from=neusta.de; dkim=pass header.d=neusta.de; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=teamneusta.onmicrosoft.com; s=selector2-teamneusta-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rYhkWPdyqFAP8x43em3jgzbMAPs5zVsX87dPkEt3E6A=; b=VTemmx87oxIL3NoTdyGwjsHYo/1kRMlaqVI5fyqsj7t12pSaAK1rYzJsXP1xFLZGPPrPL0JZoO+8iK7I7JLzvl3xfN6oQeDlc5VoXD83JgBkv1ft+gmO93LV5qEVxv6pGsX75SwsCohqizdhQ/kq0d7EkNmuluWL62MApvqz0mA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=neusta.de; From: Marco Witte To: Message-ID: <2a32d0af-d323-f38d-cbf6-6a11ba16c452@neusta.de> Date: Wed, 12 Jan 2022 14:57:51 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-ClientProxiedBy: AM0PR06CA0083.eurprd06.prod.outlook.com (2603:10a6:208:fa::24) To AM0PR02MB5667.eurprd02.prod.outlook.com (2603:10a6:208:15d::22) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bebebe26-477d-45e3-d92b-08d9d5d38686 X-MS-TrafficTypeDiagnostic: AM6PR02MB4915:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +CDXgYTylOaZ1fp8NefAeAywJQJteko9IcVxV0OjJuwDsQxa08BjQ9HVjrYqyyJ5v47v2NvdvEfxzH/oNcOVlPyrLf/vSa+9DLNBDITA0FuI1dK04v/jJGJWHio503B6L/2nHzhKlQKDMqZenw4xOMo1X51VbiT9Pz1tBSGM7W86e4vXkkKxfnlg3fj9isWJ0fA5pgMUq7ZDJLVHYnJSzwbY40CRf9shvuaaiNrNcMZwYZPdprnmG43T+la4rkFJb/efYQuUjgSVzKCDVMx630rL1YI8GGOFhHnnE+/SHX5hjChNtUdxFhru/ij1cDxqgYgGRqM/gnVxRc4O1HpyIEcaco8w5Ru+hWPpzO9nIHV2XEqXsxUi+EK181Sca8Sh+DXO0T4TghGNPIJGZbOoREbOH3M91gUPoYCwdDBcMjVrM+HdufN+ODN8TgyD51C+7VEIX8QnUc/ndaKtXcp8psF7HdQidYiThAcB5wpVnqmbDhfne8i5k+NuLXTMAdR2cyHgSR24EdLXR6RyN5mN0VPY0+yAJzT7n+5B96AdPg3LZ+EGfPECy5duek8uMDDmbf1By7U1WtAW7kyNkxYzI4Ms9QUNDIcot2jZKJdIBcZ/deYyijcvIjY24XPv04sI0Guk9S2GLyKrCJKrPV0FGyERyRI1/1km0/kbCojxx3juHDMIR2SmN67TpiVvTcqcBQ1xUuVRlUncYNKUk1I+4HMc8moPsN99U0JbJjoXXdA= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR02MB5667.eurprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(316002)(8936002)(36756003)(6512007)(26005)(5660300002)(6916009)(31696002)(8676002)(38100700002)(186003)(6486002)(508600001)(2906002)(86362001)(66946007)(2616005)(66556008)(66476007)(31686004)(6506007)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZFZNK0pVUzVLUGY2Q1B6MGpuS0tjakg3MStPNGg4emRoRGhmdmNCRlUxSXQz?= =?utf-8?B?REZnd252K3NqYktWUG1WVGk5Q01WSU5TSEpwbmhkbEN0eEpLWnMvUy9RM1B2?= =?utf-8?B?aW50QnN3QmdJc25kOGViQVNzd1ZHMmY5b25VK1pndyswaHNhV29Bd2dTSmpx?= =?utf-8?B?OEZXOVZNWmV5T3hJVGxLM1RXT25OK0tReE9nL3V6TUdLNHA2MmhUbHdiUHB2?= =?utf-8?B?UGpzV2tGT2RlYVVYK25KTVF4dHRaMXZMY1dvM1VNSGppNnZXTCtFc0lWZWNS?= =?utf-8?B?RVl2WXIzV3NZaUxKMlBadmhXcmZ3V2hvalNQeXZlMEhseFc2bEg2dzRHbjFX?= =?utf-8?B?RExzdzRTYUlOY2RxV256YWJETDExU2J2QUhoeW1ZWkt5UEJiSlA2NEVSMkVr?= =?utf-8?B?a2YzOFdlVGk3MzJldEdlUmY5dHY0RitIMjN5N1FvNDNRMm5aMnpFa2Rnejl4?= =?utf-8?B?OEd3VGhkTitBN0lxcHY3d3RSSnRhRmdIcTdINDJneDJRN0NJR3NCNkhVL2JM?= =?utf-8?B?U1ZZYklnRnN6QU9XNzR4VW0rR2ZYM0VCcUN2aXF2ZU56aU9yNjZHN09tMkNB?= =?utf-8?B?bVBhblFvcUd2VVBUZjFhWmhMdkxLL01DQkw4T0tGbTNRT2tuanNUZkVYbFky?= =?utf-8?B?dlpwS1kvclVkQUdURnYwbEVJeU1xaXBJaUpNL2RWWGlnSGVaOTJBOHdJc0tJ?= =?utf-8?B?bWsvSFNDVm9BVmxEd051ZFZiZFZkcmVxWmN1UDFYSjlZODVLenZvTkNPR2lu?= =?utf-8?B?UVcyd2YwU1RldXZGVDNHczV4UUwrc05rUXkzS2VaZjFXWmliSXM1QXNsNTY0?= =?utf-8?B?MHNqbUdLSHZGZUE4Wml1bjVmRjFKM3IwM0N2MTczNWZaNFltQlRaSFpycFlR?= =?utf-8?B?YjJWTlFzQnozQkdqZEhjbnRMbDN4Tlo2aTY4dUVCdTB0T2JFT01FUy9oV3M0?= =?utf-8?B?cDVzVGdXbzV2MVNTQmM1Zk9MakM3b0pYTGp3aGplWktWZTdKMnp5L20wNERt?= =?utf-8?B?RFlLSm5DSDNMSWUzNm5vK1dSNi9CUFBzRTFsWXNHYS9vQmY5blUzZE1wbDVK?= =?utf-8?B?TXBaQ2pXZkRsdFN2QUl6SVZRZHFXRmhJUXNtQWxtMERSQ3RDdzEvbkxxdFlZ?= =?utf-8?B?UHpiY2p2TEdpNzErblM2UUdrc1FUV0ZtNDllUXp2ZVBBWVhGZzJUUzJjTE0y?= =?utf-8?B?NUlYYzlPQlhRa0wrcGRGOUN6eE5RM2hVL1l6ZW9WdVRhMzBCcFY5ZXllVW5s?= =?utf-8?B?em5GaHEzcmZ1N29tSTVsR3owbEFNcTY5eUNwbldwL0F4ZXdaYk1SRlIvVk9N?= =?utf-8?B?cGVPTzNhY1hsS2piL3FFSTB2c0NhcVRsTmhiTGtTeGpIenZJRTNNVzl1di9E?= =?utf-8?B?ZGF6aXFyd040QXNPRnVKendLT29YRi9rdmtrL2U5QzQ0K3M4dDJSQ1JoNjRK?= =?utf-8?B?UWx5aFZLYVNreVViakpHL3NScXhhbVRPRm1LNi91ck5MVlhOOCtPVlVyQVY5?= =?utf-8?B?RVdzVGlWVWdwdlB2TDdFYWdHcDh1T3pOY1lDOFhwS2pMcVE5NjRCbmNUcUIw?= =?utf-8?B?Z1dPdlJzSlFpbWo5ano1SDNMK2RXeVp4UGhEb2xva2Jnci8wK3llQ1NCclh4?= =?utf-8?B?REJXTGU2YUNZNnNNZUFMR24wQmRZVFlmRFdXeU03SUxjSWZpZ3NkOHRhdFFv?= =?utf-8?B?cDhqalVla0xlNnVscnd6d1Jic1ZCRW03aXBSUEhVblpUb2Q1Rkd0d09IVkY0?= =?utf-8?B?ZDdFcE5uejBNSlZBc1BRMVNzbTJrbS9GTUJvVGtXQmQrT3RUUjZIRDNRZ0Fa?= =?utf-8?B?djhRZlM2ZDlkU0krS1RxUGNDYWh2Q2ZnMzNMVTArRm0wSCtwSlEvcmR3RmhI?= =?utf-8?B?eUwvSnl5RGNVSVo2bTk5dXVROEc4UUJyY2g3Q0s4YXN1by9MeStBWnhYdGd4?= =?utf-8?B?dFk3NWNoQS9OLzlybU4vU3BJcGdRWXNZMnBRbmFveDBqQkZsalZoaVdGV1Zr?= =?utf-8?B?Nm5SY2t5eW0yL1Vjc1c2MTdDRVpnLzhuNEg1TDl4bmpKWllaV3VmbHpvcTZN?= =?utf-8?B?K2k1TDV0a2ZYak0yREI2R3V0V0tCL2UzV2pqY1FRaHpmNzVWRlZ0ZDFETFR5?= =?utf-8?B?ZnBINXlhSDhndEJYRHZ2WjN5Q0pxcTRSc2o0dHh5NEpYeVoyV2xVQUEvdHkz?= =?utf-8?Q?PxKRYDExoI2g1voCzYDdNhQ=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: bebebe26-477d-45e3-d92b-08d9d5d38686 X-MS-Exchange-CrossTenant-AuthSource: AM0PR02MB5667.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jan 2022 13:57:52.2842 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e5e89441-3abd-4c70-8445-b44087643e46 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bNi9L8BKV51JIocJ22VDmhXy7PISHNrQuk99dx/MstFxTakgwatI4SD6T21xpViCqOMAIQFuzciHbbgPKjNwBA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR02MB4915 X-OriginatorOrg: neusta.de X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [PVE-User] proxmox ceph osd option to move wal to a new device X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jan 2022 14:07:04 -0000 One wal drive was failing. So I replaced it with: pveceph osd destroy 17 --cleanup 1 pveceph osd destroy 18 --cleanup 1 pveceph osd destroy 19 --cleanup 1 This removed the three disks and removed the osd-wal. The Drive sdf is the replacement for the failed wal device that above three osds ( sdb, sdc,sdd ) used: pveceph osd create /dev/sdb -wal_dev /dev/sdf pveceph osd create /dev/sdc -wal_dev /dev/sdf pveceph osd create /dev/sdd -wal_dev /dev/sdf This approach worked fine, but took a lot of time. So I figured it would be better to change the wal for the existing osd: At this state /dev/sdf is completly empty (has no lvm/wiped) and all three osd still use the failing wal device /dev/sdh. ceph-volume lvm new-wal --osd-id 17 --osd-fsid 01234567-1234-1234-123456789012 --target /dev/sdf Which obviously fails, because the target should be --target vgname/new_wal Question part: What would be the fast way to make the new device /dev/sdf the wal device, without destroying the osds 17 18 19? Versions: pve-manager/7.1-8/5b267f33 (running kernel: 5.13.19-2-pve) ceph version 16.2.7 (f9aa029788115b5df5eeee328f584156565ee5b7) pacific (stable) Thank you