From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 92C7BC6B0 for ; Mon, 11 Apr 2022 14:17:50 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 7CF293F8D for ; Mon, 11 Apr 2022 14:17:20 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 2B00D3F7F for ; Mon, 11 Apr 2022 14:17:19 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 00AB940A61 for ; Mon, 11 Apr 2022 14:17:19 +0200 (CEST) Message-ID: <60e67e63-5229-47ce-1bc2-bca0c87ce3d3@proxmox.com> Date: Mon, 11 Apr 2022 14:17:17 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:100.0) Gecko/20100101 Thunderbird/100.0 Content-Language: en-US To: Aaron Lauterer , Proxmox VE development discussion , =?UTF-8?Q?Fabian_Gr=c3=bcnbichler?= References: <20220406114657.452190-1-a.lauterer@proxmox.com> <1649404843.ds1yioa8qv.astroid@nora.none> <2a67ca76-f10f-5c2f-44a7-9d9da0c36c78@proxmox.com> From: Thomas Lamprecht In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 1.537 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -3.086 Looks like a legit reply (A) PROLO_LEO1 0.1 Meta Catches all Leo drug variations so far SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: Re: [pve-devel] [PATCH v2 storage] rbd: alloc image: fix #3970 avoid ambiguous rbd path X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Apr 2022 12:17:50 -0000 On 11.04.22 11:08, Aaron Lauterer wrote: > On 4/11/22 09:39, Thomas Lamprecht wrote: >> The obvious question to me is: why bother with this workaround when we= can >> make udev create the symlink now already? >> >> Patching the rules file and/or binary shipped by ceph-common, or shipp= ing our >> own such script + rule, would seem relatively simple. > > The thinking was to implement a stop gap to have more time to consider = a > solution that we can upstream. If the stop-gap would be trivial and non intrusive (compared to the Corre= ct Solution=E2=84=A2) or if it would be really pressing, sure, but IMO the s= ituation here isn't exactly /that/ clear cut. >=20 > Fabian might have some more thoughts on it but yeah, right now we could= patch > the udev rules and the ceph-rbdnamer which is called by the rule to cre= ate > the current paths and then additionally the cluster specific ones. > Unfortunately, it seems like the unwieldy cluster fsid is the only iden= tifier > we have for the cluster. The unwieldiness doesn't really matter for us here though, this is just a= n internal check. >=20 > Some more (smaller) changes might be necessary, if the implementation w= e > manage to upstream will be a bit different. But that should not be much= of an > issue AFAICT. We can always ship our downstream solution to be whatever we want and syn= c up on major release, so not a real problem. FWIW, with storage getting the following patch the symlinks get created (= may need an trigger for reloading udev (or manually `udevadm control -R`). We'd only need to check to prioritize /deb/rbd-pve/$fsid/... paths first;= do you have time to give that a go? diff --git a/Makefile b/Makefile index 431db16..029b586 100644 --- a/Makefile +++ b/Makefile @@ -41,6 +41,7 @@ install: PVE pvesm.1 pvesm.bash-completion pvesm.zsh-co= mpletion install -d ${DESTDIR}${SBINDIR} install -m 0755 pvesm ${DESTDIR}${SBINDIR} make -C PVE install + make -C udev-rbd install install -d ${DESTDIR}/usr/share/man/man1 install -m 0644 pvesm.1 ${DESTDIR}/usr/share/man/man1/ gzip -9 -n ${DESTDIR}/usr/share/man/man1/pvesm.1 diff --git a/udev-rbd/50-rbd-pve.rules b/udev-rbd/50-rbd-pve.rules new file mode 100644 index 0000000..79432df --- /dev/null +++ b/udev-rbd/50-rbd-pve.rules @@ -0,0 +1,2 @@ +KERNEL=3D=3D"rbd[0-9]*", ENV{DEVTYPE}=3D=3D"disk", PROGRAM=3D"/usr/libex= ec/ceph-rbdnamer-pve %k", SYMLINK+=3D"rbd-pve/%c" +KERNEL=3D=3D"rbd[0-9]*", ENV{DEVTYPE}=3D=3D"partition", PROGRAM=3D"/usr/= libexec/ceph-rbdnamer-pve %k", SYMLINK+=3D"rbd-pve/%c-part%n" diff --git a/udev-rbd/Makefile b/udev-rbd/Makefile new file mode 100644 index 0000000..065933b --- /dev/null +++ b/udev-rbd/Makefile @@ -0,0 +1,21 @@ +PACKAGE=3Dlibpve-storage-perl + +DESTDIR=3D +PREFIX=3D/usr +LIBEXECDIR=3D${PREFIX}/libexec +LIBDIR=3D${PREFIX}/lib + +all: + +.PHONY: install +install: 50-rbd-pve.rules ceph-rbdnamer-pve + install -d ${DESTDIR}${LIBEXECDIR} + install -m 0755 ceph-rbdnamer-pve ${DESTDIR}${LIBEXECDIR} + install -d ${DESTDIR}${LIBDIR}/udev/rules.d + install -m 0644 50-rbd-pve.rules ${DESTDIR}${LIBDIR}/udev/rules.d= + +.PHONY: clean +clean: + +.PHONY: distclean +distclean: clean diff --git a/udev-rbd/ceph-rbdnamer-pve b/udev-rbd/ceph-rbdnamer-pve new file mode 100755 index 0000000..23dd626 --- /dev/null +++ b/udev-rbd/ceph-rbdnamer-pve @@ -0,0 +1,24 @@ +#!/bin/sh + +DEV=3D$1 +NUM=3D`echo $DEV | sed 's#p.*##g; s#[a-z]##g'` +POOL=3D`cat /sys/devices/rbd/$NUM/pool` +CLUSTER_FSID=3D`cat /sys/devices/rbd/$NUM/cluster_fsid` + +if [ -f /sys/devices/rbd/$NUM/pool_ns ]; then + NAMESPACE=3D`cat /sys/devices/rbd/$NUM/pool_ns` +else + NAMESPACE=3D"" +fi +IMAGE=3D`cat /sys/devices/rbd/$NUM/name` +SNAP=3D`cat /sys/devices/rbd/$NUM/current_snap` + +echo -n "/$CLUSTER_FSID/$POOL" + +if [ -n "$NAMESPACE" ]; then + echo -n "/$NAMESPACE" +fi +echo -n "/$IMAGE" +if [ "$SNAP" !=3D "-" ]; then + echo -n "@$SNAP"