From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <a.lauterer@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id 3D88EB159
 for <pve-devel@lists.proxmox.com>; Wed,  6 Apr 2022 13:47:15 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id 2FF6B2A387
 for <pve-devel@lists.proxmox.com>; Wed,  6 Apr 2022 13:47:15 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
 [94.136.29.106])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS id 2C23E2A37D
 for <pve-devel@lists.proxmox.com>; Wed,  6 Apr 2022 13:47:14 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
 by proxmox-new.maurer-it.com (Proxmox) with ESMTP id EBA7541FA8
 for <pve-devel@lists.proxmox.com>; Wed,  6 Apr 2022 13:47:13 +0200 (CEST)
From: Aaron Lauterer <a.lauterer@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Wed,  6 Apr 2022 13:46:57 +0200
Message-Id: <20220406114657.452190-1-a.lauterer@proxmox.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results:  0
 AWL 0.029 Adjusted score from AWL reputation of From: address
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 T_SCC_BODY_TEXT_LINE    -0.01 -
Subject: [pve-devel] [PATCH v2 storage] rbd: alloc image: fix #3970 avoid
 ambiguous rbd path
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 06 Apr 2022 11:47:15 -0000

If two RBD storages use the same pool, but connect to different
clusters, we cannot say to which cluster the mapped RBD image belongs to
if krbd is used. To avoid potential data loss, we need to verify that no
other storage is configured that could have a volume mapped under the
same path before we create the image.

The ambiguous mapping is in
/dev/rbd/<pool>/<ns>/<image> where the namespace <ns> is optional.

Once we can tell the clusters apart in the mapping, we can remove these
checks again.

See bug #3969 for more information on the root cause.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
changes since
v1:
* fixed code style issues
* moved check to a helper function and call it from
  - alloc_image
  - clone_image
  - rename_image
* rephrased error message with a link to the bugzilla issue

RFC:
* moved check to pve-storage since containers and VMs both have issues
  not just on a move or clone of the image, but also when creating a new
  volume
* reworked the checks, instead of large if conditions, we use
  PVE::Tools::safe_compare with comparison functions
* normalize monhost list to match correctly if the list is in different
  order
* add storage name to error message that triggered the checks
* ignore disabled storages

 PVE/Storage/RBDPlugin.pm | 45 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/PVE/Storage/RBDPlugin.pm b/PVE/Storage/RBDPlugin.pm
index e287e28..2a4e1a8 100644
--- a/PVE/Storage/RBDPlugin.pm
+++ b/PVE/Storage/RBDPlugin.pm
@@ -127,6 +127,45 @@ my $krbd_feature_update = sub {
     }
 };
 
+# check if another rbd storage with the same pool name but different
+# cluster exists. If so, allocating a new volume can potentially be
+# dangerous because the RBD mapping, exposes it in an ambiguous way under
+# /dev/rbd/<pool>/<ns>/<image>. Without any information to which cluster it
+# belongs, we cannot clearly determine which image we access and
+# potentially use the wrong one. See
+# https://bugzilla.proxmox.com/show_bug.cgi?id=3969 and
+# https://bugzilla.proxmox.com/show_bug.cgi?id=3970
+# TODO: remove these checks once #3969 is fixed and we can clearly tell to
+# which cluster an image belongs to
+my $check_blockdev_collision = sub {
+    my ($storeid, $scfg) = @_;
+
+    my $storecfg = PVE::Storage::config();
+    foreach my $store  (keys %{$storecfg->{ids}}) {
+	next if $store eq $storeid;
+
+	my $checked_scfg = $storecfg->{ids}->{$store};
+
+	next if $checked_scfg->{type} ne 'rbd';
+	next if $checked_scfg->{disable};
+	next if $scfg->{pool} ne $checked_scfg->{pool};
+
+	my $normalize_mons = sub { return join(';', sort( PVE::Tools::split_list(shift))) };
+	my $cmp_mons = sub { $normalize_mons->($_[0]) cmp $normalize_mons->($_[1]) };
+	my $cmp = sub { $_[0] cmp $_[1] };
+
+	# internal and internal, or external and external with identical monitors
+	# => same cluster
+	next if PVE::Tools::safe_compare($scfg->{monhost}, $checked_scfg->{monhost}, $cmp_mons) == 0;
+
+	# different namespaces => no clash possible
+	next if PVE::Tools::safe_compare($scfg->{namespace}, $checked_scfg->{namespace}, $cmp) != 0;
+
+	die "Cannot create volume on '$storeid' - RBD blockdev paths shared with storage '$store'. ".
+	    "See https://bugzilla.proxmox.com/show_bug.cgi?id=3969 for more details.\n";
+    }
+};
+
 sub run_rbd_command {
     my ($cmd, %args) = @_;
 
@@ -475,6 +514,8 @@ sub clone_image {
     my $snap = '__base__';
     $snap = $snapname if length $snapname;
 
+    $check_blockdev_collision->($storeid, $scfg);
+
     my ($vtype, $basename, $basevmid, undef, undef, $isBase) =
         $class->parse_volname($volname);
 
@@ -516,6 +557,8 @@ sub alloc_image {
     die "illegal name '$name' - should be 'vm-$vmid-*'\n"
 	if  $name && $name !~ m/^vm-$vmid-/;
 
+    $check_blockdev_collision->($storeid, $scfg);
+
     $name = $class->find_free_diskname($storeid, $scfg, $vmid) if !$name;
 
     my @options = (
@@ -769,6 +812,8 @@ sub volume_has_feature {
 sub rename_volume {
     my ($class, $scfg, $storeid, $source_volname, $target_vmid, $target_volname) = @_;
 
+    $check_blockdev_collision->($storeid, $scfg);
+
     my (
 	undef,
 	$source_image,
-- 
2.30.2