From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id DEAA174590 for ; Wed, 1 Jun 2022 12:11:50 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D66B51BDAF for ; Wed, 1 Jun 2022 12:11:50 +0200 (CEST) Received: from ip1-isp1-algemesi.verdnatura.es (ip1-isp1-algemesi.verdnatura.es [195.77.191.178]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 982891BDA4 for ; Wed, 1 Jun 2022 12:11:49 +0200 (CEST) Received: from webmail.verdnatura.es (swarm-worker3.static.verdnatura.es [10.0.2.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ip1-isp1-algemesi.verdnatura.es (Postfix) with ESMTPSA id 1FE56A0098; Wed, 1 Jun 2022 12:11:42 +0200 (CEST) MIME-Version: 1.0 Date: Wed, 01 Jun 2022 12:11:41 +0200 From: nada To: Proxmox VE user list In-Reply-To: References: <00ab49ec-d822-2522-c861-ed2409681f27@elettra.eu> User-Agent: Roundcube Webmail/1.4.11 Message-ID: <2757d9cfb5e8f35341b599493fb12a81@verdnatura.es> X-Sender: nada@verdnatura.es Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.007 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: Re: [PVE-User] Cannot list disks from an external CEPH pool X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jun 2022 10:11:50 -0000 hello, just correct IP address at pool1 1172.16.1.2 probably 172.16.1.2 and you may enforce access by krbd 1 and simplify your list command by symlinks example rbd: pool1 content images krbd 1 monhost 172.16.1.1,172.16.1.2,172.16.1.3 pool pool1 username admin # la /etc/ceph/ total 20 drwxr-xr-x 2 root root 7 Mar 6 13:03 . drwxr-xr-x 97 root root 193 May 19 03:10 .. lrwxrwxrwx 1 root root 27 Aug 4 2021 rbd.conf -> /etc/pve/priv/ceph/rbd.conf lrwxrwxrwx 1 root root 30 Aug 4 2021 rbd.keyring -> /etc/pve/priv/ceph/rbd.keyring -rw-r--r-- 1 root root 92 Aug 28 2019 rbdmap lrwxrwxrwx 1 root root 31 Feb 2 12:37 rbd_ssd.conf -> /etc/pve/priv/ceph/rbd_ssd.conf lrwxrwxrwx 1 root root 34 Feb 2 12:37 rbd_ssd.keyring -> /etc/pve/priv/ceph/rbd_ssd.keyring # pvesm list rbd Volid Format Type Size VMID rbd:vm-105-disk-0 raw images 42949672960 105 rbd:vm-111-disk-0 raw images 42949672960 111 # pvesm list rbd_ssd Volid Format Type Size VMID rbd_ssd:vm-102-disk-0 raw images 6442450944 102 rbd_ssd:vm-103-disk-0 raw images 4294967296 103 good luck Nada On 2022-06-01 11:51, Iztok Gregori wrote: > On 01/06/22 11:29, Aaron Lauterer wrote: >> Do you get additional errors if you run the following command? >> Assuming that the storage is also called pool1. >> >> pvesm list pool1 > > No additional errors: > > root@pmx-14:~# pvesm list pool1 > rbd error: rbd: listing images failed: (2) No such file or directory > > > >> Do you have VMs with disk images on that storage? If so, do they start >> normally? > > Yes, we have a lot of VMs with disk on that storage and yes they seems > to start normally (last start yesterday when we first notice the GUI > behaviour) > >> >> Can you show the configuration of that storage and the one of the >> working pool? (/etc/pve/storage.cfg) > > Sure (edited the IP addresses and pool names): > > [cit /etc/pve/storage.cfg] > ... > rbd: pool1 > content images > monhost 172.16.1.1;1172.16.1.2;172.16.1.3 > pool pool1 > username admin > > rbd: pool2 > content images > monhost 172.16.1.1;172.16.1.2;172.16.1.3 > pool pool2 > username admin > ... > [/cit] > > Thanks! > > Iztok > >> >> On 6/1/22 11:13, Iztok Gregori wrote: >>> Hi to all! >>> >>> I have a Proxmox cluster (7.1) connected to an external CEPH cluster >>> (octopus).  From the GUI I cannot list the content (disks) of one >>> pool (but I'm able to list all the other pools): >>> >>> rbd error: rbd: listing images failed: (2) No such file or directory >>> (500) >>> >>> The pveproxy/access.log shows the error for "pool1": >>> >>> "GET /api2/json/nodes/pmx-14/storage/pool1/content?content=images >>> HTTP/1.1" 500 13 >>> >>> but when I try another pool ("pool2") it works: >>> >>> "GET /api2/json/nodes/pmx-14/storage/pool2/content?content=images >>> HTTP/1.1" 200 841 >>> >>>  From the command line "rbd ls pool1" is working fine (because I >>> don't have a ceph.conf I ran it with "rbd -m 172.16.1.1 --keyring >>> /etc/pve/priv/ceph/pool1.keyring ls pool1") and I see the pool >>> contents. >>> >>> The cluster is running fine, the VMs access the disks on that pool >>> without a problem >>> >>> What can it be? >>> >>> The cluster is a mix of freshly installed nodes and upgraded ones, >>> all the 17 nodes (but one which is 6.4 but without any running VMs) >>> are running: >>> >>> root@pmx-14:~# pveversion -v >>> proxmox-ve: 7.1-1 (running kernel: 5.13.19-6-pve) >>> pve-manager: 7.1-12 (running version: 7.1-12/b3c09de3) >>> pve-kernel-helper: 7.1-14 >>> pve-kernel-5.13: 7.1-9 >>> pve-kernel-5.13.19-6-pve: 5.13.19-15 >>> pve-kernel-5.13.19-2-pve: 5.13.19-4 >>> ceph-fuse: 15.2.15-pve1 >>> corosync: 3.1.5-pve2 >>> criu: 3.15-1+pve-1 >>> glusterfs-client: 9.2-1 >>> ifupdown2: 3.1.0-1+pmx3 >>> ksm-control-daemon: 1.4-1 >>> libjs-extjs: 7.0.0-1 >>> libknet1: 1.22-pve2 >>> libproxmox-acme-perl: 1.4.1 >>> libproxmox-backup-qemu0: 1.2.0-1 >>> libpve-access-control: 7.1-7 >>> libpve-apiclient-perl: 3.2-1 >>> libpve-common-perl: 7.1-5 >>> libpve-guest-common-perl: 4.1-1 >>> libpve-http-server-perl: 4.1-1 >>> libpve-storage-perl: 7.1-1 >>> libspice-server1: 0.14.3-2.1 >>> lvm2: 2.03.11-2.1 >>> lxc-pve: 4.0.11-1 >>> lxcfs: 4.0.11-pve1 >>> novnc-pve: 1.3.0-2 >>> proxmox-backup-client: 2.1.5-1 >>> proxmox-backup-file-restore: 2.1.5-1 >>> proxmox-mini-journalreader: 1.3-1 >>> proxmox-widget-toolkit: 3.4-7 >>> pve-cluster: 7.1-3 >>> pve-container: 4.1-4 >>> pve-docs: 7.1-2 >>> pve-edk2-firmware: 3.20210831-2 >>> pve-firewall: 4.2-5 >>> pve-firmware: 3.3-6 >>> pve-ha-manager: 3.3-3 >>> pve-i18n: 2.6-2 >>> pve-qemu-kvm: 6.1.1-2 >>> pve-xtermjs: 4.16.0-1 >>> qemu-server: 7.1-4 >>> smartmontools: 7.2-1 >>> spiceterm: 3.2-2 >>> swtpm: 0.7.1~bpo11+1 >>> vncterm: 1.7-1 >>> zfsutils-linux: 2.1.4-pve1 >>> >>> I can provide other information if it's needed. >>> >>> Cheers >>> Iztok Gregori >>> >>> >> > > > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user