From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id 2D21C1FF17C
for <inbox@lore.proxmox.com>; Wed, 16 Apr 2025 10:20:35 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id A4B5D3006F;
Wed, 16 Apr 2025 10:20:33 +0200 (CEST)
Date: Wed, 16 Apr 2025 10:19:58 +0200
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
To: Fabian =?utf-8?Q?Gr=C3=BCnbichler?= <f.gruenbichler@proxmox.com>
Message-ID: <oydo653zz5qqbj73dwqlz6zqyutwqrfwveaqwqlpu5g3mx43vw@53vc7aqfluvx>
References: <20250211160825.254167-1-d.kral@proxmox.com>
<20250211160825.254167-24-d.kral@proxmox.com>
<2b549a4a-d9c4-47da-9eb2-2b782ff1dab1@proxmox.com>
<727ecb15-c82c-4ed4-bad3-676d54bddf3e@proxmox.com>
<wp6iyf3ylrdu2lgrcrmhfk57x2j3xgsyvzntmphw33j6asmjdp@6inpwihnpyvk>
<1744725224.c1q2wtkmtz.astroid@yuna.none>
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <1744725224.c1q2wtkmtz.astroid@yuna.none>
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.081 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: Re: [pve-devel] [PATCH container v2 03/11] alloc_disk: fail fast if
storage does not support content type rootdir
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Cc: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
On Tue, Apr 15, 2025 at 04:19:10PM +0200, Fabian Gr=FCnbichler wrote:
> On April 15, 2025 3:31 pm, Wolfgang Bumiller wrote:
> > On Tue, Apr 15, 2025 at 02:27:24PM +0200, Daniel Kral wrote:
> >> On 2/20/25 13:15, Fiona Ebner wrote:
> >> > I also noticed that we have no check against starting a container wi=
th
> >> > volumes on a storage that does not support 'rootdir'. We have such a
> >> > check for VMs IIRC. Prohibiting that would also be good, but maybe
> >> > something for PVE 9 where we can also check for misconfigured
> >> > containers/storages via the pve8to9 script up front so users can ada=
pt.
> >> =
> >> I'm preparing the v3 for this now, but I just noticed there actually i=
s a
> >> assertion for this since e6da5357cc ("fix #3421: allow custom storage
> >> plugins to support rootfs") if I'm not missing something here in
> >> __mountpoint_mount(...).
> >> =
> >> What I don't yet understand is why there is no similar check for this =
in
> >> __mountpoint_mount for subvolumes, e.g. I can't start the container if=
I
> >> have a mountpoint on a directory storage without 'rootdir' support, bu=
t I
> >> can do so if the mountpoint is on a zfs pool without 'rootdir' support.
> >> =
> >> Since starting the container results in
> >> =
> >> run_buffer: 571 Script exited with status 25
> >> lxc_init: 845 Failed to run lxc.hook.pre-start for container "101"
> >> __lxc_start: 2034 Failed to initialize container "101"
> >> TASK ERROR: startup for container '101' failed
> >> =
> >> for the WebGUI, I'll try to squeeze in a patch to make the error messa=
ge a
> >> little more readable if there's something going wrong when mounting.
> >> =
> >> ---
> >> =
> >> On another note, I've also noticed that if the root disk / mountpoint =
is
> >> already on a storage which does not support 'rootdir', the user is una=
ble to
> >> move it to another storage... Shouldn't we allow users to do that so t=
hey
> >> can easily move out error states? Either way, this can be a follow-up =
anway,
> >> so no need to make this patch series any longer.
> > =
> > The `rootdir` content type is generally a bit wonky currently.
> > The problem is we're mixing content types and "allowed contents"
> > together:
> > =
> > For ZFS and btrfs we use subvolumes which are their own *content type*:
> > "subvol".
> =
> no? that's just the format, not the volume type.
> =
> > =
> > For *other* directory storages, we have a special case for `size=3D0`
> > where we have a directory of an "unlimited" size, which is also
> > considered to be of *content type* "subvol".
> =
> same here
> =
> > In the other case we actually allocate the content type *image*,
> > *BUT*(!!!) the Plugin.pm's default `list_volumes` implementation will
> > artificially *name* it "rootdir" *if* the *associated VMID* is from a
> > container by querying the VM list via `PVE::Cluster::get_vmlist()`.
> > This is not something we can ask external storage plugin devs to do,
> > IMO.
> =
> yes
> =
> > *rootdir* as an actual *content type* does not *really* exist in
> > pve-storage, other than as a remnant from openvz times for directories
> > under the `rootdir/` directory on a directory storage, which I'm fairly
> > certain we don't support in pve-container...
> =
> it was actually `private`, and yes, this is not used anymore other than
> if you happen to have a system upgraded from PVE < 4 that still has
> images that are not usable in practice lying around..
> =
> > =
> > This means that currently what is referred to as the "rootdir" content
> > type is actually just the *permission* to put containers on the storage.
> > =
> > This is something we really need to fix up with PVE9 one way or another.
> > Personally I'd argue the content type should disappear entirely.
> > `list_volumes` should call use the correct type (images or subvol), and
> > the rootdir content permission should (and always / indepenent from the
> > storage and what *content type* we create) in pve-container's disk
> > allocation code.
> =
> yes, there are basically three levels involved
> =
> - content type on the storage itself - this defines what you can put on
> the storage, and it properly differentiates between rootdir and images
> - volume type as returned by parse_volname and consumed by other storage
> helpers (often abbreviated vtype) - this very often cannot
> differentiate whether a volume of type image is an image or a rootdir
> (this is something we want to change, but it's a tricky and long
> migration path)
> - image format (raw, qcow2, subvol, ..)
> =
> also see Max' pve-storage series thread, where some of this came up
> during review as well. but the TL;DR (for now) is: when looking at what
> a storage can store, check for rootdir for containers. when looking
> whether a volume has the expected type, check for rootdir or image. when
> actually handling it, also check the format to decide what to do with
> it.
> =
> the issue is that for properly differentiating between 'images' and
> 'rootdir' on the storage level, we need to split them both on the volid
> level (so we can differentiate without asking the storage) and on the
> storage level (so we can map back from the things we find on the storage
> to a volid). and somehow handle transitioning from the current mess to a
> cleanly separated state, and coordinate this with external plugins as
> well. Fiona and me had some chats about that already in the past, maybe
> we should sit down next week and make a concrete plan?
Yes we should. It would be good to clean this up for the trixie release.
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel