all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [RFC storage 00/26+10+3] unify vtype and content-type and
Date: Tue, 29 Jul 2025 13:15:13 +0200	[thread overview]
Message-ID: <20250729111557.136012-1-w.bumiller@proxmox.com> (raw)

This series is a first attempt at updating the storage API so that
- vtype == content type
- VM volumes have type "vm-vol"
- Container volumes have type "ct-vol"
- "images" is a generic "legacy" term which means "unknown OR vm-vol or
  ct-vol"
- remaining openvz-related 'rootdir' parts are removed
- rootdir is now intended to mean "unknown or ct-vol"

There are several reasons for adding 2 new types, rather than making the
old ones more strict:
- The "rootdir" vs "images" distinction in `list_volumes` only happened
  if our code to check the actually existing guest type was used. This
  could not be enforced for external storage plugins and is a weird and
  surprising API quirk.
- The "rootdir" *vtype* was otherwise *never* used as it referred to old
  openvz 'private/' subdirectories.
- The container code allocated "images", not "rootdirs".
- Since containers were allocated as type "images", the `content-dir`
  option for "rootdir" never had any effect. (You can now set
  content-dirs for types 'vm-vol' and 'ct-vol' which will be honored for
  any new disks allocated from the update pve-container and qemu-server
  code.)

API Notes:
  - vtype parameters are added to several API methods - but they are
    optional
  - list_images now also includes a `vtype` in its return value, but
    only if it is NOT `images` (so legacy volumes are returned unchanged
    via the API).
  - old plugins should still function as they did previously but keep
    using the legacy "images" vtype/content-type

Storage notes:

  Storages now need to be able to tell vm/ct volumes apart from
  "images", for which this not only introduced separate directories, but
  also adapts the volume id naming scheme. In order to *somewhat* void
  running into rogue `/^vm-/` regexes affecting new style volumes, the
  new prefixes are the reverse of the type names: `vol-vm-` and
  `vol-ct-`. For base images, the entire name is prefixed with `base-`
  instead of, like previously, doing `s/vm-/base-/`.

Patch notes:

  Initially I did the vtype separately from the content type, but the
  series became a bit of a back & forth. This version is mostly split by
  functionality as it seemed nicer for reviews and might sort of work
  for external plugin writers as a guide. Therefore patches add vtype
  parameters without adapting all the callers. (This *should* mostly be
  fine since they are optional, though...) And Tests are only changed as
  a whole at the end.

  For qemu-server, most of the tests were updated via `sed`.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


             reply	other threads:[~2025-07-29 11:16 UTC|newest]

Thread overview: 74+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-07-29 11:15 Wolfgang Bumiller [this message]
2025-07-29 11:15 ` [pve-devel] [PATCH storage 01/26] btrfs: remove unnecessary mkpath call Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 02/26] parse_volname: remove openvz 'rootdir' case Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 03/26] drop rootdir case in path_to_volume_id Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 04/26] escape dirs in path_to_volume_id regexes Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 05/26] tests: drop rootdir/ tests Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 06/26] common: use v5.36 Wolfgang Bumiller
2025-07-29 13:59   ` Fiona Ebner
2025-07-29 14:42     ` Thomas Lamprecht
2025-07-29 11:15 ` [pve-devel] [PATCH storage 07/26] common: add pve-storage-vtype standard option with new types Wolfgang Bumiller
2025-07-29 13:50   ` Fiona Ebner
2025-07-29 11:15 ` [pve-devel] [PATCH storage 08/26] prepare for vm-vol and ct-vol content and vtypes Wolfgang Bumiller
2025-07-30  8:38   ` Fabian Grünbichler
2025-08-08 12:01     ` Wolfgang Bumiller
2025-07-30  9:14   ` Fabian Grünbichler
2025-08-08 12:05     ` Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 09/26] plugins: add new content types to all plugindata Wolfgang Bumiller
2025-07-30  8:38   ` Fabian Grünbichler
2025-08-08 12:10     ` Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 10/26] plugins: update volname parsing for new naming convention Wolfgang Bumiller
2025-07-30  8:37   ` Fabian Grünbichler
2025-07-30  8:53     ` Wolfgang Bumiller
2025-07-30  8:58       ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 11/26] plugin: add vm/ct-vol to 'local' storage default content types Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 12/26] plugin: support new vtypes in activate_storage checks Wolfgang Bumiller
2025-07-30  8:36   ` Fabian Grünbichler
2025-08-08 13:16     ` Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 13/26] plugin, btrfs: update list_images and list_volumes Wolfgang Bumiller
2025-07-30  8:36   ` Fabian Grünbichler
2025-07-30  8:41     ` Fiona Ebner
2025-07-29 11:15 ` [pve-devel] [PATCH storage 14/26] plugins: update image/volume listing to support new types Wolfgang Bumiller
2025-07-30  8:36   ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 15/26] common: add is_volume_type and is_type_change_allowed helpers Wolfgang Bumiller
2025-07-30  9:01   ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 16/26] common: add volume_type_from_name convenience helper Wolfgang Bumiller
2025-07-30  8:36   ` Fabian Grünbichler
2025-07-30  9:09     ` Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 17/26] plugins: add vtype parameter to alloc_image Wolfgang Bumiller
2025-07-30  9:24   ` Fabian Grünbichler
2025-07-30 14:00   ` Max R. Carrara
2025-07-30 14:05     ` Max R. Carrara
2025-07-30 14:26       ` Fabian Grünbichler
2025-07-30 14:49         ` Max R. Carrara
2025-07-30 15:01           ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 18/26] plugins: update create_base methods Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 19/26] plugins: update clone_image methods Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 20/26] plugins: update rename_volumes methods Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 21/26] plugins: update volume_import methods Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 22/26] zfs: update 'path' method for new naming scheme Wolfgang Bumiller
2025-07-30  9:31   ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 23/26] pvesm: add vtype parameter to import command Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 24/26] api: add vtype parameter to create call Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 25/26] update tests Wolfgang Bumiller
2025-07-29 16:33   ` Max R. Carrara
2025-07-29 11:15 ` [pve-devel] [PATCH storage 26/26] update ApiChangeLog Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH container 1/3] add vtype to vdisk_alloc and vdisk_clone calls Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH container 2/3] expect 'vm-vol' vtype in get_replicatable_volumes Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH container 3/3] add vtype to rename_volume call Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 01/10] add vtype to vdisk_alloc and vdisk_clone calls Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 02/10] add vtype parameter to rename_volume call Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 03/10] expect 'vm-vol' vtype in get_replicatable_volumes Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 04/10] expect 'vm-vol' vtype wherever 'images' was expected Wolfgang Bumiller
2025-07-30  8:40   ` Fabian Grünbichler
2025-07-30  9:17   ` Fiona Ebner
2025-07-30  9:33     ` Fiona Ebner
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 05/10] tests: update QmMock to support vtypes Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 06/10] tests: scripted: update tests to new vtypes and paths Wolfgang Bumiller
2025-07-29 14:13   ` Max R. Carrara
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 07/10] make tidy Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 08/10] tests: fixup restore test to use new volume naming scheme Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 09/10] tests: update remaining tests to new snapshot paths Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 10/10] tests: regenerate cfg2cmd files Wolfgang Bumiller
2025-07-29 14:19   ` Max R. Carrara
2025-07-29 15:34 ` [pve-devel] partially-applied: [RFC storage 00/26+10+3] unify vtype and content-type and Fiona Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250729111557.136012-1-w.bumiller@proxmox.com \
    --to=w.bumiller@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal