public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [RFC storage 00/26+10+3] unify vtype and content-type and
@ 2025-07-29 11:15 Wolfgang Bumiller
  2025-07-29 11:15 ` [pve-devel] [PATCH storage 01/26] btrfs: remove unnecessary mkpath call Wolfgang Bumiller
                   ` (39 more replies)
  0 siblings, 40 replies; 74+ messages in thread
From: Wolfgang Bumiller @ 2025-07-29 11:15 UTC (permalink / raw)
  To: pve-devel

This series is a first attempt at updating the storage API so that
- vtype == content type
- VM volumes have type "vm-vol"
- Container volumes have type "ct-vol"
- "images" is a generic "legacy" term which means "unknown OR vm-vol or
  ct-vol"
- remaining openvz-related 'rootdir' parts are removed
- rootdir is now intended to mean "unknown or ct-vol"

There are several reasons for adding 2 new types, rather than making the
old ones more strict:
- The "rootdir" vs "images" distinction in `list_volumes` only happened
  if our code to check the actually existing guest type was used. This
  could not be enforced for external storage plugins and is a weird and
  surprising API quirk.
- The "rootdir" *vtype* was otherwise *never* used as it referred to old
  openvz 'private/' subdirectories.
- The container code allocated "images", not "rootdirs".
- Since containers were allocated as type "images", the `content-dir`
  option for "rootdir" never had any effect. (You can now set
  content-dirs for types 'vm-vol' and 'ct-vol' which will be honored for
  any new disks allocated from the update pve-container and qemu-server
  code.)

API Notes:
  - vtype parameters are added to several API methods - but they are
    optional
  - list_images now also includes a `vtype` in its return value, but
    only if it is NOT `images` (so legacy volumes are returned unchanged
    via the API).
  - old plugins should still function as they did previously but keep
    using the legacy "images" vtype/content-type

Storage notes:

  Storages now need to be able to tell vm/ct volumes apart from
  "images", for which this not only introduced separate directories, but
  also adapts the volume id naming scheme. In order to *somewhat* void
  running into rogue `/^vm-/` regexes affecting new style volumes, the
  new prefixes are the reverse of the type names: `vol-vm-` and
  `vol-ct-`. For base images, the entire name is prefixed with `base-`
  instead of, like previously, doing `s/vm-/base-/`.

Patch notes:

  Initially I did the vtype separately from the content type, but the
  series became a bit of a back & forth. This version is mostly split by
  functionality as it seemed nicer for reviews and might sort of work
  for external plugin writers as a guide. Therefore patches add vtype
  parameters without adapting all the callers. (This *should* mostly be
  fine since they are optional, though...) And Tests are only changed as
  a whole at the end.

  For qemu-server, most of the tests were updated via `sed`.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 74+ messages in thread

end of thread, other threads:[~2025-08-08 13:15 UTC | newest]

Thread overview: 74+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-07-29 11:15 [pve-devel] [RFC storage 00/26+10+3] unify vtype and content-type and Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 01/26] btrfs: remove unnecessary mkpath call Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 02/26] parse_volname: remove openvz 'rootdir' case Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 03/26] drop rootdir case in path_to_volume_id Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 04/26] escape dirs in path_to_volume_id regexes Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 05/26] tests: drop rootdir/ tests Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 06/26] common: use v5.36 Wolfgang Bumiller
2025-07-29 13:59   ` Fiona Ebner
2025-07-29 14:42     ` Thomas Lamprecht
2025-07-29 11:15 ` [pve-devel] [PATCH storage 07/26] common: add pve-storage-vtype standard option with new types Wolfgang Bumiller
2025-07-29 13:50   ` Fiona Ebner
2025-07-29 11:15 ` [pve-devel] [PATCH storage 08/26] prepare for vm-vol and ct-vol content and vtypes Wolfgang Bumiller
2025-07-30  8:38   ` Fabian Grünbichler
2025-08-08 12:01     ` Wolfgang Bumiller
2025-07-30  9:14   ` Fabian Grünbichler
2025-08-08 12:05     ` Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 09/26] plugins: add new content types to all plugindata Wolfgang Bumiller
2025-07-30  8:38   ` Fabian Grünbichler
2025-08-08 12:10     ` Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 10/26] plugins: update volname parsing for new naming convention Wolfgang Bumiller
2025-07-30  8:37   ` Fabian Grünbichler
2025-07-30  8:53     ` Wolfgang Bumiller
2025-07-30  8:58       ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 11/26] plugin: add vm/ct-vol to 'local' storage default content types Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 12/26] plugin: support new vtypes in activate_storage checks Wolfgang Bumiller
2025-07-30  8:36   ` Fabian Grünbichler
2025-08-08 13:16     ` Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 13/26] plugin, btrfs: update list_images and list_volumes Wolfgang Bumiller
2025-07-30  8:36   ` Fabian Grünbichler
2025-07-30  8:41     ` Fiona Ebner
2025-07-29 11:15 ` [pve-devel] [PATCH storage 14/26] plugins: update image/volume listing to support new types Wolfgang Bumiller
2025-07-30  8:36   ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 15/26] common: add is_volume_type and is_type_change_allowed helpers Wolfgang Bumiller
2025-07-30  9:01   ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 16/26] common: add volume_type_from_name convenience helper Wolfgang Bumiller
2025-07-30  8:36   ` Fabian Grünbichler
2025-07-30  9:09     ` Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 17/26] plugins: add vtype parameter to alloc_image Wolfgang Bumiller
2025-07-30  9:24   ` Fabian Grünbichler
2025-07-30 14:00   ` Max R. Carrara
2025-07-30 14:05     ` Max R. Carrara
2025-07-30 14:26       ` Fabian Grünbichler
2025-07-30 14:49         ` Max R. Carrara
2025-07-30 15:01           ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 18/26] plugins: update create_base methods Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 19/26] plugins: update clone_image methods Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 20/26] plugins: update rename_volumes methods Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 21/26] plugins: update volume_import methods Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 22/26] zfs: update 'path' method for new naming scheme Wolfgang Bumiller
2025-07-30  9:31   ` Fabian Grünbichler
2025-07-29 11:15 ` [pve-devel] [PATCH storage 23/26] pvesm: add vtype parameter to import command Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 24/26] api: add vtype parameter to create call Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH storage 25/26] update tests Wolfgang Bumiller
2025-07-29 16:33   ` Max R. Carrara
2025-07-29 11:15 ` [pve-devel] [PATCH storage 26/26] update ApiChangeLog Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH container 1/3] add vtype to vdisk_alloc and vdisk_clone calls Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH container 2/3] expect 'vm-vol' vtype in get_replicatable_volumes Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH container 3/3] add vtype to rename_volume call Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 01/10] add vtype to vdisk_alloc and vdisk_clone calls Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 02/10] add vtype parameter to rename_volume call Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 03/10] expect 'vm-vol' vtype in get_replicatable_volumes Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 04/10] expect 'vm-vol' vtype wherever 'images' was expected Wolfgang Bumiller
2025-07-30  8:40   ` Fabian Grünbichler
2025-07-30  9:17   ` Fiona Ebner
2025-07-30  9:33     ` Fiona Ebner
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 05/10] tests: update QmMock to support vtypes Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 06/10] tests: scripted: update tests to new vtypes and paths Wolfgang Bumiller
2025-07-29 14:13   ` Max R. Carrara
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 07/10] make tidy Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 08/10] tests: fixup restore test to use new volume naming scheme Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 09/10] tests: update remaining tests to new snapshot paths Wolfgang Bumiller
2025-07-29 11:15 ` [pve-devel] [PATCH qemu-server 10/10] tests: regenerate cfg2cmd files Wolfgang Bumiller
2025-07-29 14:19   ` Max R. Carrara
2025-07-29 15:34 ` [pve-devel] partially-applied: [RFC storage 00/26+10+3] unify vtype and content-type and Fiona Ebner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal