From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id E574E1FF09C for ; Mon, 21 Sep 2026 14:26:45 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id C0AD221432; Mon, 21 Sep 2026 14:26:45 +0200 (CEST) From: "Max R. Carrara" To: pve-devel@lists.proxmox.com Subject: [PATCH pve-qemu 1/1] block: vmdk: support raw device mappings in vmdk format Date: Mon, 21 Sep 2026 14:26:19 +0200 Message-ID: <20260921122624.304781-1-m.carrara@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1789993584758 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.500 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: W4EVJPHMNUEAOT2WBRFLTHQMW4LJH3SL X-Message-ID-Hash: W4EVJPHMNUEAOT2WBRFLTHQMW4LJH3SL X-MailFrom: m.carrara@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Implement support for raw device mappings (RDMs) in QEMU's VMDK block driver. Handle both virtual and physical compatibility modes. This allows us to import RDM disks from ESXi through our FUSE layer without any additional changes inside Proxmox VE. According to the vSphere docs, an RDM provides a mechanism for a VM to have direct access to a LUN on physical storage [rdm]. What it really does is simply pass through a block device to a VM. The VMDK for an RDM will reference a so-called "pointer file" in its extent description, with the type `VMFSRDM`, which is created alongide the VMDK file. This pointer file then points to the block device, kind of like a symlink. - Physical compatibility mode `vmkfstools --createrdmpassthru /vmfs/devices/disks/... [PATH TO NEW VMDK]` `createType="vmfsPassthroughRawDeviceMap"` This is the simplest of the two modes; the block device is simply passed through to the VM. RDMs in this compat mode are excluded from snapshots. - Virtual compatibility mode `vmkfstools --createrdm /vmfs/devices/disks/... [PATH TO NEW VMDK]` `createType="vmfsRawDeviceMap"` This mode allows RDMs to be included in snapshots. When making a snapshot, the new delta is written to a SESPARE file on the datastore instead, and the data on the passthrough disk will remain unmodified from that point onwards. This is somewhat strange, since all data written from the first snapshot onwards will exist on the datastore instead on the disk that was passed through, but alas. In order to support both of these modes, make the VMDK block driver aware of the `vmfsPassthroughRawDeviceMap` and `vmfsRawDeviceMap` createTypes. Then, handle the `VMFSRDM` extent type in the extents parsers. The `VMFSRDM` extent type works similar to the regular `VMFS` and `FLAT` types. The entire extent's description consists of four fields; the referenced file (--> block device) stores no metadata and can therefore be treated the same way as a `FLAT` extent. [rdm]: https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/7-0/vsphere-storage/raw-device-mapping-in-vsphere.html Signed-off-by: Max R. Carrara --- ...rt-raw-device-mappings-in-vmdk-forma.patch | 119 ++++++++++++++++++ debian/patches/series | 1 + 2 files changed, 120 insertions(+) create mode 100644 debian/patches/pve/0047-block-vmdk-support-raw-device-mappings-in-vmdk-forma.patch diff --git a/debian/patches/pve/0047-block-vmdk-support-raw-device-mappings-in-vmdk-forma.patch b/debian/patches/pve/0047-block-vmdk-support-raw-device-mappings-in-vmdk-forma.patch new file mode 100644 index 0000000..ef055cd --- /dev/null +++ b/debian/patches/pve/0047-block-vmdk-support-raw-device-mappings-in-vmdk-forma.patch @@ -0,0 +1,119 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: "Max R. Carrara" +Date: Mon, 21 Sep 2026 11:30:05 +0200 +Subject: [PATCH] block: vmdk: support raw device mappings in vmdk format + +Implement support for raw device mappings (RDMs) in QEMU's VMDK +block driver. Handle both virtual and physical compatibility modes. +This allows us to import RDM disks from ESXi through our FUSE layer +without any additional changes inside Proxmox VE. + +According to the vSphere docs, an RDM provides a mechanism for a VM to +have direct access to a LUN on physical storage [rdm]. What it really +does is simply pass through a block device to a VM. The VMDK for an +RDM will reference a so-called "pointer file" in its extent +description, with the type `VMFSRDM`, which is created alongide the +VMDK file. This pointer file then points to the block device, kind of +like a symlink. + +- Physical compatibility mode + + `vmkfstools --createrdmpassthru /vmfs/devices/disks/... [PATH TO NEW VMDK]` + + `createType="vmfsPassthroughRawDeviceMap"` + + This is the simplest of the two modes; the block device is simply + passed through to the VM. RDMs in this compat mode are excluded from + snapshots. + +- Virtual compatibility mode + + `vmkfstools --createrdm /vmfs/devices/disks/... [PATH TO NEW VMDK]` + + `createType="vmfsRawDeviceMap"` + + This mode allows RDMs to be included in snapshots. When making a + snapshot, the new delta is written to a SESPARE file on the + datastore instead, and the data on the passthrough disk will remain + unmodified from that point onwards. + + This is somewhat strange, since all data written from the first + snapshot onwards will exist on the datastore instead on the disk + that was passed through, but alas. + +In order to support both of these modes, make the VMDK block driver +aware of the `vmfsPassthroughRawDeviceMap` and `vmfsRawDeviceMap` +createTypes. Then, handle the `VMFSRDM` extent type in the extents +parsers. + +The `VMFSRDM` extent type works similar to the regular `VMFS` and +`FLAT` types. The entire extent's description consists of four fields; +the referenced file (--> block device) stores no metadata and can +therefore be treated the same way as a `FLAT` extent. + +[rdm]: https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/7-0/vsphere-storage/raw-device-mapping-in-vsphere.html + +Signed-off-by: Max R. Carrara +--- + block/vmdk.c | 13 ++++++++----- + 1 file changed, 8 insertions(+), 5 deletions(-) + +diff --git a/block/vmdk.c b/block/vmdk.c +index cd8b4ec7c8..22f67e351b 100644 +--- a/block/vmdk.c ++++ b/block/vmdk.c +@@ -1174,6 +1174,7 @@ vmdk_parse_extents(const char *desc, BlockDriverState *bs, QDict *options, + * RW [size in sectors] VMFS "file-name.vmdk" + * RW [size in sectors] VMFSSPARSE "file-name.vmdk" + * RW [size in sectors] SESPARSE "file-name.vmdk" ++ * RW [size in sectors] VMFSRDM "file-name.vmdk" + */ + flat_offset = -1; + matches = sscanf(p, "%10s %" SCNd64 " %10s \"%511[^\n\r\"]\" %" SCNd64, +@@ -1184,7 +1185,7 @@ vmdk_parse_extents(const char *desc, BlockDriverState *bs, QDict *options, + if (matches != 5 || flat_offset < 0) { + goto invalid; + } +- } else if (!strcmp(type, "VMFS")) { ++ } else if (!strcmp(type, "VMFS") || !strcmp(type, "VMFSRDM")) { + if (matches == 4) { + flat_offset = 0; + } else { +@@ -1197,7 +1198,7 @@ vmdk_parse_extents(const char *desc, BlockDriverState *bs, QDict *options, + if (sectors <= 0 || + (strcmp(type, "FLAT") && strcmp(type, "SPARSE") && + strcmp(type, "VMFS") && strcmp(type, "VMFSSPARSE") && +- strcmp(type, "SESPARSE")) || ++ strcmp(type, "SESPARSE") && strcmp(type, "VMFSRDM")) || + (strcmp(access, "RW"))) { + continue; + } +@@ -1224,7 +1225,7 @@ vmdk_parse_extents(const char *desc, BlockDriverState *bs, QDict *options, + assert(ret < 32); + + extent_role = BDRV_CHILD_DATA; +- if (strcmp(type, "FLAT") != 0 && strcmp(type, "VMFS") != 0) { ++ if (strcmp(type, "FLAT") != 0 && strcmp(type, "VMFS") != 0 && strcmp(type, "VMFSRDM")) { + /* non-flat extents have metadata */ + extent_role |= BDRV_CHILD_METADATA; + } +@@ -1242,7 +1243,7 @@ vmdk_parse_extents(const char *desc, BlockDriverState *bs, QDict *options, + } + + /* save to extents array */ +- if (!strcmp(type, "FLAT") || !strcmp(type, "VMFS")) { ++ if (!strcmp(type, "FLAT") || !strcmp(type, "VMFS") || !strcmp(type, "VMFSRDM")) { + /* FLAT extent */ + + ret = vmdk_add_extent(bs, extent_file, true, sectors, +@@ -1334,7 +1335,9 @@ vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf, QDict *options, + strcmp(ct, "vmfsSparse") && + strcmp(ct, "seSparse") && + strcmp(ct, "twoGbMaxExtentSparse") && +- strcmp(ct, "twoGbMaxExtentFlat")) { ++ strcmp(ct, "twoGbMaxExtentFlat") && ++ strcmp(ct, "vmfsRawDeviceMap") && ++ strcmp(ct, "vmfsPassthroughRawDeviceMap")) { + error_setg(errp, "Unsupported image type '%s'", ct); + ret = -ENOTSUP; + goto exit; diff --git a/debian/patches/series b/debian/patches/series index 3bc3548..a7f6aa3 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -61,3 +61,4 @@ pve/0043-PVE-backup-prepare-for-the-switch-to-using-blockdev-.patch pve/0044-savevm-async-reuse-migration-blocker-check-for-snaps.patch pve/0045-pbs-restore-add-no-cache-flag-to-skip-host-page-cach.patch pve/0046-ui-spice-core-work-around-broken-input-cleanup-in-li.patch +pve/0047-block-vmdk-support-raw-device-mappings-in-vmdk-forma.patch -- 2.47.3