[PATCH guest-common/qemu-server/docs/manager v3 0/11] Virtiofs improvements

[PATCH guest-common/qemu-server/docs/manager v3 0/11] Virtiofs improvements

[Markus Frank]

It seems the migration-mode find-paths is enabled by default:
https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/src/main.rs?ref_type=heads#L359

I would still disable live migration by default on our side so that
users have to actively choose the migration mode and learn about the
limitations and risks.

The VM and virtiofsd needs to be stopped and started again for the
migration mode change to take effect.

overall v3 changes:
* improved code based on Daniel Kral's review on v2, see individual
  patches
* always start virtiofsd with '--migration-on-error=guest-error' as a
  safety measure
* improved the warning messages in pve-manager
* added readonly to parameter description table in the pve-docs patch


guest-common:

Markus Frank (1):
  mapping: dir: add 'live-migration-method' parameter

 src/PVE/Mapping/Dir.pm | 11 +++++++++++
 1 file changed, 11 insertions(+)


qemu-server:

Markus Frank (3):
  fix #6370: virtiofs: add support for thread-pool-size option
  virtiofs: add readonly option
  virtiofs: add live migration support

 src/PVE/QemuMigrate/Helpers.pm | 14 +++++++++++---
 src/PVE/QemuServer/Virtiofs.pm | 35 ++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+), 3 deletions(-)


docs:

Markus Frank (4):
  virtiofs: add explanation for cache=metadata behavior
  virtiofs: add table for optional parameters
  virtiofs: add description for thread-pool-size and readonly
  virtiofs: add documentation for live migration

 qm.adoc | 53 +++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 45 insertions(+), 8 deletions(-)


manager:

Markus Frank (3):
  fix #6370: ui: virtiofs edit: add support for thread-pool-size option
  virtiofs edit: add support for readonly option
  directory mapping: add live-migration-method option for virtiofs

 www/manager6/Makefile             |  1 +
 www/manager6/Messages.js          | 15 ++++++++
 www/manager6/dc/DirMapView.js     |  8 ++++
 www/manager6/qemu/VirtiofsEdit.js | 15 ++++++++
 www/manager6/window/DirMapEdit.js | 63 +++++++++++++++++++++++++++++--
 www/manager6/window/Migrate.js    | 13 +++++++
 6 files changed, 112 insertions(+), 3 deletions(-)
 create mode 100644 www/manager6/Messages.js

-- 
2.47.3

[PATCH guest-common v3 1/11] mapping: dir: add 'live-migration-method' parameter

[Markus Frank]

If a live migration method is selected, qemu-server starts virtiofsd
with the flags to enable live migration using the chosen method.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3 changes:
* removed default option for live-migration-method
* added `make tidy` changes

 src/PVE/Mapping/Dir.pm | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/src/PVE/Mapping/Dir.pm b/src/PVE/Mapping/Dir.pm
index 5ff02d1..6a61237 100644
--- a/src/PVE/Mapping/Dir.pm
+++ b/src/PVE/Mapping/Dir.pm
@@ -74,6 +74,16 @@ my $defaultData = {
             description => "The ID of the directory mapping",
             format => 'pve-configid',
         },
+        'live-migration-method' => {
+            description =>
+                "Allow live-migration when using the directory with a virtiofs device."
+                . " Ensure you are using the same shared directory on all hosts."
+                . " Available migration methods are 'file-handles' and 'find-path'."
+                . " Live-migration is disable by default.",
+            type => 'string',
+            optional => 1,
+            enum => ['file-handles', 'find-paths'],
+        },
         description => {
             type => 'string',
             description => "Description of the directory mapping",
@@ -99,6 +109,7 @@ sub private {
 sub options {
     return {
         description => { optional => 1 },
+        'live-migration-method' => { optional => 1 },
         map => {},
     };
 }
-- 
2.47.3

[PATCH qemu-server v3 2/11] fix #6370: virtiofs: add support for thread-pool-size option

[Markus Frank]

Allows tuning the number of worker threads used by virtiofsd.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3 changes:
* added defined() to post-if

 src/PVE/QemuServer/Virtiofs.pm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/PVE/QemuServer/Virtiofs.pm b/src/PVE/QemuServer/Virtiofs.pm
index b181db5f..ba8b0dab 100644
--- a/src/PVE/QemuServer/Virtiofs.pm
+++ b/src/PVE/QemuServer/Virtiofs.pm
@@ -60,6 +60,14 @@ my $virtiofs_fmt = {
         default => 0,
         optional => 1,
     },
+    'thread-pool-size' => {
+        type => 'integer',
+        description => "Controls the maximum number of worker threads used by virtiofsd to"
+            . " handle requests. A value of '0' disables the thread pool.",
+        default => 0,
+        minimum => 0,
+        optional => 1,
+    },
 };
 PVE::JSONSchema::register_format('pve-qm-virtiofs', $virtiofs_fmt);
 
@@ -180,6 +188,8 @@ sub start_virtiofsd {
             my $cmd = [$virtiofsd_bin, "--fd=$fd", "--shared-dir=$path"];
             push @$cmd, '--xattr' if $virtiofs->{'expose-xattr'};
             push @$cmd, '--posix-acl' if $virtiofs->{'expose-acl'};
+            push @$cmd, '--thread-pool-size=' . $virtiofs->{'thread-pool-size'}
+                if defined($virtiofs->{'thread-pool-size'});
             push @$cmd, '--announce-submounts';
             push @$cmd, '--allow-direct-io' if $virtiofs->{'direct-io'};
             push @$cmd, '--cache=' . $virtiofs->{cache} if $virtiofs->{cache};
-- 
2.47.3

[PATCH qemu-server v3 3/11] virtiofs: add readonly option

[Markus Frank]

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3: no changes

 src/PVE/QemuServer/Virtiofs.pm | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/PVE/QemuServer/Virtiofs.pm b/src/PVE/QemuServer/Virtiofs.pm
index ba8b0dab..8567d2af 100644
--- a/src/PVE/QemuServer/Virtiofs.pm
+++ b/src/PVE/QemuServer/Virtiofs.pm
@@ -47,6 +47,12 @@ my $virtiofs_fmt = {
         default => 0,
         optional => 1,
     },
+    'readonly' => {
+        type => 'boolean',
+        description => "Prevent write accesses from the guest.",
+        default => 0,
+        optional => 1,
+    },
     'expose-xattr' => {
         type => 'boolean',
         description => "Enable support for extended attributes for this mount.",
@@ -192,6 +198,7 @@ sub start_virtiofsd {
                 if defined($virtiofs->{'thread-pool-size'});
             push @$cmd, '--announce-submounts';
             push @$cmd, '--allow-direct-io' if $virtiofs->{'direct-io'};
+            push @$cmd, '--readonly' if $virtiofs->{readonly};
             push @$cmd, '--cache=' . $virtiofs->{cache} if $virtiofs->{cache};
             push @$cmd, '--inode-file-handles=prefer' if $prefer_inode_fh;
             push @$cmd, '--syslog';
-- 
2.47.3

[PATCH qemu-server v3 4/11] virtiofs: add live migration support

[Markus Frank]

There are two migration methods with virtiofsd.

Virtiofsd in the 'find-paths' migration mode, tries to find the path
inside of the shared directory for every filesystem object that must be
transferred to the destination.
Add two extra parameters that verify file handles and confirm paths.

In the 'file-handles' migration mode every filesystem object is
converted to a file handle. This migration mode requires the
DAC_READ_SEARCH capability.

Add a parameter so that migration does not abort if an error occurs.
Aborting on error would result in a stopped VM on the destination node.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3 changes:
* removed the unnecessary 'if' around the $dirid initialisation
* changed if to elsif
* always start virtiofsd with '--migration-on-error=guest-error' as a
  safety measure

 src/PVE/QemuMigrate/Helpers.pm | 14 +++++++++++---
 src/PVE/QemuServer/Virtiofs.pm | 18 ++++++++++++++++++
 2 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/src/PVE/QemuMigrate/Helpers.pm b/src/PVE/QemuMigrate/Helpers.pm
index 466517da..6c6294dd 100644
--- a/src/PVE/QemuMigrate/Helpers.pm
+++ b/src/PVE/QemuMigrate/Helpers.pm
@@ -21,7 +21,6 @@ sub check_non_migratable_resources {
     if ($state) {
         push @blockers, "amd-sev" if $conf->{"amd-sev"};
         push @blockers, "intel-tdx" if $conf->{"intel-tdx"};
-        push @blockers, "virtiofs" if PVE::QemuServer::Virtiofs::virtiofs_enabled($conf);
     }
 
     if (scalar(@blockers) && !$noerr) {
@@ -100,8 +99,17 @@ sub check_local_resources {
         }
         if ($k =~ m/^virtiofs/) {
             my $entry = parse_property_string('pve-qm-virtiofs', $conf->{$k});
-            $add_missing_mapping->('dir', $k, $entry->{dirid});
-            $mapped_res->{$k} = { name => $entry->{dirid} };
+
+            my $dirid = $entry->{dirid};
+            $add_missing_mapping->('dir', $k, $dirid);
+            my $mapped_device = { name => $dirid };
+            $mapped_res->{$k} = $mapped_device;
+
+            if ($dir_map->{ids}->{$dirid}->{'live-migration-method'}) {
+                $mapped_device->{'live-migration'} = 1;
+                # don't add mapped device with live migration as blocker
+                next;
+            }
         }
         # sockets are safe: they will recreated be on the target side post-migrate
         next if $k =~ m/^serial/ && ($conf->{$k} eq 'socket');
diff --git a/src/PVE/QemuServer/Virtiofs.pm b/src/PVE/QemuServer/Virtiofs.pm
index 8567d2af..f9d7136a 100644
--- a/src/PVE/QemuServer/Virtiofs.pm
+++ b/src/PVE/QemuServer/Virtiofs.pm
@@ -177,6 +177,9 @@ sub start_virtiofsd {
 
     my $dir_cfg = PVE::Mapping::Dir::find_on_current_node($virtiofs->{dirid});
 
+    my $dirid = $virtiofs->{dirid};
+    my $dir_properties = PVE::Mapping::Dir::config()->{ids}->{$dirid};
+
     my $virtiofsd_bin = '/usr/libexec/virtiofsd';
     if (!-f $virtiofsd_bin) {
         die "virtiofsd is not installed. To use virtio-fs, install virtiofsd via apt.\n";
@@ -201,6 +204,21 @@ sub start_virtiofsd {
             push @$cmd, '--readonly' if $virtiofs->{readonly};
             push @$cmd, '--cache=' . $virtiofs->{cache} if $virtiofs->{cache};
             push @$cmd, '--inode-file-handles=prefer' if $prefer_inode_fh;
+
+            # migration-on-error=abort would result in a stopped VM on the target node
+            push @$cmd, '--migration-on-error=guest-error';
+
+            if (my $method = $dir_properties->{'live-migration-method'}) {
+                if ($method eq 'find-paths') {
+                    push @$cmd, '--migration-mode=find-paths';
+                    push @$cmd, '--migration-verify-handles';
+                    push @$cmd, '--migration-confirm-paths';
+                } elsif ($method eq 'file-handles') {
+                    push @$cmd, '--migration-mode=file-handles';
+                    push @$cmd, '--modcaps=+dac_read_search';
+                }
+            }
+
             push @$cmd, '--syslog';
             exec(@$cmd);
         } elsif (!defined($pid2)) {
-- 
2.47.3

[PATCH docs v3 5/11] virtiofs: add explanation for cache=metadata behavior

[Markus Frank]

Adds a description of the `cache=metadata` mode and references the
upstream patch that introduced it.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3: no changes

 qm.adoc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/qm.adoc b/qm.adoc
index e6b7918..46edeb3 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -1378,7 +1378,9 @@ has been configured in the resource mapping. Additionally, you can set the
 `cache` option to either `always`, `never`, `metadata`, or `auto` (default:
 `auto`), depending on your requirements. How the different caching modes behave
 can be read https://lwn.net/Articles/774495/[here under the "Caching Modes"
-section].
+section]. The https://gitlab.com/virtio-fs/virtiofsd/-/commit/598ca8e[`metadata`]
+policy is similar to `never`, but it allows guests to cache filesystem metadata,
+such as dcache and attr.
 
 The `virtiofsd` supports ACL and xattr passthrough (can be enabled with the
 `expose-acl` and `expose-xattr` options), allowing the guest to access ACLs and
-- 
2.47.3

[PATCH docs v3 6/11] virtiofs: add table for optional parameters

[Markus Frank]

Removes redundant explanatory text now covered by the table.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3: no changes

 qm.adoc | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/qm.adoc b/qm.adoc
index 46edeb3..c0ca466 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -1388,12 +1388,16 @@ xattrs if the underlying host filesystem supports them, but they must also be
 compatible with the guest filesystem (for example most Linux filesystems support
 ACLs, while Windows filesystems do not).
 
-The `expose-acl` option automatically implies `expose-xattr`, that is, it makes
-no difference if you set `expose-xattr` to `0` if `expose-acl` is set to `1`.
-
-If you want virtiofs to honor the `O_DIRECT` flag, you can set the `direct-io`
-parameter to `1` (default: `0`). This will degrade performance, but is useful if
-applications do their own caching.
+All Optional Parameters:
+
+[cols="1,3,1", options="header"]
+|===
+|Parameter |Description |Default
+|`cache` |Sets caching mode: `always`, `never`, `metadata`, or `auto`. |`auto`
+|`expose-acl` |Enables ACL passthrough; implies `expose-xattr`. Supported only on compatible filesystems. |`0`
+|`expose-xattr` |Enables xattr passthrough; redundant if `expose-acl` is `1`. |`0`
+|`direct-io` |Enables support for the `O_DIRECT` flag. May degrade performance, but is useful if applications perform their own caching. |`0`
+|===
 
 ----
 qm set <vmid> -virtiofs0 dirid=<dirid>,cache=always,direct-io=1
-- 
2.47.3

[PATCH docs v3 7/11] virtiofs: add description for thread-pool-size and readonly

[Markus Frank]

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3 changes:
* added readonly to table

 qm.adoc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/qm.adoc b/qm.adoc
index c0ca466..f6d2712 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -1397,12 +1397,14 @@ All Optional Parameters:
 |`expose-acl` |Enables ACL passthrough; implies `expose-xattr`. Supported only on compatible filesystems. |`0`
 |`expose-xattr` |Enables xattr passthrough; redundant if `expose-acl` is `1`. |`0`
 |`direct-io` |Enables support for the `O_DIRECT` flag. May degrade performance, but is useful if applications perform their own caching. |`0`
+|`thread-pool-size` |Controls the maximum number of worker threads used by virtiofsd to handle requests. Adjust for performance tuning. `0` disables the thread pool. |`0`
+|`readonly` | Prevent the guest from writing. This will not make the mount read-only. For example, the access time will still be updated upon access. |`0`
 |===
 
 ----
 qm set <vmid> -virtiofs0 dirid=<dirid>,cache=always,direct-io=1
 qm set <vmid> -virtiofs1 <dirid>,cache=never,expose-xattr=1
-qm set <vmid> -virtiofs2 <dirid>,expose-acl=1
+qm set <vmid> -virtiofs2 <dirid>,expose-acl=1,thread-pool-size=16
 ----
 
 To temporarily mount virtiofs in a guest VM with the Linux kernel virtiofs
-- 
2.47.3

[PATCH docs v3 8/11] virtiofs: add documentation for live migration

[Markus Frank]

Also update the virtiofs known limitations.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3 changes:
* fixed capitalisation of virtiofsd

 qm.adoc | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/qm.adoc b/qm.adoc
index f6d2712..0e29b22 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -1352,10 +1352,12 @@ Known Limitations
 * Memory hotplug does not work in combination with virtiofs (also results in
   hanging access).
 * Memory related features such as live migration, snapshots, and hibernate are
-  not available with virtiofs devices.
+  not available with virtiofs devices on {pve} 8.
 * Windows cannot understand ACLs in the context of virtiofs. Therefore, do not
   expose ACLs for Windows VMs, otherwise the virtiofs device will not be visible
   within the VM.
+* For further limitations, see the
+  https://gitlab.com/virtio-fs/virtiofsd#limitations[GitLab page of virtiofsd].
 
 Add Mapping for Shared Directories
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -1369,6 +1371,29 @@ pvesh create /cluster/mapping/dir --id dir1 \
     --map node=node2,path=/path/to/share2 \
 ----
 
+Live Migration
+^^^^^^^^^^^^^^
+
+Live migration is possible on {pve} 9 (Debian Trixie) because versions of
+virtiofsd starting with v1.11.0 support live migration. To enable live
+migration, select a method using the `live-migration-method` parameter in the
+Directory Mapping (xref:resource_mapping[Resource Mapping]). If a live migration
+method is selected, virtiofsd will start with the flags to enable live migration
+using the chosen method. Ensure you are using the same shared directory on both
+hosts before migrating.
+
+Available migration methods are `file-handles` and `find-path`:
+
+* In the `find-paths` migration mode, virtiofsd tries to find the path inside of
+  the shared directory for every filesystem object that must be transferred to
+  the destination.
+* In the `file-handles` migration mode every filesystem object is converted to a
+  file handle.
+
+For more information about the live migration methods, see the
+https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/doc/migration.md[virtiofsd
+migration page on GitLab].
+
 Add virtiofs to a VM
 ^^^^^^^^^^^^^^^^^^^^
 
@@ -2220,6 +2245,10 @@ Currently there are the following options:
   migrating passed through devices is an experimental feature and may not work
   or cause issues.
 
+* `live-migration-method` (Directory): This sets the live migration mode for
+  virtiofs. If a live migration method is selected, virtiofsd will start with
+  the flags to enable live migration using the chosen method. Available
+  migration methods are `file-handles` and `find-path`.
 
 Managing Virtual Machines with `qm`
 ------------------------------------
-- 
2.47.3

[PATCH manager v3 09/11] fix #6370: ui: virtiofs edit: add support for thread-pool-size option

[Markus Frank]

Requires the qemu-server commit named "fix #6370: virtiofs: add support
for thread-pool-size option".

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3 changes:
* replaced 'thread-pool-size' with 'Thread Pool Size' in fieldLabel

 www/manager6/qemu/VirtiofsEdit.js | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/www/manager6/qemu/VirtiofsEdit.js b/www/manager6/qemu/VirtiofsEdit.js
index d51ca255..e9e55816 100644
--- a/www/manager6/qemu/VirtiofsEdit.js
+++ b/www/manager6/qemu/VirtiofsEdit.js
@@ -83,6 +83,16 @@ Ext.define('PVE.qemu.VirtiofsInputPanel', {
                 name: 'direct-io',
                 fieldLabel: gettext('Allow Direct IO'),
             },
+            {
+                xtype: 'proxmoxintegerfield',
+                name: 'thread-pool-size',
+                minValue: 0,
+                maxValue: 256,
+                value: '',
+                emptyText: '0',
+                allowBlank: true,
+                fieldLabel: gettext('Thread Pool Size'),
+            },
         ];
 
         me.virtiofs = {};
-- 
2.47.3

[PATCH manager v3 10/11] virtiofs edit: add support for readonly option

[Markus Frank]

Requires the qemu-server commit named "virtiofs: add readonly option".

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3: no changes

 www/manager6/qemu/VirtiofsEdit.js | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/www/manager6/qemu/VirtiofsEdit.js b/www/manager6/qemu/VirtiofsEdit.js
index e9e55816..1d6c6602 100644
--- a/www/manager6/qemu/VirtiofsEdit.js
+++ b/www/manager6/qemu/VirtiofsEdit.js
@@ -83,6 +83,11 @@ Ext.define('PVE.qemu.VirtiofsInputPanel', {
                 name: 'direct-io',
                 fieldLabel: gettext('Allow Direct IO'),
             },
+            {
+                xtype: 'proxmoxcheckbox',
+                name: 'readonly',
+                fieldLabel: gettext('Read Only'),
+            },
             {
                 xtype: 'proxmoxintegerfield',
                 name: 'thread-pool-size',
-- 
2.47.3

[PATCH manager v3 11/11] directory mapping: add live-migration-method option for virtiofs

[Markus Frank]

Display warnings indicating that this feature is experimental.

Add the Messages.js file to reuse warnings, hints, and other messages
in different locations. Display the virtiofs warnings from the
Messages.js file in the migration and directory edit windows.

Requires the qemu-server commit named "virtiofs: add live migration
support".

Signed-off-by: Markus Frank <m.frank@proxmox.com>
---
v3 changes:
* remove severity option from Messages.js
* renamed hideComment to hideClusterOptions
* moved all live-migration related fields to columnB
* added has_virtiofs_live_migration variable
* improved the warning messages

 www/manager6/Makefile             |  1 +
 www/manager6/Messages.js          | 15 ++++++++
 www/manager6/dc/DirMapView.js     |  8 ++++
 www/manager6/window/DirMapEdit.js | 63 +++++++++++++++++++++++++++++--
 www/manager6/window/Migrate.js    | 13 +++++++
 5 files changed, 97 insertions(+), 3 deletions(-)
 create mode 100644 www/manager6/Messages.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index b506849d..80ce4a7c 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -7,6 +7,7 @@ JSSRC= 							\
 	StateProvider.js				\
 	Utils.js					\
 	UIOptions.js					\
+	Messages.js					\
 	Toolkit.js					\
 	VNCConsole.js					\
 	button/ConsoleButton.js				\
diff --git a/www/manager6/Messages.js b/www/manager6/Messages.js
new file mode 100644
index 00000000..8e412a45
--- /dev/null
+++ b/www/manager6/Messages.js
@@ -0,0 +1,15 @@
+Ext.define('PVE.Messages', {
+    singleton: true,
+
+    virtiofs: {
+        prerequisites: gettext(
+            'Before migrating with a virtiofs device, ensure that virtiofsd is installed on all hosts and that they are using the same shared directory on the same file system (e.g. CephFS).',
+        ),
+        umount: gettext(
+            'The safest way to migrate a VM with a virtiofs device is to either unmount or mount the device as read-only.',
+        ),
+        migration_method_change: gettext(
+            'Changes to the migration method will only take effect once a VM has been stopped and restarted, or migrated. When changing this from disabled, please restart VMs to ensure that additional migration checks are enabled.',
+        ),
+    },
+});
diff --git a/www/manager6/dc/DirMapView.js b/www/manager6/dc/DirMapView.js
index f0dfe28f..26f938e9 100644
--- a/www/manager6/dc/DirMapView.js
+++ b/www/manager6/dc/DirMapView.js
@@ -26,6 +26,14 @@ Ext.define('PVE.dc.DirMapView', {
             dataIndex: 'text',
             width: 200,
         },
+        {
+            header: gettext('Live Migration Method'),
+            dataIndex: 'live-migration',
+            renderer: function (value, _meta, record) {
+                return Ext.String.htmlEncode(value ?? record.data['live-migration-method']);
+            },
+            width: 150,
+        },
         {
             header: gettext('Comment'),
             dataIndex: 'description',
diff --git a/www/manager6/window/DirMapEdit.js b/www/manager6/window/DirMapEdit.js
index 841ff1fe..ac34d1a4 100644
--- a/www/manager6/window/DirMapEdit.js
+++ b/www/manager6/window/DirMapEdit.js
@@ -8,7 +8,7 @@ Ext.define('PVE.window.DirMapEditWindow', {
         me.isCreate = !me.name;
         me.method = me.isCreate ? 'POST' : 'PUT';
         me.hideMapping = !!me.entryOnly;
-        me.hideComment = me.name && !me.entryOnly;
+        me.hideClusterOptions = me.name && !me.entryOnly;
         me.hideNodeSelector = me.nodename || me.entryOnly;
         me.hideNode = !me.nodename || !me.hideNodeSelector;
         return {
@@ -17,6 +17,15 @@ Ext.define('PVE.window.DirMapEditWindow', {
         };
     },
 
+    viewModel: {
+        data: {
+            migrationMethod: '__default__',
+        },
+        formulas: {
+            hideMigrationHints: (get) => get('migrationMethod') === '__default__',
+        },
+    },
+
     submitUrl: function (_url, data) {
         let me = this;
         let name = me.isCreate ? '' : me.name;
@@ -39,8 +48,10 @@ Ext.define('PVE.window.DirMapEditWindow', {
 
             let name = values.name;
             let description = values.description;
+            let liveMigration = values['live-migration-method'];
             let deletes = values.delete;
 
+            delete values['live-migration-method'];
             delete values.description;
             delete values.name;
             delete values.delete;
@@ -69,6 +80,9 @@ Ext.define('PVE.window.DirMapEditWindow', {
             if (description) {
                 values.description = description;
             }
+            if (liveMigration) {
+                values['live-migration-method'] = liveMigration;
+            }
             if (deletes && !view.isCreate) {
                 values.delete = deletes;
             }
@@ -184,13 +198,56 @@ Ext.define('PVE.window.DirMapEditWindow', {
             ],
 
             columnB: [
+                {
+                    xtype: 'proxmoxKVComboBox',
+                    fieldLabel: `<i class="fa fa-exclamation-triangle warning"></i> ${gettext('Experimental')}:
+                        ${gettext('Live Migration Method')}`,
+                    reference: 'live-migration-method',
+                    name: 'live-migration-method',
+                    comboItems: [
+                        [
+                            '__default__',
+                            Proxmox.Utils.defaultText + ' (' + Proxmox.Utils.disabledText + ')',
+                        ],
+                        ['find-paths', 'find-paths'],
+                        ['file-handles', 'file-handles'],
+                    ],
+                    cbind: {
+                        deleteEmpty: '{!isCreate}',
+                        disabled: '{hideClusterOptions}',
+                        hidden: '{hideClusterOptions}',
+                    },
+                    bind: {
+                        value: '{migrationMethod}',
+                    },
+                },
+                {
+                    xtype: 'displayfield',
+                    reference: 'migration-method-change-hint',
+                    columnWidth: 1,
+                    value: PVE.Messages.virtiofs.migration_method_change,
+                    bind: {
+                        hidden: '{hideMigrationHints}',
+                    },
+                    userCls: 'pmx-hint',
+                },
+                {
+                    xtype: 'displayfield',
+                    reference: 'migration-hint',
+                    columnWidth: 1,
+                    value: PVE.Messages.virtiofs.prerequisites + ' ' + PVE.Messages.virtiofs.umount,
+                    bind: {
+                        hidden: '{hideMigrationHints}',
+                    },
+                    userCls: 'pmx-hint',
+                },
                 {
                     xtype: 'fieldcontainer',
                     defaultType: 'radiofield',
                     layout: 'fit',
                     cbind: {
-                        disabled: '{hideComment}',
-                        hidden: '{hideComment}',
+                        disabled: '{hideClusterOptions}',
+                        hidden: '{hideClusterOptions}',
                     },
                     items: [
                         {
diff --git a/www/manager6/window/Migrate.js b/www/manager6/window/Migrate.js
index ff80c70c..1bc0315d 100644
--- a/www/manager6/window/Migrate.js
+++ b/www/manager6/window/Migrate.js
@@ -335,6 +335,19 @@ Ext.define('PVE.window.Migrate', {
             if (vm.get('running')) {
                 let allowed = [];
                 let notAllowed = [];
+                let has_virtiofs_live_migration = Object.entries(mappedResources).some(
+                    ([key, resource]) => key.match(/^virtiofs\d+/) && resource['live-migration'],
+                );
+                if (has_virtiofs_live_migration) {
+                    migration.preconditions.push({
+                        text: PVE.Messages.virtiofs.prerequisites,
+                        severity: 'warning',
+                    });
+                    migration.preconditions.push({
+                        text: PVE.Messages.virtiofs.umount,
+                        severity: 'warning',
+                    });
+                }
                 for (const [key, resource] of Object.entries(mappedResources)) {
                     if (resource['live-migration']) {
                         allowed.push(key);
-- 
2.47.3

Re: [PATCH guest-common/qemu-server/docs/manager v3 0/11] Virtiofs improvements

[Filip Schauer]

On 27/04/2026 14:16, Markus Frank wrote:
> It seems the migration-mode find-paths is enabled by default:
> https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/src/main.rs?ref_type=heads#L359
>
> I would still disable live migration by default on our side so that
> users have to actively choose the migration mode and learn about the
> limitations and risks.
>
> The VM and virtiofsd needs to be stopped and started again for the
> migration mode change to take effect.

Tested live migration while copying a 1GiB file within the virtiofs.
I tested with NFS, CephFS, and CIFS as the shared backing file system
for the virtiofs share. The guest was running Debian 13.

These were the steps I performed:
1. Start copying inside the VM
    `/mnt/virtiofs# dd if=testA of=testB status=progress`
2. Live migrate the VM to another host
3. Check if copying continues normally
4. Once copy completed, check if the two files are identical and if
    reading and writing to the share still works


Overview of the results:

backing fs  live mig. method  data integrity  write works?  read works?
NFS         find-paths        ok              yes           yes
NFS         file-handles      data mismatch   yes           yes
CephFS      find-paths        ok              yes           yes
CephFS      file-handles      ok              yes           yes
CIFS        find-paths        copy failed     I/O error     I/O error
CIFS        file-handles      copy failed     I/O error     yes


Live Migration with CephFS backing virtiofs worked flawlessly.

NFS worked well with find-paths
but with file-handles it corrupted (zeroed out) 1536 Bytes of data.

Live Migration while writing to a virtiofs share backed by CIFS lead to
an I/O error, leaving the share in a broken state.

Logs on source node during live migration with find-paths on CIFS:
Apr 27 16:58:00 pve2 pvedaemon[1185]: <root@pam> starting task 
UPID:pve2:00001653:0001BB68:69EF7978:qmigrate:100:root@pam:
Apr 27 16:58:00 pve2 systemd[1]: Starting pve-dbus-vmstate@100.service - 
PVE DBus VMState Helper (VM 100)...
Apr 27 16:58:00 pve2 dbus-vmstate[5717]: pve-vmstate-100 listening on :1.21
Apr 27 16:58:00 pve2 systemd[1]: Started pve-dbus-vmstate@100.service - 
PVE DBus VMState Helper (VM 100).
Apr 27 16:58:01 pve2 pmxcfs[834]: [status] notice: received log
Apr 27 16:58:03 pve2 pmxcfs[834]: [status] notice: received log
Apr 27 16:58:27 pve2 virtiofsd[5199]: pve2 virtiofsd[5196]: Inode 2: 
Operation not supported (os error 95)
Apr 27 16:58:27 pve2 virtiofsd[5199]: pve2 virtiofsd[5196]: Inode 3: 
Operation not supported (os error 95)
Apr 27 16:58:32 pve2 virtiofsd[5199]: pve2 virtiofsd[5196]: Inode 2: 
Operation not supported (os error 95)
Apr 27 16:58:32 pve2 virtiofsd[5199]: pve2 virtiofsd[5196]: Inode 3: 
Operation not supported (os error 95)
Apr 27 16:58:32 pve2 virtiofsd[5199]: pve2 virtiofsd[5196]: Failed to 
serialize inode 1 (st_dev=51, mnt_id=330, st_ino=131663): Failed to 
reconstruct inode location; marking as invalid
Apr 27 16:58:32 pve2 virtiofsd[5199]: pve2 virtiofsd[5196]: Failed to 
serialize inode 2 (st_dev=51, mnt_id=330, st_ino=131665): Failed to 
reconstruct inode location; marking as invalid
Apr 27 16:58:32 pve2 virtiofsd[5199]: pve2 virtiofsd[5196]: Failed to 
serialize inode 3 (st_dev=51, mnt_id=330, st_ino=130930): Failed to 
reconstruct inode location; marking as invalid
Apr 27 16:58:32 pve2 dbus-vmstate[5717]: received 0 conntrack entries
Apr 27 16:58:32 pve2 dbus-vmstate[5717]: transferring 0 bytes of 
conntrack state
Apr 27 16:58:36 pve2 dbus-vmstate[5717]: shutting down gracefully ..
Apr 27 16:58:36 pve2 systemd[1]: pve-dbus-vmstate@100.service: 
Deactivated successfully.

Logs on target node during live migration with find-paths on CIFS:
Apr 27 16:58:01 pve3 qm[5049]: start VM 100: 
UPID:pve3:000013B9:00017FBA:69EF7979:qmstart:100:root@pam:
Apr 27 16:58:02 pve3 systemd[1]: Created slice qemu.slice - Slice /qemu.
Apr 27 16:58:02 pve3 systemd[1]: Started 100.scope.
Apr 27 16:58:02 pve3 virtiofsd[5079]: pve3 virtiofsd[5076]: Waiting for 
vhost-user socket connection...
Apr 27 16:58:02 pve3 virtiofsd[5079]: pve3 virtiofsd[5076]: Client 
connected, servicing requests
Apr 27 16:58:03 pve3 qm[5049]: VM 100 started with PID 5084.
Apr 27 16:58:03 pve3 systemd[1]: Starting pve-dbus-vmstate@100.service - 
PVE DBus VMState Helper (VM 100)...
Apr 27 16:58:03 pve3 dbus-vmstate[5148]: pve-vmstate-100 listening on :1.22
Apr 27 16:58:03 pve3 systemd[1]: Started pve-dbus-vmstate@100.service - 
PVE DBus VMState Helper (VM 100).
Apr 27 16:58:03 pve3 qm[5048]: <root@pam> end task 
UPID:pve3:000013B9:00017FBA:69EF7979:qmstart:100:root@pam: OK
Apr 27 16:58:32 pve3 virtiofsd[5079]: pve3 virtiofsd[5076]: Invalid 
inode 1 indexed: Migration source has lost inode 1
Apr 27 16:58:32 pve3 virtiofsd[5079]: pve3 virtiofsd[5076]: Invalid 
inode 3 indexed: Migration source has lost inode 3
Apr 27 16:58:32 pve3 virtiofsd[5079]: pve3 virtiofsd[5076]: Invalid 
inode 2 indexed: Migration source has lost inode 2
Apr 27 16:58:32 pve3 virtiofsd[5079]: pve3 virtiofsd[5076]: Invalid 
handle 1 is open in guest: Opening inode 2 as handle 1: Inode is invalid 
because of an error during the preceding migration, which was: Migration 
source has lost inode 2
Apr 27 16:58:32 pve3 virtiofsd[5079]: pve3 virtiofsd[5076]: Invalid 
handle 2 is open in guest: Opening inode 3 as handle 2: Inode is invalid 
because of an error during the preceding migration, which was: Migration 
source has lost inode 3


Logs on source node during live migration with file-handles on CIFS:
Apr 27 17:11:03 pve3 pvedaemon[1198]: <root@pam> starting task 
UPID:pve3:00001E6D:0002B0D4:69EF7C87:qmigrate:100:root@pam:
Apr 27 17:11:03 pve3 systemd[1]: Starting pve-dbus-vmstate@100.service - 
PVE DBus VMState Helper (VM 100)...
Apr 27 17:11:03 pve3 dbus-vmstate[7791]: pve-vmstate-100 listening on :1.56
Apr 27 17:11:03 pve3 systemd[1]: Started pve-dbus-vmstate@100.service - 
PVE DBus VMState Helper (VM 100).
Apr 27 17:11:04 pve3 pmxcfs[830]: [status] notice: received log
Apr 27 17:11:06 pve3 pmxcfs[830]: [status] notice: received log
Apr 27 17:11:29 pve3 virtiofsd[7469]: pve3 virtiofsd[7467]: Inode 2 
(/fileA): Failed to generate file handle: Operation not supported (os 
error 95)
Apr 27 17:11:29 pve3 virtiofsd[7469]: pve3 virtiofsd[7467]: Inode 3 
(/fileB): Failed to generate file handle: Operation not supported (os 
error 95)
Apr 27 17:11:34 pve3 virtiofsd[7469]: pve3 virtiofsd[7467]: Failed to 
serialize inode 2 (st_dev=51, mnt_id=338, st_ino=131665): Failed to 
reconstruct inode location; marking as invalid
Apr 27 17:11:34 pve3 virtiofsd[7469]: pve3 virtiofsd[7467]: Failed to 
serialize inode 3 (st_dev=51, mnt_id=338, st_ino=130930): Failed to 
reconstruct inode location; marking as invalid
Apr 27 17:11:34 pve3 dbus-vmstate[7791]: received 0 conntrack entries
Apr 27 17:11:34 pve3 dbus-vmstate[7791]: transferring 0 bytes of 
conntrack state
Apr 27 17:11:37 pve3 dbus-vmstate[7791]: shutting down gracefully ..
Apr 27 17:11:37 pve3 systemd[1]: pve-dbus-vmstate@100.service: 
Deactivated successfully.
Apr 27 17:11:40 pve3 qmeventd[711]: read: Connection reset by peer
Apr 27 17:11:40 pve3 virtiofsd[7469]: pve3 virtiofsd[7467]: Client 
disconnected, shutting down

Logs on target node during live migration with file-handles on CIFS:
Apr 27 17:11:04 pve2 qm[8116]: start VM 100: 
UPID:pve2:00001FB4:0002EDB2:69EF7C88:qmstart:100:root@pam:
Apr 27 17:11:05 pve2 systemd[1]: Started 100.scope.
Apr 27 17:11:05 pve2 virtiofsd[8146]: pve2 virtiofsd[8143]: Waiting for 
vhost-user socket connection...
Apr 27 17:11:05 pve2 virtiofsd[8146]: pve2 virtiofsd[8143]: Client 
connected, servicing requests
Apr 27 17:11:05 pve2 qm[8116]: VM 100 started with PID 8151.
Apr 27 17:11:05 pve2 systemd[1]: Starting pve-dbus-vmstate@100.service - 
PVE DBus VMState Helper (VM 100)...
Apr 27 17:11:06 pve2 dbus-vmstate[8208]: pve-vmstate-100 listening on :1.30
Apr 27 17:11:06 pve2 systemd[1]: Started pve-dbus-vmstate@100.service - 
PVE DBus VMState Helper (VM 100).
Apr 27 17:11:06 pve2 qm[8115]: <root@pam> end task 
UPID:pve2:00001FB4:0002EDB2:69EF7C88:qmstart:100:root@pam: OK
Apr 27 17:11:34 pve2 virtiofsd[8146]: pve2 virtiofsd[8143]: Invalid 
inode 3 indexed: Migration source has lost inode 3
Apr 27 17:11:34 pve2 virtiofsd[8146]: pve2 virtiofsd[8143]: Invalid 
inode 2 indexed: Migration source has lost inode 2
Apr 27 17:11:34 pve2 virtiofsd[8146]: pve2 virtiofsd[8143]: Invalid 
handle 1 is open in guest: Opening inode 2 as handle 1: Inode is invalid 
because of an error during the preceding migration, which was: Migration 
source has lost inode 2
Apr 27 17:11:34 pve2 virtiofsd[8146]: pve2 virtiofsd[8143]: Invalid 
handle 2 is open in guest: Opening inode 3 as handle 2: Inode is invalid 
because of an error during the preceding migration, which was: Migration 
source has lost inode 3