[pve-devel] [RFC qemu-server] api: create disks: avoid adding secondary cloud-init drives

[pve-devel] [RFC qemu-server] api: create disks: avoid adding secondary cloud-init drives

[Fabian Ebner]

This will break possibly existing workflows like
1. add second cloud-init
2. remove first cloud-init
to change the cloud-init storage.

On the other hand, it avoids unintended misconfiguration of having
mutliple cloud-init drives with potentially different settings.

Also in preparation for adding cloud-init-related API calls, where
not being able to assume that there's only one cloud-init drive/state
would complicate things quite a bit.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---

Are there any scenarios where having multiple cloud-init drives is
useful?

 PVE/API2/Qemu.pm  | 19 +++++++++++++++++++
 PVE/QemuConfig.pm | 15 +++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index a8246574..74f0181c 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -328,6 +328,15 @@ my $create_disks = sub {
 	} elsif (defined($volname) && $volname eq 'cloudinit') {
 	    $storeid = $storeid // $default_storage;
 	    die "no storage ID specified (and no default storage)\n" if !$storeid;
+
+	    if (
+		my $ci_key = PVE::QemuConfig->has_cloudinit($conf, $ds)
+		|| PVE::QemuConfig->has_cloudinit($conf->{pending} || {}, $ds)
+		|| PVE::QemuConfig->has_cloudinit($res, $ds)
+	    ) {
+		die "$ds - cloud-init drive is already attached at '$ci_key'\n";
+	    }
+
 	    my $scfg = PVE::Storage::storage_config($storecfg, $storeid);
 	    my $name = "vm-$vmid-cloudinit";
 
@@ -424,6 +433,16 @@ my $create_disks = sub {
 		my ($vtype) = PVE::Storage::parse_volname($storecfg, $volid);
 		die "cannot use volume $volid - content type needs to be 'images' or 'iso'"
 		    if $vtype ne 'images' && $vtype ne 'iso';
+
+		if (PVE::QemuServer::Drive::drive_is_cloudinit($disk)) {
+		    if (
+			my $ci_key = PVE::QemuConfig->has_cloudinit($conf, $ds)
+			|| PVE::QemuConfig->has_cloudinit($conf->{pending} || {}, $ds)
+			|| PVE::QemuConfig->has_cloudinit($res, $ds)
+		    ) {
+			die "$ds - cloud-init drive is already attached at '$ci_key'\n";
+		    }
+		}
 	    }
 
 	    PVE::Storage::activate_volumes($storecfg, [ $volid ]) if $storeid;
diff --git a/PVE/QemuConfig.pm b/PVE/QemuConfig.pm
index cfef8d37..f7557613 100644
--- a/PVE/QemuConfig.pm
+++ b/PVE/QemuConfig.pm
@@ -518,4 +518,19 @@ sub __snapshot_rollback_get_unused {
 
 # END implemented abstract methods from PVE::AbstractConfig
 
+sub has_cloudinit {
+    my ($class, $conf, $skip) = @_;
+
+    my $found;
+
+    $class->foreach_volume($conf, sub {
+	my ($key, $volume) = @_;
+
+	return if ($skip && $skip eq $key) || $found;
+	$found = $key if PVE::QemuServer::Drive::drive_is_cloudinit($volume);
+    });
+
+    return $found;
+}
+
 1;
-- 
2.30.2

Re: [pve-devel] [RFC qemu-server] api: create disks: avoid adding secondary cloud-init drives

[DERUMIER, Alexandre]

> ---
> 
> Are there any scenarios where having multiple cloud-init drives is
> useful?

I don't remember exactly how cloud-init daemon is mounting drives,
but I'm pretty sure that with multiple cloud-init drives, 
only 1 will be mounted and read. 

So,I'm 100% ok with limiting cloud-init to only 1 drive.

Re: [pve-devel] [RFC qemu-server] api: create disks: avoid adding secondary cloud-init drives

[Mira Limbeck]

On 5/16/22 10:32, DERUMIER, Alexandre wrote:
>> ---
>>
>> Are there any scenarios where having multiple cloud-init drives is
>> useful?
> I don't remember exactly how cloud-init daemon is mounting drives,
> but I'm pretty sure that with multiple cloud-init drives,
> only 1 will be mounted and read.
>
> So,I'm 100% ok with limiting cloud-init to only 1 drive.
>
Yes, only one will be mounted.

We had a support case with exactly that issue, one cloud-init disk 
attached to the VM, and host CD-Rom passthrough with another one.

This lead to strange behavior because the wrong one was used by cloud-init.

Re: [pve-devel] [RFC qemu-server] api: create disks: avoid adding secondary cloud-init drives

[DERUMIER, Alexandre]

Le vendredi 06 mai 2022 à 12:11 +0200, Fabian Ebner a écrit :
> This will break possibly existing workflows like
> 1. add second cloud-init
> 2. remove first cloud-init
> to change the cloud-init storage.

Also, currently, in the gui, we can't add 2 cloud-init drives
currently.

But we could add an "move disk" like feature.
generate current cloudinit config on a new drive and swap isos online.
(If iso is mounted in the guest os, it's not a problem, as it's
readonly, and the mountpoint content is updated too)

 

Re: [pve-devel] [RFC qemu-server] api: create disks: avoid adding secondary cloud-init drives

[Mira Limbeck]

On 5/16/22 13:04, DERUMIER, Alexandre wrote:
> Le vendredi 06 mai 2022 à 12:11 +0200, Fabian Ebner a écrit :
>> This will break possibly existing workflows like
>> 1. add second cloud-init
>> 2. remove first cloud-init
>> to change the cloud-init storage.
> Also, currently, in the gui, we can't add 2 cloud-init drives
> currently.
>
> But we could add an "move disk" like feature.
> generate current cloudinit config on a new drive and swap isos online.
> (If iso is mounted in the guest os, it's not a problem, as it's
> readonly, and the mountpoint content is updated too)
>
`Move Disk` while the VM is running, should not recreate the disk from 
config as this could lead to a changed cloud-init disk.

Offline should be fine, but online `Move Disk` must re-use the current 
cloud-init disk.

We had the same issue in the live migration case. We now offline migrate 
the cloud-init disk and pass an argument during start on the target node 
to skip generating the cloud-init disk as a workaround.

Live migration the same way all other disks are migrated doesn't work 
here because the cloud-init disk is readonly mounted in QEMU.

Re: [pve-devel] [RFC qemu-server] api: create disks: avoid adding secondary cloud-init drives

[Fabian Ebner]

Am 06.05.22 um 12:11 schrieb Fabian Ebner:
> This will break possibly existing workflows like
> 1. add second cloud-init
> 2. remove first cloud-init
> to change the cloud-init storage.
> 
> On the other hand, it avoids unintended misconfiguration of having
> mutliple cloud-init drives with potentially different settings.
> 
> Also in preparation for adding cloud-init-related API calls, where
> not being able to assume that there's only one cloud-init drive/state
> would complicate things quite a bit.
> 
> Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>

Ping

[pve-devel] applied: [RFC qemu-server] api: create disks: avoid adding secondary cloud-init drives

[Thomas Lamprecht]

Am 06/05/2022 um 12:11 schrieb Fabian Ebner:
> This will break possibly existing workflows like
> 1. add second cloud-init
> 2. remove first cloud-init
> to change the cloud-init storage.
> 
> On the other hand, it avoids unintended misconfiguration of having
> mutliple cloud-init drives with potentially different settings.
> 
> Also in preparation for adding cloud-init-related API calls, where
> not being able to assume that there's only one cloud-init drive/state
> would complicate things quite a bit.
> 
> Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
> ---
> 
> Are there any scenarios where having multiple cloud-init drives is
> useful?
> 
>  PVE/API2/Qemu.pm  | 19 +++++++++++++++++++
>  PVE/QemuConfig.pm | 15 +++++++++++++++
>  2 files changed, 34 insertions(+)
> 
>

applied, thanks!