public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config
@ 2024-07-09 15:12 Mira Limbeck
  2024-07-09 15:12 ` [pve-devel] [PATCH docs] cloudinit: add Windows cloudbase-init section Mira Limbeck
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Mira Limbeck @ 2024-07-09 15:12 UTC (permalink / raw)
  To: pve-devel

cloudbase-init, a cloud-init reimplementation for Windows, supports only
a subset of the configuration options of cloud-init. Some features
depend on support by the Metadata Service (ConfigDrive2 here) and have
further limitations [0].

To support a basic setup the following changes were made:
 - password is saved as plaintext for any Windows guests (ostype)
 - DNS servers are added to each of the interfaces
 - SSH public keys are passed via metadata

Network and metadata generation for cloudbase-init is separate from the
default ConfigDrive2 one so as to not interfere with any other OSes that
depend on the current ConfigDrive2 implementation.

[0] https://cloudbase-init.readthedocs.io/en/latest/index.html

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
---
DNS search domains are not handled at all by the cloudbase-init ENI
parser.
The password is used for the Admin user specified in the
cloudbase-init.conf inside the guest. Specifying a different user does
not work. This would require rewriting the userdata handling as
described in #5384 [1] which is a breaking change. Userdata generation
is currently shared between all implementations, but the new one could
be made cloudbase-init only for now.

To know if the password needs to be unencrypted, we have to check the
`ostype`. For this we need access to the config. That's why I moved the
`cipassword` handling inside $updatefn.

When no `citype` is specified, it will default to `configdrive2` on
Windows. The check requires `ostype` to be set correctly.
Any other `citype`s may not work correctly if used for cloudbase-init.

The docs patch adds a section on how to configure a Windows guest for
cloudbase-init.


[1] https://bugzilla.proxmox.com/show_bug.cgi?id=5384

 PVE/API2/Qemu.pm            |  13 ++---
 PVE/QemuServer/Cloudinit.pm | 100 ++++++++++++++++++++++++++++++++++--
 2 files changed, 102 insertions(+), 11 deletions(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 7414385..b75c695 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -1667,12 +1667,6 @@ my $update_vm_api  = sub {
 
     my $skip_cloud_init = extract_param($param, 'skip_cloud_init');
 
-    if (defined(my $cipassword = $param->{cipassword})) {
-	# Same logic as in cloud-init (but with the regex fixed...)
-	$param->{cipassword} = PVE::Tools::encrypt_pw($cipassword)
-	    if $cipassword !~ /^\$(?:[156]|2[ay])(\$.+){2}/;
-    }
-
     my @paramarr = (); # used for log message
     foreach my $key (sort keys %$param) {
 	my $value = $key eq 'cipassword' ? '<hidden>' : $param->{$key};
@@ -2022,6 +2016,13 @@ my $update_vm_api  = sub {
 		    my $machine_conf = PVE::QemuServer::Machine::parse_machine($param->{$opt});
 		    PVE::QemuServer::Machine::assert_valid_machine_property($conf, $machine_conf);
 		    $conf->{pending}->{$opt} = $param->{$opt};
+		} elsif ($opt eq 'cipassword') {
+		    if (!PVE::QemuServer::Helpers::windows_version($conf->{ostype})) {
+			# Same logic as in cloud-init (but with the regex fixed...)
+			$param->{cipassword} = PVE::Tools::encrypt_pw($param->{cipassword})
+			    if $param->{cipassword} !~ /^\$(?:[156]|2[ay])(\$.+){2}/;
+		    }
+		    $conf->{cipassword} = $param->{cipassword};
 		} else {
 		    $conf->{pending}->{$opt} = $param->{$opt};
 
diff --git a/PVE/QemuServer/Cloudinit.pm b/PVE/QemuServer/Cloudinit.pm
index abc6b14..d4ecfac 100644
--- a/PVE/QemuServer/Cloudinit.pm
+++ b/PVE/QemuServer/Cloudinit.pm
@@ -8,6 +8,8 @@ use Digest::SHA;
 use URI::Escape;
 use MIME::Base64 qw(encode_base64);
 use Storable qw(dclone);
+use JSON;
+use URI;
 
 use PVE::Tools qw(run_command file_set_contents);
 use PVE::Storage;
@@ -232,12 +234,23 @@ sub generate_configdrive2 {
     my ($conf, $vmid, $drive, $volname, $storeid) = @_;
 
     my ($user_data, $network_data, $meta_data, $vendor_data) = get_custom_cloudinit_files($conf);
-    $user_data = cloudinit_userdata($conf, $vmid) if !defined($user_data);
-    $network_data = configdrive2_network($conf) if !defined($network_data);
-    $vendor_data = '' if !defined($vendor_data);
+    if (PVE::QemuServer::Helpers::windows_version($conf->{ostype})) {
+	$user_data = cloudinit_userdata($conf, $vmid) if !defined($user_data);
+	$network_data = cloudbase_network_eni($conf) if !defined($network_data);
+	$vendor_data = '' if !defined($vendor_data);
+
+	if (!defined($meta_data)) {
+	    my $instance_id = cloudbase_gen_instance_id($user_data, $network_data);
+	    $meta_data = cloudbase_configdrive2_metadata($instance_id, $conf);
+	}
+    } else {
+	$user_data = cloudinit_userdata($conf, $vmid) if !defined($user_data);
+	$network_data = configdrive2_network($conf) if !defined($network_data);
+	$vendor_data = '' if !defined($vendor_data);
 
-    if (!defined($meta_data)) {
-	$meta_data = configdrive2_gen_metadata($user_data, $network_data);
+	if (!defined($meta_data)) {
+	    $meta_data = configdrive2_gen_metadata($user_data, $network_data);
+	}
     }
 
     # we always allocate a 4MiB disk for cloudinit and with the overhead of the ISO
@@ -254,6 +267,83 @@ sub generate_configdrive2 {
     commit_cloudinit_disk($conf, $vmid, $drive, $volname, $storeid, $files, 'config-2');
 }
 
+sub cloudbase_network_eni {
+    my ($conf) = @_;
+
+    my $content = "";
+
+    my ($searchdomains, $nameservers) = get_dns_conf($conf);
+    if ($nameservers && @$nameservers) {
+	$nameservers = join(' ', @$nameservers);
+    }
+
+    my @ifaces = grep { /^net(\d+)$/ } keys %$conf;
+    foreach my $iface (sort @ifaces) {
+	(my $id = $iface) =~ s/^net//;
+	next if !$conf->{"ipconfig$id"};
+	my $net = PVE::QemuServer::parse_ipconfig($conf->{"ipconfig$id"});
+	$id = "eth$id";
+
+	$content .="auto $id\n";
+	if ($net->{ip}) {
+	    if ($net->{ip} eq 'dhcp') {
+		$content .= "iface $id inet dhcp\n";
+	    } else {
+		my ($addr, $mask) = split_ip4($net->{ip});
+		$content .= "iface $id inet static\n";
+		$content .= "        address $addr\n";
+		$content .= "        netmask $mask\n";
+		$content .= "        gateway $net->{gw}\n" if $net->{gw};
+		$content .= "        dns-nameservers $nameservers\n" if $nameservers;
+	    }
+	}
+	if ($net->{ip6}) {
+	    if ($net->{ip6} =~ /^(auto|dhcp)$/) {
+		$content .= "iface $id inet6 $1\n";
+	    } else {
+		my ($addr, $mask) = split('/', $net->{ip6});
+		$content .= "iface $id inet6 static\n";
+		$content .= "        address $addr\n";
+		$content .= "        netmask $mask\n";
+		$content .= "        gateway $net->{gw6}\n" if $net->{gw6};
+		$content .= "        dns-nameservers $nameservers\n" if $nameservers;
+	    }
+	}
+    }
+
+    return $content;
+}
+
+sub cloudbase_configdrive2_metadata {
+    my ($uuid, $conf) = @_;
+    my $meta_data = {
+	uuid => $uuid,
+	'network_config' => {
+	    'content_path' => '/content/0000',
+	},
+    };
+    $meta_data->{'admin_pass'} = $conf->{cipassword} if $conf->{cipassword};
+    if (defined(my $keys = $conf->{sshkeys})) {
+	$keys = URI::Escape::uri_unescape($keys);
+	$keys = [map { my $key = $_; chomp $key; $key } split(/\n/, $keys)];
+	$keys = [grep { /\S/ } @$keys];
+	my $i = 0;
+	foreach my $k (@$keys) {
+	    $meta_data->{'public_keys'}->{"key-$i"} = $k;
+	    $i++;
+	}
+    }
+    my $json = encode_json($meta_data);
+    return $json;
+}
+
+sub cloudbase_gen_instance_id {
+    my ($user, $network) = @_;
+
+    my $uuid_str = Digest::SHA::sha1_hex($user.$network);
+    return $uuid_str;
+}
+
 sub generate_opennebula {
     my ($conf, $vmid, $drive, $volname, $storeid) = @_;
 
-- 
2.39.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [pve-devel] [PATCH docs] cloudinit: add Windows cloudbase-init section
  2024-07-09 15:12 [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config Mira Limbeck
@ 2024-07-09 15:12 ` Mira Limbeck
  2024-07-10 14:35 ` [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config Mira Limbeck
  2024-07-18 15:51 ` Friedrich Weber
  2 siblings, 0 replies; 5+ messages in thread
From: Mira Limbeck @ 2024-07-09 15:12 UTC (permalink / raw)
  To: pve-devel

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
---
 qm-cloud-init.adoc | 79 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)

diff --git a/qm-cloud-init.adoc b/qm-cloud-init.adoc
index 8686ed7..4baab31 100644
--- a/qm-cloud-init.adoc
+++ b/qm-cloud-init.adoc
@@ -169,6 +169,85 @@ qm cloudinit dump 9000 user
 The same command exists for `network` and `meta`.
 
 
+Cloud-Init on Windows
+~~~~~~~~~~~~~~~~~~~~~
+
+There is a reimplementation of Cloud-Init available for Windows called
+https://cloudbase.it/[cloudbase-init]. Not every feature of Cloud-Init is
+available with cloudbase-init, and some features differ compared to Cloud-Init.
+
+cloudbase-init requires both `ostype` set to any Windows version and the
+`citype` set to `configdrive2`, which is the default with any Windows
+`ostype`.
+
+There are no ready-made cloud images for Windows available for free. Using
+cloudbase-init requires manually installing and configuring a Windows guest.
+
+
+Preparing cloudbase-init Templates
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The first step is to install Windows in a VM. Download and install
+cloudbase-init in the guest. It may be necessary to install the Beta version.
+Don't sysprep at the end of the installation. Instead configure cloudbase-init
+first.
+
+A few common options to set would be:
+
+* 'username': This sets the username of the administrator
+
+* 'groups': This allows one to add the user to the `Administrators` group
+
+* 'inject_user_password': Set this to `true` to allow setting the password
+in the VM config
+
+* 'first_logon_behaviour': Set this to `no` to not require a new password on
+login
+
+* 'rename_admin_user': Set this to `true` to allow renaming the default
+`Administrator` user to the username specified with `username`
+
+Some plugins, for example the SetHostnamePlugin, require reboots and will do
+so automatically. To disable automatic reboots by cloudbase-init, you can set
+`allow_reboot` to `false`.
+
+A full set of configuration options can be found in the
+https://cloudbase-init.readthedocs.io/en/latest/config.html[official
+cloudbase-init documentation]
+
+It can make sense to make a snapshot after configuring in case some parts of
+the config still need adjustments.
+After configuring cloudbase-init you can start creating the template. Shutdown
+the Windows guest, add a Cloud-Init disk and make it into a template.
+
+----
+qm set 9000 --ide2 local-lvm:cloudinit
+qm template 9000
+----
+
+Clone the template into a new VM:
+
+----
+qm clone 9000 123 --name windows123
+----
+
+Then set the password, network config and SSH key:
+
+----
+qm set 123 --cipassword <password>
+qm set 123 --ipconfig0 ip=10.0.10.123/24,gw=10.0.10.1
+qm set 123 --sshkey ~/.ssh/id_rsa.pub
+----
+
+Make sure that the `ostype` is set to any Windows version before setting the
+password. Otherwise the password will be encrypted and cloudbase-init will use
+the encrypted password as plaintext password.
+
+When everything is set, start the cloned guest. On the first boot the login
+won't work and it will reboot automatically for the changed hostname.
+After the reboot the new password should be set and login should work.
+
+
 Cloud-Init specific Options
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-- 
2.39.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config
  2024-07-09 15:12 [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config Mira Limbeck
  2024-07-09 15:12 ` [pve-devel] [PATCH docs] cloudinit: add Windows cloudbase-init section Mira Limbeck
@ 2024-07-10 14:35 ` Mira Limbeck
  2024-07-18 15:51 ` Friedrich Weber
  2 siblings, 0 replies; 5+ messages in thread
From: Mira Limbeck @ 2024-07-10 14:35 UTC (permalink / raw)
  To: pve-devel

There seems to be an issue with the network adapter. Whenever the guest
is shutdown and started again it finds a new network adapter. Rebooting
instead of shutting down doesn't show the same behavior.

I'm not sure yet why this happens, but it seems to be caused by
cloudbase-init.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config
  2024-07-09 15:12 [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config Mira Limbeck
  2024-07-09 15:12 ` [pve-devel] [PATCH docs] cloudinit: add Windows cloudbase-init section Mira Limbeck
  2024-07-10 14:35 ` [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config Mira Limbeck
@ 2024-07-18 15:51 ` Friedrich Weber
  2024-07-22 10:01   ` Mira Limbeck
  2 siblings, 1 reply; 5+ messages in thread
From: Friedrich Weber @ 2024-07-18 15:51 UTC (permalink / raw)
  To: Proxmox VE development discussion, Mira Limbeck

On 09/07/2024 17:12, Mira Limbeck wrote:
> cloudbase-init, a cloud-init reimplementation for Windows, supports only
> a subset of the configuration options of cloud-init. Some features
> depend on support by the Metadata Service (ConfigDrive2 here) and have
> further limitations [0].
> 
> To support a basic setup the following changes were made:
>  - password is saved as plaintext for any Windows guests (ostype)
>  - DNS servers are added to each of the interfaces
>  - SSH public keys are passed via metadata
> 
> Network and metadata generation for cloudbase-init is separate from the
> default ConfigDrive2 one so as to not interfere with any other OSes that
> depend on the current ConfigDrive2 implementation.

I tested the following:

- Install cloudbase-init beta in a Windows 2022 Server VM
- Shutdown VM
- Attach cloudinit drive, set network config
- Start VM
- After some time, Windows renames itself to the VM name and reboots
- Network config gets applied after some time (see below)

One thing I noticed: Without modifying the the standard in-guest
cloudbase-init configuration, it takes ~2min until cloudbase-init does
anything (in particular apply the network config). Apparently
cloudbase-init tries to find an HTTP server with the cloudinit data
first, and only looks into the configdrive after a timeout.

To avoid that, Mira suggested to change `metadata_services` [1] in the
cloudbase-init config to
`cloudbaseinit.metadata.services.configdrive.ConfigDriveService`.
Indeed, with this setting the network config gets applied immediately on
boot. It may be nice to add this to the documentation.

I'm not sure if the default behavior of changing the hostname to match
the VM name is something that Windows admins expect? Should the docs
mention more prominently how to disable this?

> There seems to be an issue with the network adapter. Whenever the guest
> is shutdown and started again it finds a new network adapter. Rebooting
> instead of shutting down doesn't show the same behavior.
> 
> I'm not sure yet why this happens, but it seems to be caused by
> cloudbase-init.

I couldn't reproduce this issue so far (using a virtio NIC).


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config
  2024-07-18 15:51 ` Friedrich Weber
@ 2024-07-22 10:01   ` Mira Limbeck
  0 siblings, 0 replies; 5+ messages in thread
From: Mira Limbeck @ 2024-07-22 10:01 UTC (permalink / raw)
  To: Friedrich Weber, Proxmox VE development discussion

Thank you for testing it!

On 7/18/24 17:51, Friedrich Weber wrote:
> On 09/07/2024 17:12, Mira Limbeck wrote:
>> cloudbase-init, a cloud-init reimplementation for Windows, supports only
>> a subset of the configuration options of cloud-init. Some features
>> depend on support by the Metadata Service (ConfigDrive2 here) and have
>> further limitations [0].
>>
>> To support a basic setup the following changes were made:
>>  - password is saved as plaintext for any Windows guests (ostype)
>>  - DNS servers are added to each of the interfaces
>>  - SSH public keys are passed via metadata
>>
>> Network and metadata generation for cloudbase-init is separate from the
>> default ConfigDrive2 one so as to not interfere with any other OSes that
>> depend on the current ConfigDrive2 implementation.
> 
> I tested the following:
> 
> - Install cloudbase-init beta in a Windows 2022 Server VM
> - Shutdown VM
> - Attach cloudinit drive, set network config
> - Start VM
> - After some time, Windows renames itself to the VM name and reboots
> - Network config gets applied after some time (see below)
> 
> One thing I noticed: Without modifying the the standard in-guest
> cloudbase-init configuration, it takes ~2min until cloudbase-init does
> anything (in particular apply the network config). Apparently
> cloudbase-init tries to find an HTTP server with the cloudinit data
> first, and only looks into the configdrive after a timeout.
> 
> To avoid that, Mira suggested to change `metadata_services` [1] in the
> cloudbase-init config to
> `cloudbaseinit.metadata.services.configdrive.ConfigDriveService`.
> Indeed, with this setting the network config gets applied immediately on
> boot. It may be nice to add this to the documentation.
>
I'll add the `metadata_services` config option to the docs as well in a v2.

> I'm not sure if the default behavior of changing the hostname to match
> the VM name is something that Windows admins expect? Should the docs
> mention more prominently how to disable this?

I'd say it makes sense to use the VM name as hostname. This keeps the
names in sync and makes it easy to match one to the other.
Would you like to make it configurable via VM config instead? Or what
would you expect?

> 
>> There seems to be an issue with the network adapter. Whenever the guest
>> is shutdown and started again it finds a new network adapter. Rebooting
>> instead of shutting down doesn't show the same behavior.
>>
>> I'm not sure yet why this happens, but it seems to be caused by
>> cloudbase-init.
> 
> I couldn't reproduce this issue so far (using a virtio NIC).



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2024-07-22 10:01 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-07-09 15:12 [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config Mira Limbeck
2024-07-09 15:12 ` [pve-devel] [PATCH docs] cloudinit: add Windows cloudbase-init section Mira Limbeck
2024-07-10 14:35 ` [pve-devel] [PATCH qemu-server] fix 4493: cloud-init: fix generated Windows config Mira Limbeck
2024-07-18 15:51 ` Friedrich Weber
2024-07-22 10:01   ` Mira Limbeck

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal