From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <proxmox8914@herold.me> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 951E06D737 for <pve-devel@lists.proxmox.com>; Mon, 16 Aug 2021 16:51:07 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 855E919FAA for <pve-devel@lists.proxmox.com>; Mon, 16 Aug 2021 16:51:07 +0200 (CEST) Received: from mail.herold.me (mail.herold.me [90.187.0.209]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id BE69719F93 for <pve-devel@lists.proxmox.com>; Mon, 16 Aug 2021 16:51:02 +0200 (CEST) X-Virus-Scanned: Yes DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=herold.me; s=mail; t=1629125452; bh=WMOVpeA7GtlDFp15b6GSUHXtoh4DRPul6I4ciX56Hzg=; h=Subject:To:References:From:In-Reply-To; b=Wakr+u2IYbwh984KyL518UPo+GgBSB4A7fsdbVIMyFhi1VyL4eCDL9dc1qWJEwzc0 mhVZJrIlsMsb3gdAzVqQOsU8VYzWgKcfkdk0kJqkk+qa+rQr9MJ3gJvADaj3iUTsbb MS75YqFsBdwI/GwT1EuD22KvRdrEhK9z9YsJqzZI= To: Mira Limbeck <m.limbeck@proxmox.com>, pve-devel@lists.proxmox.com References: <20210812191749.2055-1-proxmox8914@herold.me> <c8a6289c-9b7d-ee95-cb09-8b0098fe2cd9@proxmox.com> From: Constantin Herold <proxmox8914@herold.me> Message-ID: <19bcff67-e19e-9d66-14f2-2d8b2c9e9bb1@herold.me> Date: Mon, 16 Aug 2021 16:50:17 +0200 Mime-Version: 1.0 In-Reply-To: <c8a6289c-9b7d-ee95-cb09-8b0098fe2cd9@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL 1.499 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain HTML_MESSAGE 0.001 HTML included in message KAM_INFOUSMEBIZ 0.75 Prevalent use of .info|.us|.me|.me.uk|.biz|xyz|id|rocks|life domains in spam/malware SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [pve-devel] [PATCH qemu-server] fix #2429: allow to specify cloud-init vendor snippet via cicustom X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> List-Post: <mailto:pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Mon, 16 Aug 2021 14:51:07 -0000 Just ignore my patch, I did not know that there was already one on pve-devel. I tested both your and mine patch with the generic cloud images from debian https://cdimage.debian.org/images/cloud/ Direct link to latest: https://cdimage.debian.org/images/cloud/bullseye/daily/latest/debian-11-generic-amd64-daily.qcow2 The bullseye image ships with cloud-init 20.4.1, buster with 20.2 After any of the patches gets merged it would be great to add a example to the existing proxmox wiki. Since the wiki is not in the git repo I can't submit a patch for it. https://pve.proxmox.com/wiki/Cloud-Init_Support ##Bootstrap Cloud-Init Image## A vendor config can be used to bootstrap cloud-init images. For example to install qemu-guest-agent on debian/ubuntu distros after the vm has been deployed create the following snippet. Note that the vendor config is executed on first boot only ! qm set 9000 --cicustom "vendor=local:snippets/vendor.yaml" /var/lib/vz/snippets/vendor.yaml: #cloud-config runcmd: - apt update - apt install -y qemu-guest-agent - systemctl start qemu-guest-agent Am 16.08.2021 um 13:46 schrieb Mira Limbeck: > Thank you for the patch. Sorry that I missed that on friday. > > Which distributions and cloud-init versions did you test this on? > > > The patch looks good to me. One addition I'd make, as we have a hard > limit of 4MiB currently for the generated ISO, inline. > > On 8/12/21 9:17 PM, Constantin Herold wrote: >> Signed-off-by: Constantin Herold <proxmox8914@herold.me> >> --- >> PVE/QemuServer.pm | 8 ++++++++ >> PVE/QemuServer/Cloudinit.pm | 18 +++++++++++++----- >> 2 files changed, 21 insertions(+), 5 deletions(-) >> >> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm >> index cc73af8..90a4496 100644 >> --- a/PVE/QemuServer.pm >> +++ b/PVE/QemuServer.pm >> @@ -713,6 +713,14 @@ my $cicustom_fmt = { >> format => 'pve-volume-id', >> format_description => 'volume', >> }, >> + vendor => { >> + type => 'string', >> + optional => 1, >> + description => 'Specify a custom file containing all vendor data >> passed to the VM via' >> + .' cloud-init.', >> + format => 'pve-volume-id', >> + format_description => 'volume', >> + }, >> }; >> PVE::JSONSchema::register_format('pve-qm-cicustom', $cicustom_fmt); >> diff --git a/PVE/QemuServer/Cloudinit.pm b/PVE/QemuServer/Cloudinit.pm >> index a5474d3..ab5d358 100644 >> --- a/PVE/QemuServer/Cloudinit.pm >> +++ b/PVE/QemuServer/Cloudinit.pm >> @@ -227,7 +227,7 @@ EOF >> sub generate_configdrive2 { >> my ($conf, $vmid, $drive, $volname, $storeid) = @_; >> - my ($user_data, $network_data, $meta_data) = >> get_custom_cloudinit_files($conf); >> + my ($user_data, $network_data, $meta_data, $vendor_data) = >> get_custom_cloudinit_files($conf); >> $user_data = cloudinit_userdata($conf, $vmid) if >> !defined($user_data); >> $network_data = configdrive2_network($conf) if >> !defined($network_data); >> @@ -237,7 +237,8 @@ sub generate_configdrive2 { > > Add a size check for the total amount of bytes to never exceed 3MiB here. > > e.g. > > + # we always allocate a 4MiB disk for cloudinit and with the > overhead of the ISO > + # make sure we always stay below it by keeping the sum of all > files below 3 MiB > + my $sum = length($user_data) + length($vendor_data) + > length($network_data) + length($meta_data); > + die "Cloud-Init sum of snippets too big (> 3 MiB)\n" if $sum > (3 > * 1024 * 1024); > > This is taken from my patch sent previously [0]. > >> my $files = { >> '/openstack/latest/user_data' => $user_data, >> '/openstack/content/0000' => $network_data, >> - '/openstack/latest/meta_data.json' => $meta_data >> + '/openstack/latest/meta_data.json' => $meta_data, >> + '/openstack/latest/vendor_data.json' => $vendor_data >> }; >> commit_cloudinit_disk($conf, $vmid, $drive, $volname, $storeid, >> $files, 'config-2'); >> } >> @@ -493,7 +494,7 @@ sub nocloud_gen_metadata { >> sub generate_nocloud { >> my ($conf, $vmid, $drive, $volname, $storeid) = @_; >> - my ($user_data, $network_data, $meta_data) = >> get_custom_cloudinit_files($conf); >> + my ($user_data, $network_data, $meta_data, $vendor_data) = >> get_custom_cloudinit_files($conf); >> $user_data = cloudinit_userdata($conf, $vmid) if >> !defined($user_data); >> $network_data = nocloud_network($conf) if !defined($network_data); >> @@ -504,7 +505,8 @@ sub generate_nocloud { >> my $files = { >> '/user-data' => $user_data, >> '/network-config' => $network_data, >> - '/meta-data' => $meta_data >> + '/meta-data' => $meta_data, >> + '/vendor-data' => $vendor_data >> }; >> commit_cloudinit_disk($conf, $vmid, $drive, $volname, $storeid, >> $files, 'cidata'); >> } >> @@ -518,6 +520,7 @@ sub get_custom_cloudinit_files { >> my $network_volid = $files->{network}; >> my $user_volid = $files->{user}; >> my $meta_volid = $files->{meta}; >> + my $vendor_volid = $files->{vendor}; >> my $storage_conf = PVE::Storage::config(); >> @@ -536,7 +539,12 @@ sub get_custom_cloudinit_files { >> $meta_data = read_cloudinit_snippets_file($storage_conf, >> $meta_volid); >> } >> - return ($user_data, $network_data, $meta_data); >> + my $vendor_data; >> + if ($vendor_volid) { >> + $vendor_data = read_cloudinit_snippets_file($storage_conf, >> $vendor_volid); >> + } >> + >> + return ($user_data, $network_data, $meta_data, $vendor_data); >> } >> sub read_cloudinit_snippets_file { > [0] https://lists.proxmox.com/pipermail/pve-devel/2021-June/049025.html > > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >