From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 951E06D737 for ; Mon, 16 Aug 2021 16:51:07 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 855E919FAA for ; Mon, 16 Aug 2021 16:51:07 +0200 (CEST) Received: from mail.herold.me (mail.herold.me [90.187.0.209]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id BE69719F93 for ; Mon, 16 Aug 2021 16:51:02 +0200 (CEST) X-Virus-Scanned: Yes DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=herold.me; s=mail; t=1629125452; bh=WMOVpeA7GtlDFp15b6GSUHXtoh4DRPul6I4ciX56Hzg=; h=Subject:To:References:From:In-Reply-To; b=Wakr+u2IYbwh984KyL518UPo+GgBSB4A7fsdbVIMyFhi1VyL4eCDL9dc1qWJEwzc0 mhVZJrIlsMsb3gdAzVqQOsU8VYzWgKcfkdk0kJqkk+qa+rQr9MJ3gJvADaj3iUTsbb MS75YqFsBdwI/GwT1EuD22KvRdrEhK9z9YsJqzZI= To: Mira Limbeck , pve-devel@lists.proxmox.com References: <20210812191749.2055-1-proxmox8914@herold.me> From: Constantin Herold Message-ID: <19bcff67-e19e-9d66-14f2-2d8b2c9e9bb1@herold.me> Date: Mon, 16 Aug 2021 16:50:17 +0200 Mime-Version: 1.0 In-Reply-To: X-SPAM-LEVEL: Spam detection results: 0 AWL 1.499 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain HTML_MESSAGE 0.001 HTML included in message KAM_INFOUSMEBIZ 0.75 Prevalent use of .info|.us|.me|.me.uk|.biz|xyz|id|rocks|life domains in spam/malware SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [pve-devel] [PATCH qemu-server] fix #2429: allow to specify cloud-init vendor snippet via cicustom X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Aug 2021 14:51:07 -0000 Just ignore my patch, I did not know that there was already one on pve-devel. I tested both your and mine patch with the generic cloud images from debian https://cdimage.debian.org/images/cloud/ Direct link to latest: https://cdimage.debian.org/images/cloud/bullseye/daily/latest/debian-11-generic-amd64-daily.qcow2 The bullseye image ships with cloud-init 20.4.1, buster with 20.2 After any of the patches gets merged it would be great to add a example to the existing proxmox wiki. Since the wiki is not in the git repo I can't submit a patch for it. https://pve.proxmox.com/wiki/Cloud-Init_Support ##Bootstrap Cloud-Init Image## A vendor config can be used to bootstrap cloud-init images. For example to install qemu-guest-agent on debian/ubuntu distros after the vm has been deployed create the following snippet. Note that the vendor config is executed on first boot only ! qm set 9000 --cicustom "vendor=local:snippets/vendor.yaml" /var/lib/vz/snippets/vendor.yaml: #cloud-config runcmd: - apt update - apt install -y qemu-guest-agent - systemctl start qemu-guest-agent Am 16.08.2021 um 13:46 schrieb Mira Limbeck: > Thank you for the patch. Sorry that I missed that on friday. > > Which distributions and cloud-init versions did you test this on? > > > The patch looks good to me. One addition I'd make, as we have a hard > limit of 4MiB currently for the generated ISO, inline. > > On 8/12/21 9:17 PM, Constantin Herold wrote: >> Signed-off-by: Constantin Herold >> --- >>   PVE/QemuServer.pm           |  8 ++++++++ >>   PVE/QemuServer/Cloudinit.pm | 18 +++++++++++++----- >>   2 files changed, 21 insertions(+), 5 deletions(-) >> >> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm >> index cc73af8..90a4496 100644 >> --- a/PVE/QemuServer.pm >> +++ b/PVE/QemuServer.pm >> @@ -713,6 +713,14 @@ my $cicustom_fmt = { >>       format => 'pve-volume-id', >>       format_description => 'volume', >>       }, >> +    vendor => { >> +    type => 'string', >> +    optional => 1, >> +    description => 'Specify a custom file containing all vendor data >> passed to the VM via' >> +        .' cloud-init.', >> +    format => 'pve-volume-id', >> +    format_description => 'volume', >> +    }, >>   }; >>   PVE::JSONSchema::register_format('pve-qm-cicustom', $cicustom_fmt); >>   diff --git a/PVE/QemuServer/Cloudinit.pm b/PVE/QemuServer/Cloudinit.pm >> index a5474d3..ab5d358 100644 >> --- a/PVE/QemuServer/Cloudinit.pm >> +++ b/PVE/QemuServer/Cloudinit.pm >> @@ -227,7 +227,7 @@ EOF >>   sub generate_configdrive2 { >>       my ($conf, $vmid, $drive, $volname, $storeid) = @_; >>   -    my ($user_data, $network_data, $meta_data) = >> get_custom_cloudinit_files($conf); >> +    my ($user_data, $network_data, $meta_data, $vendor_data) = >> get_custom_cloudinit_files($conf); >>       $user_data = cloudinit_userdata($conf, $vmid) if >> !defined($user_data); >>       $network_data = configdrive2_network($conf) if >> !defined($network_data); >>   @@ -237,7 +237,8 @@ sub generate_configdrive2 { > > Add a size check for the total amount of bytes to never exceed 3MiB here. > > e.g. > > +    # we always allocate a 4MiB disk for cloudinit and with the > overhead of the ISO > +    # make sure we always stay below it by keeping the sum of all > files below 3 MiB > +    my $sum = length($user_data) + length($vendor_data) + > length($network_data) + length($meta_data); > +    die "Cloud-Init sum of snippets too big (> 3 MiB)\n" if $sum > (3 > * 1024 * 1024); > > This is taken from my patch sent previously [0]. > >>       my $files = { >>       '/openstack/latest/user_data' => $user_data, >>       '/openstack/content/0000' => $network_data, >> -    '/openstack/latest/meta_data.json' => $meta_data >> +    '/openstack/latest/meta_data.json' => $meta_data, >> +    '/openstack/latest/vendor_data.json' => $vendor_data >>       }; >>       commit_cloudinit_disk($conf, $vmid, $drive, $volname, $storeid, >> $files, 'config-2'); >>   } >> @@ -493,7 +494,7 @@ sub nocloud_gen_metadata { >>   sub generate_nocloud { >>       my ($conf, $vmid, $drive, $volname, $storeid) = @_; >>   -    my ($user_data, $network_data, $meta_data) = >> get_custom_cloudinit_files($conf); >> +    my ($user_data, $network_data, $meta_data, $vendor_data) = >> get_custom_cloudinit_files($conf); >>       $user_data = cloudinit_userdata($conf, $vmid) if >> !defined($user_data); >>       $network_data = nocloud_network($conf) if !defined($network_data); >>   @@ -504,7 +505,8 @@ sub generate_nocloud { >>       my $files = { >>       '/user-data' => $user_data, >>       '/network-config' => $network_data, >> -    '/meta-data' => $meta_data >> +    '/meta-data' => $meta_data, >> +    '/vendor-data' => $vendor_data >>       }; >>       commit_cloudinit_disk($conf, $vmid, $drive, $volname, $storeid, >> $files, 'cidata'); >>   } >> @@ -518,6 +520,7 @@ sub get_custom_cloudinit_files { >>       my $network_volid = $files->{network}; >>       my $user_volid = $files->{user}; >>       my $meta_volid = $files->{meta}; >> +    my $vendor_volid = $files->{vendor}; >>         my $storage_conf = PVE::Storage::config(); >>   @@ -536,7 +539,12 @@ sub get_custom_cloudinit_files { >>       $meta_data = read_cloudinit_snippets_file($storage_conf, >> $meta_volid); >>       } >>   -    return ($user_data, $network_data, $meta_data); >> +    my $vendor_data; >> +    if ($vendor_volid) { >> +        $vendor_data = read_cloudinit_snippets_file($storage_conf, >> $vendor_volid); >> +    } >> + >> +    return ($user_data, $network_data, $meta_data, $vendor_data); >>   } >>     sub read_cloudinit_snippets_file { > [0] https://lists.proxmox.com/pipermail/pve-devel/2021-June/049025.html > > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >