From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 6C7986BF2B for ; Sat, 20 Feb 2021 19:34:56 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5FE7021FE2 for ; Sat, 20 Feb 2021 19:34:56 +0100 (CET) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id F31A621FD7 for ; Sat, 20 Feb 2021 19:34:52 +0100 (CET) Received: by mail-wr1-x42b.google.com with SMTP id v14so14459998wro.7 for ; Sat, 20 Feb 2021 10:34:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=odiso-com.20150623.gappssmtp.com; s=20150623; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=eOPaPWkKUhVONMwmUVKfk8+Fwu+Rmg/OSP7q6KWwoDc=; b=nhsvD0h9phK4L9cVAkKd3vIH9/8ZSyU8IGNOf/fnDIrmlqMDzJBC7SR4uS0YyX8bSL dWaw51vWE0x2lCRmoH/j7wnK9ssLfv+oDLle8ZFhhqwUQXAM6fWXsiTLNpg3Pokpb5/p NpVEOnjm3Y2fRUxU6fRVJssN+DsPBv0xPQY6l6Sgm8udeR8ffeAuDxej/wo8QEPetPF6 p9d6Du/Bm1jOqo9oWG0nezD/0yke13SrIgpD6c2L2qj1EpeUTZx9wShpTlrgJyEtkioW H48tx5jUqGknUag5f3QnpQDnRypAuqOZYnBhYk7CrpNJuEHiwUeo+lejpxhibqVUjVdr FjwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=eOPaPWkKUhVONMwmUVKfk8+Fwu+Rmg/OSP7q6KWwoDc=; b=t/7qs1T+hztDduEoWedfGLOql/Z9SMKQWRdMQs+LqIhAdvVF65Jaoa2gDIfScRTPKA uOzwEkDKn4SEVdZBCWSOR0YZVjYr0+7/Kuqdv0noUgshucGNxv0Z1MpKcMyKYyZxujdV VmjoRb5gE6zfL02+T2w3UEb5nvLvXhtXJs/uYHl88sb9OZ7M/uU7Ug0/ELbR1mypA+Yt 7RXmEgg01k8yXXkLw1K6vsOqytsaf51vHmoJLZIfTuUbIVfV7ZLEYI8NUeRE9Vtypa3b 6k9xPHydJ2VDq7roXoymRjpPdwSHNnHgtdrB5GpyNBbrUWGLjRFe0AD4iMLXATQZCQ1V D1XA== X-Gm-Message-State: AOAM531NUFtomgbUm+dEPlJtyPdkKclr3XY+Zo3AuyoWbi54c/0MQEa2 Ai6DoPpr236A+9ECdtp/EfxAowmdGINKd2I1 X-Google-Smtp-Source: ABdhPJz9eq7q0wm4+pDmvGXgcB26Ydr91NYKdubjnyuCQm8p+NSr1fl8qRDwPWXQjFGMo6J3F0aL9Q== X-Received: by 2002:adf:f14b:: with SMTP id y11mr7709034wro.80.1613846086307; Sat, 20 Feb 2021 10:34:46 -0800 (PST) Received: from ?IPv6:2a0a:1580:0:1::100c? (ovpn1.odiso.net. [2a0a:1580:2000::3f]) by smtp.gmail.com with ESMTPSA id o129sm18486496wme.21.2021.02.20.10.34.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 Feb 2021 10:34:45 -0800 (PST) Message-ID: From: aderumier@odiso.com To: Proxmox VE development discussion Date: Sat, 20 Feb 2021 19:34:43 +0100 In-Reply-To: <8c94a7d6-542b-8927-143d-7ad733748202@proxmox.com> References: <20210207141832.3283706-1-aderumier@odiso.com> <8c94a7d6-542b-8927-143d-7ad733748202@proxmox.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.4 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.122 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [qemuserver.pm, cloudinit.pm, proxmox.com] Subject: Re: [pve-devel] [PATCH V2 qemu-server] fix #1739: cloudinit: add cisshdeletehostkeys option X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2021 18:34:56 -0000 >What did you use to test this patch (OS, cloud-init version, specific > cloudimg)? I have tested with debian10. I have done more tests, on my side, the first time I change instance- id.( so changing ip address for example), it's correctly regenerated or not the ssh keys. But, if I change multiple time ip address to change instance-id, and reuse same (change ip from 192.168.0.1 (generation cloudinit: sshgen yes : ok) -> 192.168.0.2 (generation cloudinit - sshgen yes : ok) -> 192.168.0.1 - (generation cloudinit: sshgen yes : not ok) So, maybe it's related to local cache. I don't have problem with ip config generation. Maybe it's buggy, or not well tested . (anyway cloud-init is not really done to be reused multiple time). so I think it's better to not wasting time to workaround it. Thanks for your time to review and test. Personnaly, I'm currently working to improve opennebula one context  (Thomas have merged my patch for opennebule config drive format this month), adding missing dhcp config support) and removing non-needed dependencies. https://github.com/aderumier/addon-context-linux/commits/proxmox It's really working better than cloudinit, online changes are working out of the box, fs resize on disk resize is working too, simple bash scripts,... Alexandre Le mercredi 17 février 2021 à 16:01 +0100, Mira Limbeck a écrit : > Thank you for the patch. > > I tried to test this patch with both a CentOS 8 and Ubuntu 20.04 VM and > it never worked right. > > I added another line for 'once|yes|on|true' where ssh_deletekeys is set > to 'true' as that is missing in this patch. Then I tried it with CentOS > 8 and cloud-init 19.4. > > In the beginning the keys were deleted and regenerated, but after a few > change anything. Even with a new instance-id the keys were never > deleted > no regenerated. > > In Ubuntu 20.04 with cloud-init 20.4 it was the other way around, even > rebooting without any changes to the cloud-init config (no instance-id > every time. > > cloudimg)? > > > Some additional comments inline > > On 2/7/21 3:18 PM, Alexandre Derumier wrote: > > This define behaviour of ssh server keys generation on cloudinit > > config change. > > > > different value: > > > > - once : only once at vmstart  (default value) > > - no|off|false : never generate ssh key > > - yes|on|true: always generate ssh key > > > > When value is defined to 'once', the value is rewriten to 'no' > > in vmconfig after vm start > > > > We need to change instance-id at each config change because > > network configuration is only done once in pre-init local service, > > and a lock is done on instance-id, and they are no way to change > > that in cloudinit vm config. > > > > The ssh host keys are regenerated by ssh module, once by instance. > > user could change that,and do only once, > > but in this same ssh module, the user public keys are also deployed. > > > > Signed-off-by: Alexandre Derumier > > --- > >   PVE/QemuServer.pm           |  9 ++++++++- > >   PVE/QemuServer/Cloudinit.pm | 10 +++++++++- > >   2 files changed, 17 insertions(+), 2 deletions(-) > > > > diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm > > index f401baf..c899b35 100644 > > --- a/PVE/QemuServer.pm > > +++ b/PVE/QemuServer.pm > > @@ -760,6 +760,13 @@ my $confdesc_cloudinit = { > >         format => 'urlencoded', > >         description => "cloud-init: Setup public SSH keys (one key > > per line, OpenSSH format).", > >       }, > > +    cisshdeletehostkeys => { > > +       optional => 1, > > +       type => 'string', > > +       enum => [qw(once yes no on off true false)], > > +       default_key => 1, > > +       description => "cloud-init: Regenerate host SSH keys on > > config change.", > > +    }, > >   }; > > As mentioned on the previous patch, I'm not sure if we want to add > another cloud-init config to the global config namespace. We could > reuse > the option 'cicustom' for this. > > Perhaps @Thomas could chime in regarding this? > > >   > >   # what about other qemu settings ? > > @@ -4951,7 +4958,7 @@ sub vm_start_nolock { > >         $conf = PVE::QemuConfig->load_config($vmid); # update/reload > >       } > >   > > -    PVE::QemuServer::Cloudinit::generate_cloudinitconfig($conf, > > $vmid); > > +    PVE::QemuServer::Cloudinit::generate_cloudinitconfig($conf, > > $vmid, 1); > >   > >       my $defaults = load_defaults(); > >   > > diff --git a/PVE/QemuServer/Cloudinit.pm > > b/PVE/QemuServer/Cloudinit.pm > > index c464bf3..7b24f93 100644 > > --- a/PVE/QemuServer/Cloudinit.pm > > +++ b/PVE/QemuServer/Cloudinit.pm > > @@ -135,6 +135,7 @@ sub cloudinit_userdata { > >         foreach my $k (@$keys) { > >             $content .= "  - $k\n"; > >         } > > +       $content .= "ssh_deletekeys: false\n" if defined($conf- > > >{cisshdeletehostkeys}) && $conf->{cisshdeletehostkeys} =~ > > m/^(off|no|false)/; > > This is missing the case for 'once', 'on', 'true' and 'yes' where > ssh_deletekeys is set to 'true'. > > And there's an additional whitespace at the end of the line. > > >   > >       } > >       $content .= "chpasswd:\n"; > >       $content .= "  expire: False\n"; > > @@ -554,9 +555,10 @@ my $cloudinit_methods = { > >   }; > >   > >   sub generate_cloudinitconfig { > > -    my ($conf, $vmid) = @_; > > +    my ($conf, $vmid, $vmstart) = @_; > >   > >       my $format = get_cloudinit_format($conf); > > +    my $generated = undef; > >   > >       PVE::QemuConfig->foreach_volume($conf, sub { > >           my ($ds, $drive) = @_; > > @@ -569,7 +571,13 @@ sub generate_cloudinitconfig { > >             or die "missing cloudinit methods for format > > '$format'\n"; > >   > >         $generator->($conf, $vmid, $drive, $volname, $storeid); > > +       $generated = 1; > >       }); > > + > > +    if ($vmstart && $generated && (!defined($conf- > > >{cisshdeletehostkeys}) || $conf->{cisshdeletehostkeys} eq 'once')) { > > +       $conf->{cisshdeletehostkeys} = 'no'; > > +       PVE::QemuConfig->write_config($vmid, $conf); > > +    } > > This always sets 'cisshdeletehostkeys' to 'no' if it is not set at all > in the config which changes the default. I'd rather not do that for > all, > but only if it is set explicitly to 'once'. > > So that would then be: > > if ($vmstart && $generated && defined($conf->{cisshdeletehostkeys}) && > $conf->{cisshdeletehostkeys} eq 'once') > > >   } > >   > >   sub dump_cloudinit_config { > > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >