From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 32988BE44 for ; Fri, 8 Apr 2022 11:07:48 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 217DACD0A for ; Fri, 8 Apr 2022 11:07:18 +0200 (CEST) Received: from mx1.tetaneutral.net (mx1.tetaneutral.net [IPv6:2a03:7220:8081:5300::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 25D28CD00 for ; Fri, 8 Apr 2022 11:07:17 +0200 (CEST) Received: by mx1.tetaneutral.net (Postfix, from userid 109) id 33F088C0CB; Fri, 8 Apr 2022 11:07:04 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mx1.tetaneutral.net X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, FSL_HELO_NON_FQDN_1 autolearn=ham autolearn_force=no version=3.4.2 Received: from lgnuc (ip165.tetaneutral.net [IPv6:2a03:7220:8080:a500::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.tetaneutral.net (Postfix) with ESMTPS id 05CD88C027 for ; Fri, 8 Apr 2022 11:07:00 +0200 (CEST) Message-ID: <1649408819.6346.22.camel@guerby.net> From: Laurent GUERBY To: Proxmox VE development discussion Date: Fri, 08 Apr 2022 11:06:59 +0200 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.22.6-1+deb9u2 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: Re: [pve-devel] New API endpoint to manage snippets / bugzilla 2208 / updated patch but missing something, need help X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2022 09:07:48 -0000 On Thu, 2022-04-07 at 12:27 +0200, Patryk Ściborek wrote: > Hi Guys! > > I'm using Terraform to manage VMs running on Proxmox. I need to be > able to > upload snippets with custom cloud-init configuration. Right now I > have to > use scp/sftp to upload snippets so I have to use a system account. It > looks > that I'm not the only one having this issue: > https://bugzilla.proxmox.com/show_bug.cgi?id=2208 > > So I thought that I could implement a new API endpoint which would > allow to > create, read, update and delete snippets so other tools like > Terraform > would be able to use it. > > What do you think about this idea? > > Best regards, > Patryk Hi, I've tried to adapt the patch to current PVE 7.1-12 (see below) but I still get proxmoxer.core.ResourceException: 501 Not Implemented: upload failed - {'errors': b''} When I try to upload a snippet. My proxmoxer setup works for iso, the following succeeds: proxmox.nodes(h).storage(s).upload.post(content="iso",filename=f) But the following fails (f being read "rb" from some xxx.yaml) proxmox.nodes(h).storage(s).upload.post(content="snippets",filename=f) Not having snippet upload makes it impossible to use PVE auth realm tokens to control permissions, you have to give a priviledged system account to users *just* to be able to do cloud init with a yaml which is not great security wise (and not practical). I'm probably missing a few things to have a patch that works, I'm willing to put some time on it if someone gives me directions. Thanks! Sincerely, Laurent (paying PVE+PBS customer at work) root@test:/usr/share/perl5# diff -u PVE/Storage.pm{-orig,} --- PVE/Storage.pm-orig 2022-04-08 09:15:52.443943197 +0200 +++ PVE/Storage.pm 2022-04-08 09:17:23.457073570 +0200 @@ -412,6 +412,15 @@ return $plugin->get_subdir($scfg, 'iso'); } +sub get_snippet_dir { + my ($cfg, $storeid) = @_; + + my $scfg = storage_config($cfg, $storeid); + my $plugin = PVE::Storage::Plugin->lookup($scfg->{type}); + + return $plugin->get_subdir($scfg, 'snippets'); +} + sub get_vztmpl_dir { my ($cfg, $storeid) = @_; root@test:/usr/share/perl5# diff -u ./PVE/API2/Storage/Status.pm{-orig,} --- ./PVE/API2/Storage/Status.pm-orig 2022-04-08 09:15:43.883836880 +0200 +++ ./PVE/API2/Storage/Status.pm 2022-04-08 10:23:43.914401204 +0200 @@ -381,7 +381,7 @@ content => { description => "Content type.", type => 'string', format => 'pve-storage-content', - enum => ['iso', 'vztmpl'], + enum => ['iso', 'vztmpl', 'snippets'], }, filename => { description => "The name of the file to create. Caution: This will be normalized!", @@ -446,8 +446,10 @@ raise_param_exc({ filename => "wrong file extension" }); } $path = PVE::Storage::get_vztmpl_dir($cfg, $param->{storage}); - } else { - raise_param_exc({ content => "upload content type '$content' not allowed" }); + } elsif ($content eq 'snippets') { + $path = PVE::Storage::get_snippet_dir($cfg, $param->{storage}); + } else { + raise_param_exc({ content => "upload content type '$content' not allowed" }); } die "storage '$param->{storage}' does not support '$content' content\n" @@ -564,7 +566,7 @@ content => { description => "Content type.", # TODO: could be optional & detected in most cases type => 'string', format => 'pve-storage-content', - enum => ['iso', 'vztmpl'], + enum => ['iso', 'vztmpl', 'snippets'], }, filename => { description => "The name of the file to create. Caution: This will be normalized!", @@ -627,6 +629,8 @@ raise_param_exc({ filename => "wrong file extension" }); } $path = PVE::Storage::get_vztmpl_dir($cfg, $storage); + } elsif ($content eq 'snippets') { + $path = PVE::Storage::get_snippet_dir($cfg, $storage); } else { raise_param_exc({ content => "upload content-type '$content' is not allowed" }); }