* [pve-devel] [PATCH cluster v2 1/1] add profiles.cfg to cluster fs
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH guest-common v2 1/1] add profiles section config plugin Dominik Csapak
` (8 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/PVE/Cluster.pm | 1 +
src/pmxcfs/status.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index cfa2583..c01bf89 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -80,6 +80,7 @@ my $observed = {
'sdn/dns.cfg' => 1,
'sdn/.running-config' => 1,
'virtual-guest/cpu-models.conf' => 1,
+ 'virtual-guest/profiles.cfg' => 1,
'mapping/pci.cfg' => 1,
'mapping/usb.cfg' => 1,
};
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index c8094ac..bcec079 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -109,6 +109,7 @@ static memdb_change_t memdb_change_array[] = {
{ .path = "sdn/dns.cfg" },
{ .path = "sdn/.running-config" },
{ .path = "virtual-guest/cpu-models.conf" },
+ { .path = "virtual-guest/profiles.cfg" },
{ .path = "firewall/cluster.fw" },
{ .path = "mapping/pci.cfg" },
{ .path = "mapping/usb.cfg" },
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH guest-common v2 1/1] add profiles section config plugin
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH cluster v2 1/1] add profiles.cfg to cluster fs Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 12:41 ` Thomas Lamprecht
2023-11-14 10:35 ` [pve-devel] [PATCH qemu-server v2 1/3] add the VM profiles plugin Dominik Csapak
` (7 subsequent siblings)
9 siblings, 1 reply; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
this is intended to house custom profiles which can be used
on guest creation instead of manually needing to specify every option.
we do special things here:
* we always set 'allow_unknown' to 1, because when using the guest
specific parts in the cli, we cannot depend on the other one, else
we'd get a cyclic dependency, and without that we need to ignore
unknown sections
* we pack the type and id in the global options, so that when using
it with a seperated section config they get included in the
create/updateSchema
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/Makefile | 2 ++
src/PVE/Profiles/Plugin.pm | 74 ++++++++++++++++++++++++++++++++++++++
2 files changed, 76 insertions(+)
create mode 100644 src/PVE/Profiles/Plugin.pm
diff --git a/src/Makefile b/src/Makefile
index cbc40c1..d99645c 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -17,6 +17,8 @@ install: PVE
install -d ${PERL5DIR}/PVE/Mapping
install -m 0644 PVE/Mapping/PCI.pm ${PERL5DIR}/PVE/Mapping/
install -m 0644 PVE/Mapping/USB.pm ${PERL5DIR}/PVE/Mapping/
+ install -d ${PERL5DIR}/PVE/Profiles
+ install -m 0644 PVE/Profiles/Plugin.pm ${PERL5DIR}/PVE/Profiles/
install -d ${PERL5DIR}/PVE/VZDump
install -m 0644 PVE/VZDump/Plugin.pm ${PERL5DIR}/PVE/VZDump/
install -m 0644 PVE/VZDump/Common.pm ${PERL5DIR}/PVE/VZDump/
diff --git a/src/PVE/Profiles/Plugin.pm b/src/PVE/Profiles/Plugin.pm
new file mode 100644
index 0000000..6ea2b5f
--- /dev/null
+++ b/src/PVE/Profiles/Plugin.pm
@@ -0,0 +1,74 @@
+package PVE::Profiles::Plugin;
+
+use strict;
+use warnings;
+
+use PVE::Cluster qw(cfs_register_file);
+use PVE::SectionConfig;
+
+use base qw(PVE::SectionConfig);
+
+my $CFG_PATH = 'virtual-guest/profiles.cfg';
+
+cfs_register_file ($CFG_PATH,
+ sub { __PACKAGE__->parse_config(@_); },
+ sub { __PACKAGE__->write_config(@_); });
+
+my $defaultData = {
+ propertyList => {
+ type => { description => 'Profile type.' },
+ id => {
+ type => 'string',
+ description => "The ID of the profile.",
+ format => 'pve-configid',
+ },
+ 'profile-description' => {
+ description => "Description.",
+ type => 'string',
+ optional => 1,
+ maxLength => 4096,
+ },
+ },
+ options => {
+ type => {},
+ id => {},
+ 'profile-description' => { optional => 1 },
+ },
+};
+
+sub private {
+ return $defaultData;
+}
+
+sub parse_config {
+ my ($class, $filename, $raw, $allow_unknown) = @_;
+
+ # always allow unknown, so that qemu-server/pct-container
+ # can parse the file without loading the other plugin type
+ return $class->SUPER::parse_config($filename, $raw, 1);
+}
+
+sub write_config {
+ my ($class, $filename, $cfg, $allow_unknown) = @_;
+
+ return $class->SUPER::write_config($filename, $cfg, 1);
+}
+
+# gets, checks and prepares the guest config
+# throws an error if it does not exist or the type is wrong
+sub get_guest_ready_config {
+ my ($id, $type) = @_;
+
+ my $cfg = PVE::Cluster::cfs_read_file($CFG_PATH);
+
+ my $profile = $cfg->{ids}->{$id};
+ die "no such profile '$id'\n" if !defined $profile;
+ die "wrong type '$profile->{type}'\n" if $profile->{type} ne $type;
+
+ delete $profile->{type};
+ delete $profile->{'profile-description'};
+
+ return $profile;
+}
+
+1;
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [pve-devel] [PATCH guest-common v2 1/1] add profiles section config plugin
2023-11-14 10:35 ` [pve-devel] [PATCH guest-common v2 1/1] add profiles section config plugin Dominik Csapak
@ 2023-11-14 12:41 ` Thomas Lamprecht
0 siblings, 0 replies; 12+ messages in thread
From: Thomas Lamprecht @ 2023-11-14 12:41 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
Am 14/11/2023 um 11:35 schrieb Dominik Csapak:
> this is intended to house custom profiles which can be used
> on guest creation instead of manually needing to specify every option.
>
> we do special things here:
> * we always set 'allow_unknown' to 1, because when using the guest
> specific parts in the cli, we cannot depend on the other one, else
> we'd get a cyclic dependency, and without that we need to ignore
> unknown sections
>
> * we pack the type and id in the global options, so that when using
> it with a seperated section config they get included in the
> create/updateSchema
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> src/Makefile | 2 ++
> src/PVE/Profiles/Plugin.pm | 74 ++++++++++++++++++++++++++++++++++++++
> 2 files changed, 76 insertions(+)
> create mode 100644 src/PVE/Profiles/Plugin.pm
>
> diff --git a/src/Makefile b/src/Makefile
> index cbc40c1..d99645c 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -17,6 +17,8 @@ install: PVE
> install -d ${PERL5DIR}/PVE/Mapping
> install -m 0644 PVE/Mapping/PCI.pm ${PERL5DIR}/PVE/Mapping/
> install -m 0644 PVE/Mapping/USB.pm ${PERL5DIR}/PVE/Mapping/
> + install -d ${PERL5DIR}/PVE/Profiles
> + install -m 0644 PVE/Profiles/Plugin.pm ${PERL5DIR}/PVE/Profiles/
> install -d ${PERL5DIR}/PVE/VZDump
> install -m 0644 PVE/VZDump/Plugin.pm ${PERL5DIR}/PVE/VZDump/
> install -m 0644 PVE/VZDump/Common.pm ${PERL5DIR}/PVE/VZDump/
> diff --git a/src/PVE/Profiles/Plugin.pm b/src/PVE/Profiles/Plugin.pm
> new file mode 100644
> index 0000000..6ea2b5f
> --- /dev/null
> +++ b/src/PVE/Profiles/Plugin.pm
> @@ -0,0 +1,74 @@
> +package PVE::Profiles::Plugin;
> +
> +use strict;
> +use warnings;
> +
> +use PVE::Cluster qw(cfs_register_file);
> +use PVE::SectionConfig;
> +
> +use base qw(PVE::SectionConfig);
> +
> +my $CFG_PATH = 'virtual-guest/profiles.cfg';
> +
> +cfs_register_file ($CFG_PATH,
extra space before opening parenthesis.
> + sub { __PACKAGE__->parse_config(@_); },
> + sub { __PACKAGE__->write_config(@_); });
let's use some more modern indentation here
cfs_register_file(
$CFG_PATH,
sub { __PACKAGE__->parse_config(@_); },
sub { __PACKAGE__->write_config(@_); },
);
> +
> +my $defaultData = {
> + propertyList => {
> + type => { description => 'Profile type.' },
> + id => {
> + type => 'string',
> + description => "The ID of the profile.",
> + format => 'pve-configid',
> + },
> + 'profile-description' => {
> + description => "Description.",
I mean, yes, but a sentence is still nice:
description => "Use this to add a short comment about a profile.",
> + type => 'string',
> + optional => 1,
> + maxLength => 4096,
would start out with a bit less, as this is rendered in grids and the like
and doesn't needs to hold that many information than e.g., node or guest
instances might have.
> + },
> + },
> + options => {
> + type => {},
> + id => {},
> + 'profile-description' => { optional => 1 },
> + },
> +};
> +
> +sub private {
> + return $defaultData;
> +}
> +
> +sub parse_config {
> + my ($class, $filename, $raw, $allow_unknown) = @_;
> +
> + # always allow unknown, so that qemu-server/pct-container
to make it slightly easier to understand it'd IMO help to:
s/unknown/unknown section types/
> + # can parse the file without loading the other plugin type
> + return $class->SUPER::parse_config($filename, $raw, 1);
> +}
> +
> +sub write_config {
> + my ($class, $filename, $cfg, $allow_unknown) = @_;
> +
> + return $class->SUPER::write_config($filename, $cfg, 1);
> +}
> +
> +# gets, checks and prepares the guest config
> +# throws an error if it does not exist or the type is wrong
> +sub get_guest_ready_config {
This is rather covoluted name for what happens, wouldn't naming this
`load_profile` be clear enough? Or are there future changes, or overrides
in child-implementations that warrant this name (even then we probably
find something better)
> + my ($id, $type) = @_;
Otherwise I'd use at least the following variable names to clarify what
$id's we're talking about, because from the method name I initially thought
those are vmid's (was cleared up quickly when continue reading the code, but
IMO still not ideal as is)
my ($profile_id, $profile_type) = @_;
> +
> + my $cfg = PVE::Cluster::cfs_read_file($CFG_PATH);
> +
> + my $profile = $cfg->{ids}->{$id};
> + die "no such profile '$id'\n" if !defined $profile;
> + die "wrong type '$profile->{type}'\n" if $profile->{type} ne $type;
> +
> + delete $profile->{type};
> + delete $profile->{'profile-description'};
> +
> + return $profile;
> +}
> +
> +1;
^ permalink raw reply [flat|nested] 12+ messages in thread
* [pve-devel] [PATCH qemu-server v2 1/3] add the VM profiles plugin
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH cluster v2 1/1] add profiles.cfg to cluster fs Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH guest-common v2 1/1] add profiles section config plugin Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH qemu-server v2 2/3] api: add profile option to create vm api call Dominik Csapak
` (6 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
simply uses the json_config_properties for the vm config
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* no prefixing with vm anymore
PVE/Makefile | 1 +
PVE/Profiles/Makefile | 5 +++++
PVE/Profiles/VM.pm | 28 ++++++++++++++++++++++++++++
3 files changed, 34 insertions(+)
create mode 100644 PVE/Profiles/Makefile
create mode 100644 PVE/Profiles/VM.pm
diff --git a/PVE/Makefile b/PVE/Makefile
index dc173681..d09ca98d 100644
--- a/PVE/Makefile
+++ b/PVE/Makefile
@@ -12,3 +12,4 @@ install:
$(MAKE) -C API2 install
$(MAKE) -C CLI install
$(MAKE) -C QemuServer install
+ $(MAKE) -C Profiles install
diff --git a/PVE/Profiles/Makefile b/PVE/Profiles/Makefile
new file mode 100644
index 00000000..e5f56833
--- /dev/null
+++ b/PVE/Profiles/Makefile
@@ -0,0 +1,5 @@
+SOURCES=VM.pm
+
+.PHONY: install
+install: ${SOURCES}
+ for i in ${SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/Profiles/$$i; done
diff --git a/PVE/Profiles/VM.pm b/PVE/Profiles/VM.pm
new file mode 100644
index 00000000..dc664ec5
--- /dev/null
+++ b/PVE/Profiles/VM.pm
@@ -0,0 +1,28 @@
+package PVE::Profiles::VM;
+
+use strict;
+use warnings;
+
+use PVE::Profiles::Plugin;
+use PVE::QemuServer;
+
+use base qw(PVE::Profiles::Plugin);
+
+sub type {
+ return "vm";
+}
+
+sub properties {
+ return PVE::QemuServer::json_config_properties();
+}
+
+sub options {
+ my $props = PVE::QemuServer::json_config_properties();
+ my $opts = {};
+ for my $opt (keys $props->%*) {
+ $opts->{$opt} = { optional => 1 };
+ }
+ return $opts;
+}
+
+1;
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH qemu-server v2 2/3] api: add profile option to create vm api call
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
` (2 preceding siblings ...)
2023-11-14 10:35 ` [pve-devel] [PATCH qemu-server v2 1/3] add the VM profiles plugin Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH qemu-server v2 3/3] qm: register and init the profiles plugins Dominik Csapak
` (5 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
we use the the profile cfg as the 'param' hash, but overwrite the values
with the ones from the api call, so one can overwrite options from the
profile easily
also we add the used profile to the log, since
it might be interesting which one was used
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* log profile instead of putting it into the meta object
* use helper from Plugin class
* use /mapping/guest-profile as acl path now
PVE/API2/Qemu.pm | 23 +++++++++++++++++++++++
PVE/QemuServer.pm | 6 ++++++
2 files changed, 29 insertions(+)
diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 38bdaabd..0d8bbdea 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -49,6 +49,9 @@ use PVE::SSHInfo;
use PVE::Replication;
use PVE::StorageTunnel;
+use PVE::Profiles::Plugin;
+use PVE::Profiles::VM;
+
BEGIN {
if (!$ENV{PVE_GENERATING_DOCS}) {
require PVE::HA::Env::PVE2;
@@ -837,6 +840,11 @@ __PACKAGE__->register_method({
default => 0,
description => "Start VM after it was created successfully.",
},
+ profile => {
+ optional => 1,
+ type => 'string',
+ description => "The profile to use as base config.",
+ },
},
1, # with_disk_alloc
),
@@ -850,6 +858,19 @@ __PACKAGE__->register_method({
my $rpcenv = PVE::RPCEnvironment::get();
my $authuser = $rpcenv->get_user();
+ my $profile = extract_param($param, 'profile');
+ if (defined($profile)) {
+ $rpcenv->check_full($authuser, "/mapping/guest-profile/${profile}", ['Mapping.Use']);
+ my $profile_cfg = eval { PVE::Profiles::Plugin::get_guest_ready_config($profile, 'vm') };
+ raise_param_exc({ profile => "$@"}) if $@;
+
+ for my $opt (keys $param->%*) {
+ $profile_cfg->{$opt} = $param->{$opt};
+ }
+
+ $param = $profile_cfg;
+ }
+
my $node = extract_param($param, 'node');
my $vmid = extract_param($param, 'vmid');
@@ -1013,6 +1034,8 @@ __PACKAGE__->register_method({
my $conf = $param;
my $arch = PVE::QemuServer::get_vm_arch($conf);
+ print "using profile '$profile'\n" if $profile;
+
$conf->{meta} = PVE::QemuServer::new_meta_info_string();
my $vollist = [];
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index c465fb6f..1c7e65ea 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -293,6 +293,12 @@ my $meta_info_fmt = {
pattern => '\d+(\.\d+)+',
optional => 1,
},
+ 'profile' => {
+ type => 'string',
+ description => 'The Profile used during creation.',
+ format => 'pve-configid',
+ optional => 1,
+ },
};
my $confdesc = {
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH qemu-server v2 3/3] qm: register and init the profiles plugins
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
` (3 preceding siblings ...)
2023-11-14 10:35 ` [pve-devel] [PATCH qemu-server v2 2/3] api: add profile option to create vm api call Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH container v2 1/3] add the CT profiles plugin Dominik Csapak
` (4 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
we have to that here, so the properties/options are correctly configured
when using that feature on the cli
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* use init(1) for separated plugin schemas
PVE/CLI/qm.pm | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/PVE/CLI/qm.pm b/PVE/CLI/qm.pm
index b17b4fe2..c52c972d 100755
--- a/PVE/CLI/qm.pm
+++ b/PVE/CLI/qm.pm
@@ -37,6 +37,12 @@ use PVE::QemuServer::Monitor qw(mon_cmd);
use PVE::QemuServer::OVF;
use PVE::QemuServer;
+use PVE::Profiles::Plugin;
+use PVE::Profiles::VM;
+
+PVE::Profiles::VM->register();
+PVE::Profiles::Plugin->init(1);
+
use PVE::CLIHandler;
use base qw(PVE::CLIHandler);
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH container v2 1/3] add the CT profiles plugin
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
` (4 preceding siblings ...)
2023-11-14 10:35 ` [pve-devel] [PATCH qemu-server v2 3/3] qm: register and init the profiles plugins Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH container v2 2/3] api: add profile option to create ct api call Dominik Csapak
` (3 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
simply uses the json_config_properties for the ct config
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* no prefixing with ct_ anymore
src/PVE/Makefile | 1 +
src/PVE/Profiles/CT.pm | 28 ++++++++++++++++++++++++++++
src/PVE/Profiles/Makefile | 4 ++++
3 files changed, 33 insertions(+)
create mode 100644 src/PVE/Profiles/CT.pm
create mode 100644 src/PVE/Profiles/Makefile
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index 40742e4..e9ad9a3 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -8,5 +8,6 @@ install: ${SOURCES}
make -C LXC install
make -C VZDump install
make -C CLI install
+ make -C Profiles install
diff --git a/src/PVE/Profiles/CT.pm b/src/PVE/Profiles/CT.pm
new file mode 100644
index 0000000..513fad4
--- /dev/null
+++ b/src/PVE/Profiles/CT.pm
@@ -0,0 +1,28 @@
+package PVE::Profiles::CT;
+
+use strict;
+use warnings;
+
+use PVE::Profiles::Plugin;
+use PVE::LXC::Config;
+
+use base qw(PVE::Profiles::Plugin);
+
+sub type {
+ return "ct";
+}
+
+sub properties {
+ return PVE::LXC::Config::json_config_properties();
+}
+
+sub options {
+ my $props = PVE::LXC::Config::json_config_properties();
+ my $opts = {};
+ for my $opt (keys $props->%*) {
+ $opts->{$opt} = { optional => 1 };
+ }
+ return $opts;
+}
+
+1;
diff --git a/src/PVE/Profiles/Makefile b/src/PVE/Profiles/Makefile
new file mode 100644
index 0000000..e63a9f2
--- /dev/null
+++ b/src/PVE/Profiles/Makefile
@@ -0,0 +1,4 @@
+
+.PHONY: install
+install:
+ install -D -m 0644 CT.pm ${PERLDIR}/PVE/Profiles/CT.pm
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH container v2 2/3] api: add profile option to create ct api call
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
` (5 preceding siblings ...)
2023-11-14 10:35 ` [pve-devel] [PATCH container v2 1/3] add the CT profiles plugin Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH container v2 3/3] pct: register and init the profiles plugins Dominik Csapak
` (2 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
we use the profile cfg as the 'param' hash, but overwrite the values
with the ones from the api call, so one can overwrite options from
the profile easily
we also log the used profile
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* use helper from Plugin
* log profile when used
* use /mapping/guest-profile as acl path now
src/PVE/API2/LXC.pm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 28d14de..2b17f8f 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -27,6 +27,10 @@ use PVE::API2::LXC::Config;
use PVE::API2::LXC::Status;
use PVE::API2::LXC::Snapshot;
use PVE::JSONSchema qw(get_standard_option);
+
+use PVE::Profiles::Plugin;
+use PVE::Profiles::CT;
+
use base qw(PVE::RESTHandler);
BEGIN {
@@ -196,6 +200,11 @@ __PACKAGE__->register_method({
default => 0,
description => "Start the CT after its creation finished successfully.",
},
+ profile => {
+ optional => 1,
+ type => 'string',
+ description => "The profile to use as base config.",
+ },
}),
},
returns => {
@@ -209,6 +218,19 @@ __PACKAGE__->register_method({
my $rpcenv = PVE::RPCEnvironment::get();
my $authuser = $rpcenv->get_user();
+ my $profile = extract_param($param, 'profile');
+ if (defined($profile)) {
+ $rpcenv->check_full($authuser, "/mapping/guest-profile/${profile}", ['Mapping.Use']);
+ my $profile_cfg = eval { PVE::Profiles::Plugin::get_guest_ready_config($profile, 'ct') };
+ raise_param_exc({ profile => "$@" }) if $@;
+
+ for my $opt (keys $param->%*) {
+ $profile_cfg->{$opt} = $param->{$opt};
+ }
+
+ $param = $profile_cfg;
+ }
+
my $node = extract_param($param, 'node');
my $vmid = extract_param($param, 'vmid');
my $ignore_unpack_errors = extract_param($param, 'ignore-unpack-errors');
@@ -381,6 +403,7 @@ __PACKAGE__->register_method({
my $vollist = [];
eval {
my $orig_mp_param; # only used if $restore
+ print "using profile '$profile'\n" if $profile;
if ($restore) {
die "can't overwrite running container\n" if PVE::LXC::check_running($vmid);
if ($archive ne '-') {
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH container v2 3/3] pct: register and init the profiles plugins
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
` (6 preceding siblings ...)
2023-11-14 10:35 ` [pve-devel] [PATCH container v2 2/3] api: add profile option to create ct api call Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 10:35 ` [pve-devel] [PATCH manager v2 1/1] api: add guest profile api endpoint Dominik Csapak
2023-11-14 13:15 ` [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Thomas Lamprecht
9 siblings, 0 replies; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
we have to that here, so the properties/options are correctly configured
when using that feature on the cli
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* use init(1) for separated plugins
src/PVE/CLI/pct.pm | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/PVE/CLI/pct.pm b/src/PVE/CLI/pct.pm
index a0b9bce..a2a08c2 100755
--- a/src/PVE/CLI/pct.pm
+++ b/src/PVE/CLI/pct.pm
@@ -24,6 +24,12 @@ use PVE::API2::LXC::Snapshot;
use PVE::API2::LXC::Status;
use PVE::API2::LXC;
+use PVE::Profiles::Plugin;
+use PVE::Profiles::CT;
+
+PVE::Profiles::CT->register();
+PVE::Profiles::Plugin->init(1);
+
use base qw(PVE::CLIHandler);
my $nodename = PVE::INotify::nodename();
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* [pve-devel] [PATCH manager v2 1/1] api: add guest profile api endpoint
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
` (7 preceding siblings ...)
2023-11-14 10:35 ` [pve-devel] [PATCH container v2 3/3] pct: register and init the profiles plugins Dominik Csapak
@ 2023-11-14 10:35 ` Dominik Csapak
2023-11-14 13:15 ` [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Thomas Lamprecht
9 siblings, 0 replies; 12+ messages in thread
From: Dominik Csapak @ 2023-11-14 10:35 UTC (permalink / raw)
To: pve-devel
basic CRUD for the profile section config
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* use raise_param_exc when id or type is wrong
* extract the type from param
* use /mapping/guest-profile as acl path
* add missing index entry for profiles
PVE/API2/Cluster.pm | 7 ++
PVE/API2/Cluster/Makefile | 1 +
PVE/API2/Cluster/Profiles.pm | 230 +++++++++++++++++++++++++++++++++++
3 files changed, 238 insertions(+)
create mode 100644 PVE/API2/Cluster/Profiles.pm
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index 04387ab4..d628df85 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -30,6 +30,7 @@ use PVE::API2::Cluster::Mapping;
use PVE::API2::Cluster::Jobs;
use PVE::API2::Cluster::MetricServer;
use PVE::API2::Cluster::Notifications;
+use PVE::API2::Cluster::Profiles;
use PVE::API2::ClusterConfig;
use PVE::API2::Firewall::Cluster;
use PVE::API2::HAConfig;
@@ -103,6 +104,11 @@ __PACKAGE__->register_method ({
path => 'mapping',
});
+__PACKAGE__->register_method ({
+ subclass => "PVE::API2::Cluster::Profiles",
+ path => 'profiles',
+});
+
if ($have_sdn) {
__PACKAGE__->register_method ({
subclass => "PVE::API2::Network::SDN",
@@ -158,6 +164,7 @@ __PACKAGE__->register_method ({
{ name => 'notifications' },
{ name => 'nextid' },
{ name => 'options' },
+ { name => 'profiles' },
{ name => 'replication' },
{ name => 'resources' },
{ name => 'status' },
diff --git a/PVE/API2/Cluster/Makefile b/PVE/API2/Cluster/Makefile
index b109e5cb..35a3f871 100644
--- a/PVE/API2/Cluster/Makefile
+++ b/PVE/API2/Cluster/Makefile
@@ -9,6 +9,7 @@ PERLSOURCE= \
MetricServer.pm \
Mapping.pm \
Notifications.pm \
+ Profiles.pm \
Jobs.pm \
Ceph.pm
diff --git a/PVE/API2/Cluster/Profiles.pm b/PVE/API2/Cluster/Profiles.pm
new file mode 100644
index 00000000..3d1ec67c
--- /dev/null
+++ b/PVE/API2/Cluster/Profiles.pm
@@ -0,0 +1,230 @@
+package PVE::API2::Cluster::Profiles;
+
+use warnings;
+use strict;
+
+use PVE::Tools qw(extract_param extract_sensitive_params);
+use PVE::Exception qw(raise_perm_exc raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::RPCEnvironment;
+
+use PVE::Profiles::Plugin;
+use PVE::Profiles::VM;
+use PVE::Profiles::CT;
+
+PVE::Profiles::VM->register();
+PVE::Profiles::CT->register();
+PVE::Profiles::Plugin->init(1);
+
+use PVE::RESTHandler;
+
+use base qw(PVE::RESTHandler);
+
+__PACKAGE__->register_method ({
+ name => 'profile_index',
+ path => '',
+ method => 'GET',
+ description => "List configured guest profiles.",
+ permissions => {
+ user => 'all',
+ description => "Only lists entries where you have 'Mapping.Modify', 'Mapping.Use' or".
+ " 'Mapping.Audit' permissions on 'mapping/guest-profile/<id>'.",
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => "object",
+ properties => {
+ id => {
+ description => "The ID of the entry.",
+ type => 'string'
+ },
+ type => {
+ description => "Plugin type.",
+ type => 'string',
+ },
+ },
+ },
+ links => [ { rel => 'child', href => "{id}" } ],
+ },
+ code => sub {
+ my ($param) = @_;
+
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $authuser = $rpcenv->get_user();
+
+ my $res = [];
+ my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+ my $can_see_mapping_privs = ['Mapping.Modify', 'Mapping.Use', 'Mapping.Audit'];
+
+ for my $id (sort keys $cfg->{ids}->%*) {
+ next if !$rpcenv->check_any($authuser, "/mapping/guest-profile/$id", $can_see_mapping_privs, 1);
+ my $plugin_config = $cfg->{ids}->{$id};
+ push @$res, {
+ id => $id,
+ type => $plugin_config->{type},
+ };
+ }
+
+ return $res;
+ }});
+
+__PACKAGE__->register_method ({
+ name => 'read',
+ path => '{id}',
+ method => 'GET',
+ description => "Read profile configuration.",
+ permissions => {
+ check =>['or',
+ ['perm', '/mapping/guest-profile/{id}', ['Mapping.Use']],
+ ['perm', '/mapping/guest-profile/{id}', ['Mapping.Modify']],
+ ['perm', '/mapping/guest-profile/{id}', ['Mapping.Audit']],
+ ],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ id => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ },
+ },
+ returns => { type => 'object' },
+ code => sub {
+ my ($param) = @_;
+
+ my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+ my $id = $param->{id};
+
+ raise_param_exc({id => "no such profile '$id'"}) if !defined($cfg->{ids}->{$id});
+
+ return $cfg->{ids}->{$id};
+ }});
+
+__PACKAGE__->register_method ({
+ name => 'create',
+ path => '{id}',
+ protected => 1,
+ method => 'POST',
+ description => "Create a new profile.",
+ permissions => {
+ check => ['perm', '/mapping/guest-profile', ['Mapping.Modify']],
+ },
+ parameters => PVE::Profiles::Plugin->createSchema(),
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $type = extract_param($param, 'type');
+ my $plugin = PVE::Profiles::Plugin->lookup($type);
+ my $id = extract_param($param, 'id');
+
+ PVE::Cluster::cfs_lock_file('virtual-guest/profiles.cfg', undef, sub {
+ my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+
+ raise_param_exc({id => "Profile '$id' already exists"})
+ if $cfg->{ids}->{$id};
+
+ my $opts = $plugin->check_config($id, $param, 1, 1);
+
+ $cfg->{ids}->{$id} = $opts;
+
+ PVE::Cluster::cfs_write_file('virtual-guest/profiles.cfg', $cfg);
+ });
+ die $@ if $@;
+
+ return;
+ }});
+
+
+__PACKAGE__->register_method ({
+ name => 'update',
+ protected => 1,
+ path => '{id}',
+ method => 'PUT',
+ description => "Update profile configuration.",
+ permissions => {
+ check => ['perm', '/mapping/guest-profile/{id}', ['Mapping.Modify']],
+ },
+ parameters => PVE::Profiles::Plugin->updateSchema(),
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $id = extract_param($param, 'id');
+ my $type = extract_param($param, 'type');
+ my $digest = extract_param($param, 'digest');
+ my $delete = extract_param($param, 'delete');
+
+ if ($delete) {
+ $delete = [PVE::Tools::split_list($delete)];
+ }
+
+ PVE::Cluster::cfs_lock_file('virtual-guest/profiles.cfg', undef, sub {
+ my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+
+ PVE::SectionConfig::assert_if_modified($cfg, $digest);
+
+ my $data = $cfg->{ids}->{$id};
+ raise_param_exc({id => "no such profile '$id'"}) if !defined($data);
+ raise_param_exc({type => "wrong type '$type"}) if $type ne $data->{type};
+
+ my $plugin = PVE::Profiles::Plugin->lookup($data->{type});
+ my $opts = $plugin->check_config($id, $param, 0, 1);
+
+ my $options = $plugin->private()->{options}->{$data->{type}};
+ PVE::SectionConfig::delete_from_config($data, $options, $opts, $delete);
+
+ $data->{$_} = $opts->{$_} for keys $opts->%*;
+
+ PVE::Cluster::cfs_write_file('virtual-guest/profiles.cfg', $cfg);
+ });
+ die $@ if $@;
+
+ return;
+ }});
+
+__PACKAGE__->register_method ({
+ name => 'delete',
+ protected => 1,
+ path => '{id}',
+ method => 'DELETE',
+ description => "Remove profile.",
+ permissions => {
+ check => [ 'perm', '/mapping/guest-profile', ['Mapping.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ id => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ }
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $id = $param->{id};
+
+ PVE::Cluster::cfs_lock_file('virtual-guest/profiles.cfg', undef, sub {
+ my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+
+ if ($cfg->{ids}->{$id}) {
+ delete $cfg->{ids}->{$id};
+ }
+
+ PVE::Cluster::cfs_write_file('virtual-guest/profiles.cfg', $cfg);
+ });
+ die $@ if $@;
+
+ return;
+ }});
+
+1;
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support
2023-11-14 10:35 [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support Dominik Csapak
` (8 preceding siblings ...)
2023-11-14 10:35 ` [pve-devel] [PATCH manager v2 1/1] api: add guest profile api endpoint Dominik Csapak
@ 2023-11-14 13:15 ` Thomas Lamprecht
9 siblings, 0 replies; 12+ messages in thread
From: Thomas Lamprecht @ 2023-11-14 13:15 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
Am 14/11/2023 um 11:35 schrieb Dominik Csapak:
> This series aims to provide profile support when creating guests (ct/vm)
> so that users can reuse options without having to specify them every
> time.
Great! Nice to see that my hopes^Wexpectations about this being really
not that much work/code are being met.
As mentioned off-list, some way for configuring the base defaults for
disks/mountpoints and networks without having to already explicitly
specify the single would be nice to have.
As talked, having a net-defaults, scsi-defaults, mp-defaults, ...
property supported by the respective VM/CT config itself could solve
that and even provide an additional feature.
The values decoded there would be merged with the actual instances,
e.g., scsi0, scsi1, ... or net0, net1, ..., on start.
There might be a few edge cases, and me might not want to support
every option (e.g., a default serial for all disks is rather counter
productive), but otherwise this could be a nice way to encode things
like "discard=on,sdd=1" for all (future) scsi disks, be it for a
specific VM or CT, or also in a profile.
^ permalink raw reply [flat|nested] 12+ messages in thread