[pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support

[pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support

[Dominik Csapak]

This series aims to provide profile support when creating guests (ct/vm)
so that users can reuse options without having to specify them every
time.

UI is still to follow, depends on [0] for the oneOf schema support.

changes from rfc/v1:
* uses a section config with separated plugins, using the oneOf schema
* move the parsing with preparing into a helper into the Plugin class
* don't save the profile into the 'meta' option, but log it instead

0: https://lists.proxmox.com/pipermail/pve-devel/2023-November/060073.html

pve-cluster:

Dominik Csapak (1):
  add profiles.cfg to cluster fs

 src/PVE/Cluster.pm  | 1 +
 src/pmxcfs/status.c | 1 +
 2 files changed, 2 insertions(+)

pve-guest-common:

Dominik Csapak (1):
  add profiles section config plugin

 src/Makefile               |  2 ++
 src/PVE/Profiles/Plugin.pm | 74 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+)
 create mode 100644 src/PVE/Profiles/Plugin.pm

qemu-server:

Dominik Csapak (3):
  add the VM profiles plugin
  api: add profile option to create vm api call
  qm: register and init the profiles plugins

 PVE/API2/Qemu.pm      | 23 +++++++++++++++++++++++
 PVE/CLI/qm.pm         |  6 ++++++
 PVE/Makefile          |  1 +
 PVE/Profiles/Makefile |  5 +++++
 PVE/Profiles/VM.pm    | 28 ++++++++++++++++++++++++++++
 PVE/QemuServer.pm     |  6 ++++++
 6 files changed, 69 insertions(+)
 create mode 100644 PVE/Profiles/Makefile
 create mode 100644 PVE/Profiles/VM.pm

pve-container:

Dominik Csapak (3):
  add the CT profiles plugin
  api: add profile option to create ct api call
  pct: register and init the profiles plugins

 src/PVE/API2/LXC.pm       | 23 +++++++++++++++++++++++
 src/PVE/CLI/pct.pm        |  6 ++++++
 src/PVE/Makefile          |  1 +
 src/PVE/Profiles/CT.pm    | 28 ++++++++++++++++++++++++++++
 src/PVE/Profiles/Makefile |  4 ++++
 5 files changed, 62 insertions(+)
 create mode 100644 src/PVE/Profiles/CT.pm
 create mode 100644 src/PVE/Profiles/Makefile

pve-manager:

Dominik Csapak (1):
  api: add guest profile api endpoint

 PVE/API2/Cluster.pm          |   7 ++
 PVE/API2/Cluster/Makefile    |   1 +
 PVE/API2/Cluster/Profiles.pm | 230 +++++++++++++++++++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 PVE/API2/Cluster/Profiles.pm

-- 
2.30.2

[pve-devel] [PATCH cluster v2 1/1] add profiles.cfg to cluster fs

[Dominik Csapak]

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/PVE/Cluster.pm  | 1 +
 src/pmxcfs/status.c | 1 +
 2 files changed, 2 insertions(+)

diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index cfa2583..c01bf89 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -80,6 +80,7 @@ my $observed = {
     'sdn/dns.cfg' => 1,
     'sdn/.running-config' => 1,
     'virtual-guest/cpu-models.conf' => 1,
+    'virtual-guest/profiles.cfg' => 1,
     'mapping/pci.cfg' => 1,
     'mapping/usb.cfg' => 1,
 };
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index c8094ac..bcec079 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -109,6 +109,7 @@ static memdb_change_t memdb_change_array[] = {
 	{ .path = "sdn/dns.cfg" },
 	{ .path = "sdn/.running-config" },
 	{ .path = "virtual-guest/cpu-models.conf" },
+	{ .path = "virtual-guest/profiles.cfg" },
 	{ .path = "firewall/cluster.fw" },
 	{ .path = "mapping/pci.cfg" },
 	{ .path = "mapping/usb.cfg" },
-- 
2.30.2

[pve-devel] [PATCH guest-common v2 1/1] add profiles section config plugin

[Dominik Csapak]

this is intended to house custom profiles which can be used
on guest creation instead of manually needing to specify every option.

we do special things here:
* we always set 'allow_unknown' to 1, because when using the guest
  specific parts in the cli, we cannot depend on the other one, else
  we'd get a cyclic dependency, and without that we need to ignore
  unknown sections

* we pack the type and id in the global options, so that when using
  it with a seperated section config they get included in the
  create/updateSchema

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/Makefile               |  2 ++
 src/PVE/Profiles/Plugin.pm | 74 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+)
 create mode 100644 src/PVE/Profiles/Plugin.pm

diff --git a/src/Makefile b/src/Makefile
index cbc40c1..d99645c 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -17,6 +17,8 @@ install: PVE
 	install -d ${PERL5DIR}/PVE/Mapping
 	install -m 0644 PVE/Mapping/PCI.pm ${PERL5DIR}/PVE/Mapping/
 	install -m 0644 PVE/Mapping/USB.pm ${PERL5DIR}/PVE/Mapping/
+	install -d ${PERL5DIR}/PVE/Profiles
+	install -m 0644 PVE/Profiles/Plugin.pm ${PERL5DIR}/PVE/Profiles/
 	install -d ${PERL5DIR}/PVE/VZDump
 	install -m 0644 PVE/VZDump/Plugin.pm ${PERL5DIR}/PVE/VZDump/
 	install -m 0644 PVE/VZDump/Common.pm ${PERL5DIR}/PVE/VZDump/
diff --git a/src/PVE/Profiles/Plugin.pm b/src/PVE/Profiles/Plugin.pm
new file mode 100644
index 0000000..6ea2b5f
--- /dev/null
+++ b/src/PVE/Profiles/Plugin.pm
@@ -0,0 +1,74 @@
+package PVE::Profiles::Plugin;
+
+use strict;
+use warnings;
+
+use PVE::Cluster qw(cfs_register_file);
+use PVE::SectionConfig;
+
+use base qw(PVE::SectionConfig);
+
+my $CFG_PATH = 'virtual-guest/profiles.cfg';
+
+cfs_register_file ($CFG_PATH,
+		   sub { __PACKAGE__->parse_config(@_); },
+		   sub { __PACKAGE__->write_config(@_); });
+
+my $defaultData = {
+    propertyList => {
+	type => { description => 'Profile type.' },
+	id => {
+	    type => 'string',
+	    description => "The ID of the profile.",
+	    format => 'pve-configid',
+	},
+	'profile-description' => {
+	    description => "Description.",
+	    type => 'string',
+	    optional => 1,
+	    maxLength => 4096,
+	},
+    },
+    options => {
+	type => {},
+	id => {},
+	'profile-description' => { optional => 1 },
+    },
+};
+
+sub private {
+    return $defaultData;
+}
+
+sub parse_config {
+    my ($class, $filename, $raw, $allow_unknown) = @_;
+
+    # always allow unknown, so that qemu-server/pct-container
+    # can parse the file without loading the other plugin type
+    return $class->SUPER::parse_config($filename, $raw, 1);
+}
+
+sub write_config {
+    my ($class, $filename, $cfg, $allow_unknown) = @_;
+
+    return $class->SUPER::write_config($filename, $cfg, 1);
+}
+
+# gets, checks and prepares the guest config
+# throws an error if it does not exist or the type is wrong
+sub get_guest_ready_config {
+    my ($id, $type) = @_;
+
+    my $cfg = PVE::Cluster::cfs_read_file($CFG_PATH);
+
+    my $profile = $cfg->{ids}->{$id};
+    die "no such profile '$id'\n" if !defined $profile;
+    die "wrong type '$profile->{type}'\n" if $profile->{type} ne $type;
+
+    delete $profile->{type};
+    delete $profile->{'profile-description'};
+
+    return $profile;
+}
+
+1;
-- 
2.30.2

[pve-devel] [PATCH qemu-server v2 1/3] add the VM profiles plugin

[Dominik Csapak]

simply uses the json_config_properties for the vm config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* no prefixing with vm anymore

 PVE/Makefile          |  1 +
 PVE/Profiles/Makefile |  5 +++++
 PVE/Profiles/VM.pm    | 28 ++++++++++++++++++++++++++++
 3 files changed, 34 insertions(+)
 create mode 100644 PVE/Profiles/Makefile
 create mode 100644 PVE/Profiles/VM.pm

diff --git a/PVE/Makefile b/PVE/Makefile
index dc173681..d09ca98d 100644
--- a/PVE/Makefile
+++ b/PVE/Makefile
@@ -12,3 +12,4 @@ install:
 	$(MAKE) -C API2 install
 	$(MAKE) -C CLI install
 	$(MAKE) -C QemuServer install
+	$(MAKE) -C Profiles install
diff --git a/PVE/Profiles/Makefile b/PVE/Profiles/Makefile
new file mode 100644
index 00000000..e5f56833
--- /dev/null
+++ b/PVE/Profiles/Makefile
@@ -0,0 +1,5 @@
+SOURCES=VM.pm
+
+.PHONY: install
+install: ${SOURCES}
+	for i in ${SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/Profiles/$$i; done
diff --git a/PVE/Profiles/VM.pm b/PVE/Profiles/VM.pm
new file mode 100644
index 00000000..dc664ec5
--- /dev/null
+++ b/PVE/Profiles/VM.pm
@@ -0,0 +1,28 @@
+package PVE::Profiles::VM;
+
+use strict;
+use warnings;
+
+use PVE::Profiles::Plugin;
+use PVE::QemuServer;
+
+use base qw(PVE::Profiles::Plugin);
+
+sub type {
+    return "vm";
+}
+
+sub properties {
+    return PVE::QemuServer::json_config_properties();
+}
+
+sub options {
+    my $props = PVE::QemuServer::json_config_properties();
+    my $opts = {};
+    for my $opt (keys $props->%*) {
+	$opts->{$opt} = { optional => 1 };
+    }
+    return $opts;
+}
+
+1;
-- 
2.30.2

[pve-devel] [PATCH qemu-server v2 2/3] api: add profile option to create vm api call

[Dominik Csapak]

we use the the profile cfg as the 'param' hash, but overwrite the values
with the ones from the api call, so one can overwrite options from the
profile easily

also we add the used profile to the log, since
it might be interesting which one was used

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* log profile instead of putting it into the meta object
* use helper from Plugin class
* use /mapping/guest-profile as acl path now

 PVE/API2/Qemu.pm  | 23 +++++++++++++++++++++++
 PVE/QemuServer.pm |  6 ++++++
 2 files changed, 29 insertions(+)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 38bdaabd..0d8bbdea 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -49,6 +49,9 @@ use PVE::SSHInfo;
 use PVE::Replication;
 use PVE::StorageTunnel;
 
+use PVE::Profiles::Plugin;
+use PVE::Profiles::VM;
+
 BEGIN {
     if (!$ENV{PVE_GENERATING_DOCS}) {
 	require PVE::HA::Env::PVE2;
@@ -837,6 +840,11 @@ __PACKAGE__->register_method({
 		    default => 0,
 		    description => "Start VM after it was created successfully.",
 		},
+		profile => {
+		    optional => 1,
+		    type => 'string',
+		    description => "The profile to use as base config.",
+		},
 	    },
 	    1, # with_disk_alloc
 	),
@@ -850,6 +858,19 @@ __PACKAGE__->register_method({
 	my $rpcenv = PVE::RPCEnvironment::get();
 	my $authuser = $rpcenv->get_user();
 
+	my $profile = extract_param($param, 'profile');
+	if (defined($profile)) {
+	    $rpcenv->check_full($authuser, "/mapping/guest-profile/${profile}", ['Mapping.Use']);
+	    my $profile_cfg = eval { PVE::Profiles::Plugin::get_guest_ready_config($profile, 'vm') };
+	    raise_param_exc({ profile => "$@"}) if $@;
+
+	    for my $opt (keys $param->%*) {
+		$profile_cfg->{$opt} = $param->{$opt};
+	    }
+
+	    $param = $profile_cfg;
+	}
+
 	my $node = extract_param($param, 'node');
 	my $vmid = extract_param($param, 'vmid');
 
@@ -1013,6 +1034,8 @@ __PACKAGE__->register_method({
 		my $conf = $param;
 		my $arch = PVE::QemuServer::get_vm_arch($conf);
 
+		print "using profile '$profile'\n" if $profile;
+
 		$conf->{meta} = PVE::QemuServer::new_meta_info_string();
 
 		my $vollist = [];
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index c465fb6f..1c7e65ea 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -293,6 +293,12 @@ my $meta_info_fmt = {
 	pattern => '\d+(\.\d+)+',
 	optional => 1,
     },
+    'profile' => {
+	type => 'string',
+	description => 'The Profile used during creation.',
+	format => 'pve-configid',
+	optional => 1,
+    },
 };
 
 my $confdesc = {
-- 
2.30.2

[pve-devel] [PATCH qemu-server v2 3/3] qm: register and init the profiles plugins

[Dominik Csapak]

we have to that here, so the properties/options are correctly configured
when using that feature on the cli

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* use init(1) for separated plugin schemas
 PVE/CLI/qm.pm | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/PVE/CLI/qm.pm b/PVE/CLI/qm.pm
index b17b4fe2..c52c972d 100755
--- a/PVE/CLI/qm.pm
+++ b/PVE/CLI/qm.pm
@@ -37,6 +37,12 @@ use PVE::QemuServer::Monitor qw(mon_cmd);
 use PVE::QemuServer::OVF;
 use PVE::QemuServer;
 
+use PVE::Profiles::Plugin;
+use PVE::Profiles::VM;
+
+PVE::Profiles::VM->register();
+PVE::Profiles::Plugin->init(1);
+
 use PVE::CLIHandler;
 use base qw(PVE::CLIHandler);
 
-- 
2.30.2

[pve-devel] [PATCH container v2 1/3] add the CT profiles plugin

[Dominik Csapak]

simply uses the json_config_properties for the ct config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* no prefixing with ct_ anymore

 src/PVE/Makefile          |  1 +
 src/PVE/Profiles/CT.pm    | 28 ++++++++++++++++++++++++++++
 src/PVE/Profiles/Makefile |  4 ++++
 3 files changed, 33 insertions(+)
 create mode 100644 src/PVE/Profiles/CT.pm
 create mode 100644 src/PVE/Profiles/Makefile

diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index 40742e4..e9ad9a3 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -8,5 +8,6 @@ install: ${SOURCES}
 	make -C LXC install
 	make -C VZDump install
 	make -C CLI install
+	make -C Profiles install
 
 
diff --git a/src/PVE/Profiles/CT.pm b/src/PVE/Profiles/CT.pm
new file mode 100644
index 0000000..513fad4
--- /dev/null
+++ b/src/PVE/Profiles/CT.pm
@@ -0,0 +1,28 @@
+package PVE::Profiles::CT;
+
+use strict;
+use warnings;
+
+use PVE::Profiles::Plugin;
+use PVE::LXC::Config;
+
+use base qw(PVE::Profiles::Plugin);
+
+sub type {
+    return "ct";
+}
+
+sub properties {
+    return PVE::LXC::Config::json_config_properties();
+}
+
+sub options {
+    my $props = PVE::LXC::Config::json_config_properties();
+    my $opts = {};
+    for my $opt (keys $props->%*) {
+	$opts->{$opt} = { optional => 1 };
+    }
+    return $opts;
+}
+
+1;
diff --git a/src/PVE/Profiles/Makefile b/src/PVE/Profiles/Makefile
new file mode 100644
index 0000000..e63a9f2
--- /dev/null
+++ b/src/PVE/Profiles/Makefile
@@ -0,0 +1,4 @@
+
+.PHONY: install
+install:
+	install -D -m 0644 CT.pm ${PERLDIR}/PVE/Profiles/CT.pm
-- 
2.30.2

[pve-devel] [PATCH container v2 2/3] api: add profile option to create ct api call

[Dominik Csapak]

we use the profile cfg as the 'param' hash, but overwrite the values
with the ones from the api call, so one can overwrite options from
the profile easily

we also log the used profile

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* use helper from Plugin
* log profile when used
* use /mapping/guest-profile as acl path now

 src/PVE/API2/LXC.pm | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 28d14de..2b17f8f 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -27,6 +27,10 @@ use PVE::API2::LXC::Config;
 use PVE::API2::LXC::Status;
 use PVE::API2::LXC::Snapshot;
 use PVE::JSONSchema qw(get_standard_option);
+
+use PVE::Profiles::Plugin;
+use PVE::Profiles::CT;
+
 use base qw(PVE::RESTHandler);
 
 BEGIN {
@@ -196,6 +200,11 @@ __PACKAGE__->register_method({
 		default => 0,
 		description => "Start the CT after its creation finished successfully.",
 	    },
+	    profile => {
+		optional => 1,
+		type => 'string',
+		description => "The profile to use as base config.",
+	    },
 	}),
     },
     returns => {
@@ -209,6 +218,19 @@ __PACKAGE__->register_method({
 	my $rpcenv = PVE::RPCEnvironment::get();
 	my $authuser = $rpcenv->get_user();
 
+	my $profile = extract_param($param, 'profile');
+	if (defined($profile)) {
+	    $rpcenv->check_full($authuser, "/mapping/guest-profile/${profile}", ['Mapping.Use']);
+	    my $profile_cfg = eval { PVE::Profiles::Plugin::get_guest_ready_config($profile, 'ct') };
+	    raise_param_exc({ profile => "$@" }) if $@;
+
+	    for my $opt (keys $param->%*) {
+		$profile_cfg->{$opt} = $param->{$opt};
+	    }
+
+	    $param = $profile_cfg;
+	}
+
 	my $node = extract_param($param, 'node');
 	my $vmid = extract_param($param, 'vmid');
 	my $ignore_unpack_errors = extract_param($param, 'ignore-unpack-errors');
@@ -381,6 +403,7 @@ __PACKAGE__->register_method({
 	    my $vollist = [];
 	    eval {
 		my $orig_mp_param; # only used if $restore
+		print "using profile '$profile'\n" if $profile;
 		if ($restore) {
 		    die "can't overwrite running container\n" if PVE::LXC::check_running($vmid);
 		    if ($archive ne '-') {
-- 
2.30.2

[pve-devel] [PATCH container v2 3/3] pct: register and init the profiles plugins

[Dominik Csapak]

we have to that here, so the properties/options are correctly configured
when using that feature on the cli

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* use init(1) for separated plugins
 src/PVE/CLI/pct.pm | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/src/PVE/CLI/pct.pm b/src/PVE/CLI/pct.pm
index a0b9bce..a2a08c2 100755
--- a/src/PVE/CLI/pct.pm
+++ b/src/PVE/CLI/pct.pm
@@ -24,6 +24,12 @@ use PVE::API2::LXC::Snapshot;
 use PVE::API2::LXC::Status;
 use PVE::API2::LXC;
 
+use PVE::Profiles::Plugin;
+use PVE::Profiles::CT;
+
+PVE::Profiles::CT->register();
+PVE::Profiles::Plugin->init(1);
+
 use base qw(PVE::CLIHandler);
 
 my $nodename = PVE::INotify::nodename();
-- 
2.30.2

[pve-devel] [PATCH manager v2 1/1] api: add guest profile api endpoint

[Dominik Csapak]

basic CRUD for the profile section config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* use raise_param_exc when id or type is wrong
* extract the type from param
* use /mapping/guest-profile as acl path
* add missing index entry for profiles

 PVE/API2/Cluster.pm          |   7 ++
 PVE/API2/Cluster/Makefile    |   1 +
 PVE/API2/Cluster/Profiles.pm | 230 +++++++++++++++++++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 PVE/API2/Cluster/Profiles.pm

diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index 04387ab4..d628df85 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -30,6 +30,7 @@ use PVE::API2::Cluster::Mapping;
 use PVE::API2::Cluster::Jobs;
 use PVE::API2::Cluster::MetricServer;
 use PVE::API2::Cluster::Notifications;
+use PVE::API2::Cluster::Profiles;
 use PVE::API2::ClusterConfig;
 use PVE::API2::Firewall::Cluster;
 use PVE::API2::HAConfig;
@@ -103,6 +104,11 @@ __PACKAGE__->register_method ({
     path => 'mapping',
 });
 
+__PACKAGE__->register_method ({
+    subclass => "PVE::API2::Cluster::Profiles",
+    path => 'profiles',
+});
+
 if ($have_sdn) {
     __PACKAGE__->register_method ({
        subclass => "PVE::API2::Network::SDN",
@@ -158,6 +164,7 @@ __PACKAGE__->register_method ({
 	    { name => 'notifications' },
 	    { name => 'nextid' },
 	    { name => 'options' },
+	    { name => 'profiles' },
 	    { name => 'replication' },
 	    { name => 'resources' },
 	    { name => 'status' },
diff --git a/PVE/API2/Cluster/Makefile b/PVE/API2/Cluster/Makefile
index b109e5cb..35a3f871 100644
--- a/PVE/API2/Cluster/Makefile
+++ b/PVE/API2/Cluster/Makefile
@@ -9,6 +9,7 @@ PERLSOURCE= 			\
 	MetricServer.pm		\
 	Mapping.pm		\
 	Notifications.pm		\
+	Profiles.pm		\
 	Jobs.pm			\
 	Ceph.pm
 
diff --git a/PVE/API2/Cluster/Profiles.pm b/PVE/API2/Cluster/Profiles.pm
new file mode 100644
index 00000000..3d1ec67c
--- /dev/null
+++ b/PVE/API2/Cluster/Profiles.pm
@@ -0,0 +1,230 @@
+package PVE::API2::Cluster::Profiles;
+
+use warnings;
+use strict;
+
+use PVE::Tools qw(extract_param extract_sensitive_params);
+use PVE::Exception qw(raise_perm_exc raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::RPCEnvironment;
+
+use PVE::Profiles::Plugin;
+use PVE::Profiles::VM;
+use PVE::Profiles::CT;
+
+PVE::Profiles::VM->register();
+PVE::Profiles::CT->register();
+PVE::Profiles::Plugin->init(1);
+
+use PVE::RESTHandler;
+
+use base qw(PVE::RESTHandler);
+
+__PACKAGE__->register_method ({
+    name => 'profile_index',
+    path => '',
+    method => 'GET',
+    description => "List configured guest profiles.",
+    permissions => {
+	user => 'all',
+	description => "Only lists entries where you have 'Mapping.Modify', 'Mapping.Use' or".
+	    " 'Mapping.Audit' permissions on 'mapping/guest-profile/<id>'.",
+    },
+    parameters => {
+	additionalProperties => 0,
+	properties => {},
+    },
+    returns => {
+	type => 'array',
+	items => {
+	    type => "object",
+	    properties => {
+		id => {
+		    description => "The ID of the entry.",
+		    type => 'string'
+		},
+		type => {
+		    description => "Plugin type.",
+		    type => 'string',
+		},
+	    },
+	},
+	links => [ { rel => 'child', href => "{id}" } ],
+    },
+    code => sub {
+	my ($param) = @_;
+
+	my $rpcenv = PVE::RPCEnvironment::get();
+	my $authuser = $rpcenv->get_user();
+
+	my $res = [];
+	my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+	my $can_see_mapping_privs = ['Mapping.Modify', 'Mapping.Use', 'Mapping.Audit'];
+
+	for my $id (sort keys $cfg->{ids}->%*) {
+	    next if !$rpcenv->check_any($authuser, "/mapping/guest-profile/$id", $can_see_mapping_privs, 1);
+	    my $plugin_config = $cfg->{ids}->{$id};
+	    push @$res, {
+		id => $id,
+		type => $plugin_config->{type},
+	    };
+	}
+
+	return $res;
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'read',
+    path => '{id}',
+    method => 'GET',
+    description => "Read profile configuration.",
+    permissions => {
+	check =>['or',
+	    ['perm', '/mapping/guest-profile/{id}', ['Mapping.Use']],
+	    ['perm', '/mapping/guest-profile/{id}', ['Mapping.Modify']],
+	    ['perm', '/mapping/guest-profile/{id}', ['Mapping.Audit']],
+	],
+    },
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    id => {
+		type => 'string',
+		format => 'pve-configid',
+	    },
+	},
+    },
+    returns => { type => 'object' },
+    code => sub {
+	my ($param) = @_;
+
+	my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+	my $id = $param->{id};
+
+	raise_param_exc({id => "no such profile '$id'"}) if !defined($cfg->{ids}->{$id});
+
+	return $cfg->{ids}->{$id};
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'create',
+    path => '{id}',
+    protected => 1,
+    method => 'POST',
+    description => "Create a new profile.",
+    permissions => {
+	check => ['perm', '/mapping/guest-profile', ['Mapping.Modify']],
+    },
+    parameters => PVE::Profiles::Plugin->createSchema(),
+    returns => { type => 'null' },
+    code => sub {
+	my ($param) = @_;
+
+	my $type = extract_param($param, 'type');
+	my $plugin = PVE::Profiles::Plugin->lookup($type);
+	my $id = extract_param($param, 'id');
+
+	PVE::Cluster::cfs_lock_file('virtual-guest/profiles.cfg', undef, sub {
+	    my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+
+	    raise_param_exc({id => "Profile '$id' already exists"})
+		if $cfg->{ids}->{$id};
+
+	    my $opts = $plugin->check_config($id, $param, 1, 1);
+
+	    $cfg->{ids}->{$id} = $opts;
+
+	    PVE::Cluster::cfs_write_file('virtual-guest/profiles.cfg', $cfg);
+	});
+	die $@ if $@;
+
+	return;
+    }});
+
+
+__PACKAGE__->register_method ({
+    name => 'update',
+    protected => 1,
+    path => '{id}',
+    method => 'PUT',
+    description => "Update profile configuration.",
+    permissions => {
+	check => ['perm', '/mapping/guest-profile/{id}', ['Mapping.Modify']],
+    },
+    parameters => PVE::Profiles::Plugin->updateSchema(),
+    returns => { type => 'null' },
+    code => sub {
+	my ($param) = @_;
+
+	my $id = extract_param($param, 'id');
+	my $type = extract_param($param, 'type');
+	my $digest = extract_param($param, 'digest');
+	my $delete = extract_param($param, 'delete');
+
+	if ($delete) {
+	    $delete = [PVE::Tools::split_list($delete)];
+	}
+
+	PVE::Cluster::cfs_lock_file('virtual-guest/profiles.cfg', undef, sub {
+	    my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+
+	    PVE::SectionConfig::assert_if_modified($cfg, $digest);
+
+	    my $data = $cfg->{ids}->{$id};
+	    raise_param_exc({id => "no such profile '$id'"}) if !defined($data);
+	    raise_param_exc({type => "wrong type '$type"}) if $type ne $data->{type};
+
+	    my $plugin = PVE::Profiles::Plugin->lookup($data->{type});
+	    my $opts = $plugin->check_config($id, $param, 0, 1);
+
+	    my $options = $plugin->private()->{options}->{$data->{type}};
+	    PVE::SectionConfig::delete_from_config($data, $options, $opts, $delete);
+
+	    $data->{$_} = $opts->{$_} for keys $opts->%*;
+
+	    PVE::Cluster::cfs_write_file('virtual-guest/profiles.cfg', $cfg);
+	});
+	die $@ if $@;
+
+	return;
+    }});
+
+__PACKAGE__->register_method ({
+    name => 'delete',
+    protected => 1,
+    path => '{id}',
+    method => 'DELETE',
+    description => "Remove profile.",
+    permissions => {
+	check => [ 'perm', '/mapping/guest-profile', ['Mapping.Modify']],
+    },
+    parameters => {
+	additionalProperties => 0,
+	properties => {
+	    id => {
+		type => 'string',
+		format => 'pve-configid',
+	    },
+	}
+    },
+    returns => { type => 'null' },
+    code => sub {
+	my ($param) = @_;
+
+	my $id = $param->{id};
+
+	PVE::Cluster::cfs_lock_file('virtual-guest/profiles.cfg', undef, sub {
+	    my $cfg = PVE::Cluster::cfs_read_file('virtual-guest/profiles.cfg');
+
+	    if ($cfg->{ids}->{$id}) {
+		delete $cfg->{ids}->{$id};
+	    }
+
+	    PVE::Cluster::cfs_write_file('virtual-guest/profiles.cfg', $cfg);
+	});
+	die $@ if $@;
+
+	return;
+    }});
+
+1;
-- 
2.30.2

Re: [pve-devel] [PATCH guest-common v2 1/1] add profiles section config plugin

[Thomas Lamprecht]

Am 14/11/2023 um 11:35 schrieb Dominik Csapak:
> this is intended to house custom profiles which can be used
> on guest creation instead of manually needing to specify every option.
> 
> we do special things here:
> * we always set 'allow_unknown' to 1, because when using the guest
>   specific parts in the cli, we cannot depend on the other one, else
>   we'd get a cyclic dependency, and without that we need to ignore
>   unknown sections
> 
> * we pack the type and id in the global options, so that when using
>   it with a seperated section config they get included in the
>   create/updateSchema
> 
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
>  src/Makefile               |  2 ++
>  src/PVE/Profiles/Plugin.pm | 74 ++++++++++++++++++++++++++++++++++++++
>  2 files changed, 76 insertions(+)
>  create mode 100644 src/PVE/Profiles/Plugin.pm
> 
> diff --git a/src/Makefile b/src/Makefile
> index cbc40c1..d99645c 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -17,6 +17,8 @@ install: PVE
>  	install -d ${PERL5DIR}/PVE/Mapping
>  	install -m 0644 PVE/Mapping/PCI.pm ${PERL5DIR}/PVE/Mapping/
>  	install -m 0644 PVE/Mapping/USB.pm ${PERL5DIR}/PVE/Mapping/
> +	install -d ${PERL5DIR}/PVE/Profiles
> +	install -m 0644 PVE/Profiles/Plugin.pm ${PERL5DIR}/PVE/Profiles/
>  	install -d ${PERL5DIR}/PVE/VZDump
>  	install -m 0644 PVE/VZDump/Plugin.pm ${PERL5DIR}/PVE/VZDump/
>  	install -m 0644 PVE/VZDump/Common.pm ${PERL5DIR}/PVE/VZDump/
> diff --git a/src/PVE/Profiles/Plugin.pm b/src/PVE/Profiles/Plugin.pm
> new file mode 100644
> index 0000000..6ea2b5f
> --- /dev/null
> +++ b/src/PVE/Profiles/Plugin.pm
> @@ -0,0 +1,74 @@
> +package PVE::Profiles::Plugin;
> +
> +use strict;
> +use warnings;
> +
> +use PVE::Cluster qw(cfs_register_file);
> +use PVE::SectionConfig;
> +
> +use base qw(PVE::SectionConfig);
> +
> +my $CFG_PATH = 'virtual-guest/profiles.cfg';
> +
> +cfs_register_file ($CFG_PATH,

extra space before opening parenthesis.

> +		   sub { __PACKAGE__->parse_config(@_); },
> +		   sub { __PACKAGE__->write_config(@_); });

let's use some more modern indentation here

cfs_register_file(
    $CFG_PATH,
    sub { __PACKAGE__->parse_config(@_); },
    sub { __PACKAGE__->write_config(@_); },
);

> +
> +my $defaultData = {
> +    propertyList => {
> +	type => { description => 'Profile type.' },
> +	id => {
> +	    type => 'string',
> +	    description => "The ID of the profile.",
> +	    format => 'pve-configid',
> +	},
> +	'profile-description' => {
> +	    description => "Description.",

I mean, yes, but a sentence is still nice:

description => "Use this to add a short comment about a profile.",

> +	    type => 'string',
> +	    optional => 1,
> +	    maxLength => 4096,

would start out with a bit less, as this is rendered in grids and the like
and doesn't needs to hold that many information than e.g., node or guest
instances might have.

> +	},
> +    },
> +    options => {
> +	type => {},
> +	id => {},
> +	'profile-description' => { optional => 1 },
> +    },
> +};
> +
> +sub private {
> +    return $defaultData;
> +}
> +
> +sub parse_config {
> +    my ($class, $filename, $raw, $allow_unknown) = @_;
> +
> +    # always allow unknown, so that qemu-server/pct-container

to make it slightly easier to understand it'd IMO help to:
s/unknown/unknown section types/

> +    # can parse the file without loading the other plugin type
> +    return $class->SUPER::parse_config($filename, $raw, 1);
> +}
> +
> +sub write_config {
> +    my ($class, $filename, $cfg, $allow_unknown) = @_;
> +
> +    return $class->SUPER::write_config($filename, $cfg, 1);
> +}
> +
> +# gets, checks and prepares the guest config
> +# throws an error if it does not exist or the type is wrong
> +sub get_guest_ready_config {

This is rather covoluted name for what happens, wouldn't naming this
`load_profile` be clear enough? Or are there future changes, or overrides
in child-implementations that warrant this name (even then we probably
find something better)

> +    my ($id, $type) = @_;

Otherwise I'd use at least the following variable names to clarify what
$id's we're talking about, because from the method name I initially thought
those are vmid's (was cleared up quickly when continue reading the code, but
IMO still not ideal as is)

my ($profile_id, $profile_type) = @_;


> +
> +    my $cfg = PVE::Cluster::cfs_read_file($CFG_PATH);
> +
> +    my $profile = $cfg->{ids}->{$id};
> +    die "no such profile '$id'\n" if !defined $profile;
> +    die "wrong type '$profile->{type}'\n" if $profile->{type} ne $type;
> +
> +    delete $profile->{type};
> +    delete $profile->{'profile-description'};
> +
> +    return $profile;
> +}
> +
> +1;

Re: [pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support

[Thomas Lamprecht]

Am 14/11/2023 um 11:35 schrieb Dominik Csapak:
> This series aims to provide profile support when creating guests (ct/vm)
> so that users can reuse options without having to specify them every
> time.

Great! Nice to see that my hopes^Wexpectations about this being really
not that much work/code are being met.

As mentioned off-list, some way for configuring the base defaults for
disks/mountpoints and networks without having to already explicitly
specify the single would be nice to have.

As talked, having a net-defaults, scsi-defaults, mp-defaults, ...
property supported by the respective VM/CT config itself could solve
that and even provide an additional feature.
The values decoded there would be merged with the actual instances,
e.g., scsi0, scsi1, ... or net0, net1, ..., on start.
There might be a few edge cases, and me might not want to support
every option (e.g., a default serial for all disks is rather counter
productive), but otherwise this could be a nice way to encode things
like "discard=on,sdd=1" for all (future) scsi disks, be it for a
specific VM or CT, or also in a profile.

[pve-devel] [PATCH cluster/guest-common/qemu-server/container/manager v2] add backend profile support

[Dominik Csapak]

This series aims to provide profile support when creating guests (ct/vm)
so that users can reuse options without having to specify them every
time.

UI is still to follow, depends on [0] for the oneOf schema support.

changes from v2:
* rename get_guest_ready_config into load_profile
* rename some variables
* add type parameter to profile listing so we can filter
* whitespace fixes
* comment fixes

changes from rfc/v1:
* uses a section config with separated plugins, using the oneOf schema
* move the parsing with preparing into a helper into the Plugin class
* don't save the profile into the 'meta' option, but log it instead

0: https://lists.proxmox.com/pipermail/pve-devel/2023-November/060300.html

pve-cluster:

Dominik Csapak (1):
  add profiles.cfg to cluster fs

 src/PVE/Cluster.pm  | 1 +
 src/pmxcfs/status.c | 1 +
 2 files changed, 2 insertions(+)

pve-guest-common:

Dominik Csapak (1):
  add profiles section config plugin

 src/Makefile               |  2 +
 src/PVE/Profiles/Plugin.pm | 83 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+)
 create mode 100644 src/PVE/Profiles/Plugin.pm

qemu-server:

Dominik Csapak (3):
  add the VM profiles plugin
  api: add profile option to create vm api call
  qm: register and init the profiles plugins

 PVE/API2/Qemu.pm      | 23 +++++++++++++++++++++++
 PVE/CLI/qm.pm         |  6 ++++++
 PVE/Makefile          |  1 +
 PVE/Profiles/Makefile |  5 +++++
 PVE/Profiles/VM.pm    | 28 ++++++++++++++++++++++++++++
 PVE/QemuServer.pm     |  6 ++++++
 6 files changed, 69 insertions(+)
 create mode 100644 PVE/Profiles/Makefile
 create mode 100644 PVE/Profiles/VM.pm

pve-container:

Dominik Csapak (3):
  add the CT profiles plugin
  api: add profile option to create ct api call
  pct: register and init the profiles plugins

 src/PVE/API2/LXC.pm       | 23 +++++++++++++++++++++++
 src/PVE/CLI/pct.pm        |  6 ++++++
 src/PVE/Makefile          |  1 +
 src/PVE/Profiles/CT.pm    | 28 ++++++++++++++++++++++++++++
 src/PVE/Profiles/Makefile |  4 ++++
 5 files changed, 62 insertions(+)
 create mode 100644 src/PVE/Profiles/CT.pm
 create mode 100644 src/PVE/Profiles/Makefile

pve-manager:

Dominik Csapak (1):
  api: add guest profile api endpoint

 PVE/API2/Cluster.pm          |   7 +
 PVE/API2/Cluster/Makefile    |   1 +
 PVE/API2/Cluster/Profiles.pm | 239 +++++++++++++++++++++++++++++++++++
 3 files changed, 247 insertions(+)
 create mode 100644 PVE/API2/Cluster/Profiles.pm

-- 
2.30.2