* [pve-devel] [PATCH common v3 1/1] SectionConfig: add helper to delete keys from a section config entry
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
@ 2023-01-17 11:46 ` Dominik Csapak
2023-03-08 6:53 ` Thomas Lamprecht
2023-01-17 11:46 ` [pve-devel] [PATCH access-control v3 1/2] realm sync: refactor scope/remove-vanished into a standard option Dominik Csapak
` (9 subsequent siblings)
10 siblings, 1 reply; 17+ messages in thread
From: Dominik Csapak @ 2023-01-17 11:46 UTC (permalink / raw)
To: pve-devel
this is a pattern we have very often, so we can implement that here and
reuse it
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/PVE/SectionConfig.pm | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/src/PVE/SectionConfig.pm b/src/PVE/SectionConfig.pm
index 0a9f1cd..e354655 100644
--- a/src/PVE/SectionConfig.pm
+++ b/src/PVE/SectionConfig.pm
@@ -543,4 +543,19 @@ sub assert_if_modified {
PVE::Tools::assert_if_modified($cfg->{digest}, $digest);
}
+sub delete_from_config {
+ my ($config, $option_schema, $new_options, $to_delete) = @_;
+
+ for my $k ($to_delete->@*) {
+ my $d = $option_schema->{$k} || die "no such option '$k'\n";
+ die "unable to delete required option '$k'\n" if !$d->{optional};
+ die "unable to delete fixed option '$k'\n" if $d->{fixed};
+ die "cannot set and delete property '$k' at the same time!\n"
+ if defined($new_options->{$k});
+ delete $config->{$k};
+ }
+
+ return $config;
+}
+
1;
--
2.30.2
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [pve-devel] [PATCH common v3 1/1] SectionConfig: add helper to delete keys from a section config entry
2023-01-17 11:46 ` [pve-devel] [PATCH common v3 1/1] SectionConfig: add helper to delete keys from a section config entry Dominik Csapak
@ 2023-03-08 6:53 ` Thomas Lamprecht
2023-03-11 17:23 ` [pve-devel] applied: " Thomas Lamprecht
0 siblings, 1 reply; 17+ messages in thread
From: Thomas Lamprecht @ 2023-03-08 6:53 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
Am 17/01/2023 um 12:46 schrieb Dominik Csapak:
> this is a pattern we have very often, so we can implement that here and
> reuse it
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> src/PVE/SectionConfig.pm | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/src/PVE/SectionConfig.pm b/src/PVE/SectionConfig.pm
> index 0a9f1cd..e354655 100644
> --- a/src/PVE/SectionConfig.pm
> +++ b/src/PVE/SectionConfig.pm
> @@ -543,4 +543,19 @@ sub assert_if_modified {
> PVE::Tools::assert_if_modified($cfg->{digest}, $digest);
> }
>
> +sub delete_from_config {
> + my ($config, $option_schema, $new_options, $to_delete) = @_;
> +
> + for my $k ($to_delete->@*) {
> + my $d = $option_schema->{$k} || die "no such option '$k'\n";
> + die "unable to delete required option '$k'\n" if !$d->{optional};
Should we use the (here already imported but not used) raise_param_exc ?
As that would then also make the magic "mark-field-as-invalid" work if
used in API calls?
FWIW, we could use a cummulative approach, e.g. something like like:
my $errors = {};
for ... {
eval {
die "unable to delete fixed option\n" if $d->{fixed};
...
};
if (my $err = $@) {
$errors->{$k} = $err;
} else {
delete ...
}
}
raise_param_exc($errors) if scalar($errors->%*);
eval + die could be replaced by $err string in the loop for accumulating even
more errors, not sure how relevant in practice though.
Anyhow, method signature wouldn't change, so we could always do this as
followup too.
> + die "unable to delete fixed option '$k'\n" if $d->{fixed};
> + die "cannot set and delete property '$k' at the same time!\n"
> + if defined($new_options->{$k});
> + delete $config->{$k};
> + }
> +
> + return $config;
> +}
> +
> 1;
^ permalink raw reply [flat|nested] 17+ messages in thread* [pve-devel] applied: [PATCH common v3 1/1] SectionConfig: add helper to delete keys from a section config entry
2023-03-08 6:53 ` Thomas Lamprecht
@ 2023-03-11 17:23 ` Thomas Lamprecht
0 siblings, 0 replies; 17+ messages in thread
From: Thomas Lamprecht @ 2023-03-11 17:23 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
Am 08/03/2023 um 07:53 schrieb Thomas Lamprecht:
> Am 17/01/2023 um 12:46 schrieb Dominik Csapak:
>> this is a pattern we have very often, so we can implement that here and
>> reuse it
>>
>> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
>> ---
>> src/PVE/SectionConfig.pm | 15 +++++++++++++++
>> 1 file changed, 15 insertions(+)
>>
>> diff --git a/src/PVE/SectionConfig.pm b/src/PVE/SectionConfig.pm
>> index 0a9f1cd..e354655 100644
>> --- a/src/PVE/SectionConfig.pm
>> +++ b/src/PVE/SectionConfig.pm
>> @@ -543,4 +543,19 @@ sub assert_if_modified {
>> PVE::Tools::assert_if_modified($cfg->{digest}, $digest);
>> }
>>
>> +sub delete_from_config {
>> + my ($config, $option_schema, $new_options, $to_delete) = @_;
>> +
>> + for my $k ($to_delete->@*) {
>> + my $d = $option_schema->{$k} || die "no such option '$k'\n";
>> + die "unable to delete required option '$k'\n" if !$d->{optional};
>
> Should we use the (here already imported but not used) raise_param_exc ?
Well, without any reply here I'll still apply it for now, as said signature
change should not be required anyway.
> As that would then also make the magic "mark-field-as-invalid" work if
> used in API calls?
>
> FWIW, we could use a cumulative approach, e.g. something like like:
>
> my $errors = {};
>
> for ... {
> eval {
> die "unable to delete fixed option\n" if $d->{fixed};
> ...
> };
> if (my $err = $@) {
> $errors->{$k} = $err;
> } else {
> delete ...
> }
> }
>
> raise_param_exc($errors) if scalar($errors->%*);
I actually thought shortly that if, this would need to be:
raise_param_exc({ delete => $joined_error_string });
As deletes normally come in via the 'delete' key, but from top of my head I don't
know if we ever use our field invalidation magic for deletes..
^ permalink raw reply [flat|nested] 17+ messages in thread
* [pve-devel] [PATCH access-control v3 1/2] realm sync: refactor scope/remove-vanished into a standard option
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
2023-01-17 11:46 ` [pve-devel] [PATCH common v3 1/1] SectionConfig: add helper to delete keys from a section config entry Dominik Csapak
@ 2023-01-17 11:46 ` Dominik Csapak
2023-03-08 11:43 ` [pve-devel] applied: " Thomas Lamprecht
2023-01-17 11:46 ` [pve-devel] [PATCH access-control v3 2/2] add realm-sync plugin for jobs and CRUD api for realm-sync-jobs Dominik Csapak
` (8 subsequent siblings)
10 siblings, 1 reply; 17+ messages in thread
From: Dominik Csapak @ 2023-01-17 11:46 UTC (permalink / raw)
To: pve-devel
so that we can reuse it easily
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/PVE/Auth/Plugin.pm | 42 +++++++++++++++++++++++-------------------
1 file changed, 23 insertions(+), 19 deletions(-)
diff --git a/src/PVE/Auth/Plugin.pm b/src/PVE/Auth/Plugin.pm
index bae9fb9..b341046 100755
--- a/src/PVE/Auth/Plugin.pm
+++ b/src/PVE/Auth/Plugin.pm
@@ -51,26 +51,30 @@ PVE::JSONSchema::register_standard_option('realm', {
my $remove_options = "(?:acl|properties|entry)";
+PVE::JSONSchema::register_standard_option('sync-scope', {
+ description => "Select what to sync.",
+ type => 'string',
+ enum => [qw(users groups both)],
+ optional => '1',
+});
+
+PVE::JSONSchema::register_standard_option('sync-remove-vanished', {
+ description => "A semicolon-seperated list of things to remove when they or the user"
+ ." vanishes during a sync. The following values are possible: 'entry' removes the"
+ ." user/group when not returned from the sync. 'properties' removes the set"
+ ." properties on existing user/group that do not appear in the source (even custom ones)."
+ ." 'acl' removes acls when the user/group is not returned from the sync."
+ ." Instead of a list it also can be 'none' (the default).",
+ type => 'string',
+ default => 'none',
+ typetext => "([acl];[properties];[entry])|none",
+ pattern => "(?:(?:$remove_options\;)*$remove_options)|none",
+ optional => '1',
+});
+
my $realm_sync_options_desc = {
- scope => {
- description => "Select what to sync.",
- type => 'string',
- enum => [qw(users groups both)],
- optional => '1',
- },
- 'remove-vanished' => {
- description => "A semicolon-seperated list of things to remove when they or the user"
- ." vanishes during a sync. The following values are possible: 'entry' removes the"
- ." user/group when not returned from the sync. 'properties' removes the set"
- ." properties on existing user/group that do not appear in the source (even custom ones)."
- ." 'acl' removes acls when the user/group is not returned from the sync."
- ." Instead of a list it also can be 'none' (the default).",
- type => 'string',
- default => 'none',
- typetext => "([acl];[properties];[entry])|none",
- pattern => "(?:(?:$remove_options\;)*$remove_options)|none",
- optional => '1',
- },
+ scope => get_standard_option('sync-scope'),
+ 'remove-vanished' => get_standard_option('sync-remove-vanished'),
# TODO check/rewrite in pve7to8, and remove with 8.0
full => {
description => "DEPRECATED: use 'remove-vanished' instead. If set, uses the LDAP Directory as source of truth,"
--
2.30.2
^ permalink raw reply [flat|nested] 17+ messages in thread* [pve-devel] [PATCH access-control v3 2/2] add realm-sync plugin for jobs and CRUD api for realm-sync-jobs
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
2023-01-17 11:46 ` [pve-devel] [PATCH common v3 1/1] SectionConfig: add helper to delete keys from a section config entry Dominik Csapak
2023-01-17 11:46 ` [pve-devel] [PATCH access-control v3 1/2] realm sync: refactor scope/remove-vanished into a standard option Dominik Csapak
@ 2023-01-17 11:46 ` Dominik Csapak
2023-06-07 8:38 ` [pve-devel] applied: " Thomas Lamprecht
2023-01-17 11:46 ` [pve-devel] [PATCH widget-toolkit v3 1/1] RealmComboBox: add custom store filters for callers Dominik Csapak
` (7 subsequent siblings)
10 siblings, 1 reply; 17+ messages in thread
From: Dominik Csapak @ 2023-01-17 11:46 UTC (permalink / raw)
To: pve-devel
to be able to define automated jobs that sync ldap/ad
The jobs plugin contains special handling when no node is given, since
we only want it to run on a single node when that triggers. For that,
we save a statefile in /etc/pve/priv/jobs/ which contains the
node/time/upid of the node that runs the job. The first node that
is able to lock the realm (via cfs_lock_domain) "wins" and may
sync from the ldap.
in case a specific node was selected, this is omitted and the Jobs
handling will not let it run on other nodes anyway
the API part is our usual sectionconfig CRUD api, but specialized
for the specific type of job.
the api will be at /cluster/jobs/realm-sync
(this must be done in pve-manager)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/PVE/API2/AccessControl/Makefile | 6 +
src/PVE/API2/AccessControl/RealmSync.pm | 292 ++++++++++++++++++++++++
src/PVE/API2/Makefile | 4 +
src/PVE/Jobs/Makefile | 6 +
src/PVE/Jobs/RealmSync.pm | 201 ++++++++++++++++
src/PVE/Makefile | 1 +
6 files changed, 510 insertions(+)
create mode 100644 src/PVE/API2/AccessControl/Makefile
create mode 100644 src/PVE/API2/AccessControl/RealmSync.pm
create mode 100644 src/PVE/Jobs/Makefile
create mode 100644 src/PVE/Jobs/RealmSync.pm
diff --git a/src/PVE/API2/AccessControl/Makefile b/src/PVE/API2/AccessControl/Makefile
new file mode 100644
index 0000000..8f9d749
--- /dev/null
+++ b/src/PVE/API2/AccessControl/Makefile
@@ -0,0 +1,6 @@
+API2_SOURCES= \
+ RealmSync.pm \
+
+.PHONY: install
+install:
+ for i in ${API2_SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/API2/AccessControl/$$i; done
diff --git a/src/PVE/API2/AccessControl/RealmSync.pm b/src/PVE/API2/AccessControl/RealmSync.pm
new file mode 100644
index 0000000..4a689ca
--- /dev/null
+++ b/src/PVE/API2/AccessControl/RealmSync.pm
@@ -0,0 +1,292 @@
+package PVE::API2::AccessControl::RealmSync;
+
+use strict;
+use warnings;
+
+use JSON;
+
+use PVE::Cluster qw (cfs_read_file cfs_write_file cfs_lock_file);
+use PVE::Exception qw(raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Job::Registry;
+use PVE::RESTHandler;
+use PVE::SafeSyslog;
+use PVE::SectionConfig;
+use PVE::Tools qw(extract_param);
+
+use PVE::AccessControl;
+use PVE::Auth::Plugin;
+use PVE::Jobs::RealmSync;
+
+
+use base qw(PVE::RESTHandler);
+
+my $get_cluster_last_run = sub {
+ my ($jobid) = @_;
+
+ my $state = eval { PVE::Jobs::RealmSync::get_state($jobid) };
+ die "error on getting state for '$jobid': $@\n" if $@;
+
+ if (my $upid = $state->{upid}) {
+ if (my $decoded = PVE::Tools::upid_decode($upid)) {
+ return $decoded->{starttime};
+ }
+ } else {
+ return $state->{time};
+ }
+
+ return undef;
+};
+
+__PACKAGE__->register_method ({
+ name => 'syncjob_index',
+ path => '',
+ method => 'GET',
+ description => "List configured realm-sync-jobs.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => "object",
+ properties => {
+ id => {
+ description => "The ID of the entry.",
+ type => 'string'
+ },
+ enabled => {
+ description => "If the job is enabled or not.",
+ type => 'boolean',
+ },
+ comment => {
+ description => "A comment for the job.",
+ type => 'string',
+ optional => 1,
+ },
+ schedule => {
+ description => "The configured sync schedule.",
+ type => 'string',
+ },
+ realm => get_standard_option('realm'),
+ scope => get_standard_option('sync-scope'),
+ 'remove-vanished' => get_standard_option('sync-remove-vanished'),
+ 'last-run' => {
+ description => "Last execution time of the job in seconds since the beginning of the UNIX epoch",
+ type => 'integer',
+ optional => 1,
+ },
+ 'next-run' => {
+ description => "Next planned execution time of the job in seconds since the beginning of the UNIX epoch.",
+ type => 'integer',
+ optional => 1,
+ },
+ },
+ },
+ links => [ { rel => 'child', href => "{id}" } ],
+ },
+ code => sub {
+ my ($param) = @_;
+
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $user = $rpcenv->get_user();
+
+ my $jobs_data = cfs_read_file('jobs.cfg');
+ my $order = $jobs_data->{order};
+ my $jobs = $jobs_data->{ids};
+
+ my $res = [];
+ for my $jobid (sort { $order->{$a} <=> $order->{$b} } keys %$jobs) {
+ my $job = $jobs->{$jobid};
+ next if $job->{type} ne 'realm-sync';
+
+ $job->{id} = $jobid;
+ if (my $schedule = $job->{schedule}) {
+ $job->{'last-run'} = eval { $get_cluster_last_run->($jobid) };
+ my $last_run = $job->{'last-run'} // time(); # current time as fallback
+ my $calendar_event = Proxmox::RS::CalendarEvent->new($schedule);
+ my $next_run = $calendar_event->compute_next_event($last_run);
+ $job->{'next-run'} = $next_run if defined($next_run);
+ }
+
+ push @$res, $job;
+ }
+
+ return $res;
+ }});
+
+__PACKAGE__->register_method({
+ name => 'read_job',
+ path => '{id}',
+ method => 'GET',
+ description => "Read realm-sync job definition.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ id => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ },
+ },
+ returns => {
+ type => 'object',
+ },
+ code => sub {
+ my ($param) = @_;
+
+ my $jobs = cfs_read_file('jobs.cfg');
+ my $id = $param->{id};
+ my $job = $jobs->{ids}->{$id};
+ return $job if $job && $job->{type} eq 'realm-sync';
+
+ raise_param_exc({ id => "No such job '$id'" });
+
+ }});
+
+__PACKAGE__->register_method({
+ name => 'create_job',
+ path => '{id}',
+ method => 'POST',
+ protected => 1,
+ description => "Create new realm-sync job.",
+ permissions => {
+ description => "'Realm.AllocateUser' on '/access/realm/<realm>' and "
+ ."'User.Modify' permissions to '/access/groups/'.",
+ check => [ 'and',
+ ['perm', '/access/realm/{realm}', ['Realm.AllocateUser']],
+ ['perm', '/access/groups', ['User.Modify']],
+ ],
+ },
+ parameters => PVE::Jobs::RealmSync->createSchema(),
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $id = extract_param($param, 'id');
+
+ cfs_lock_file('jobs.cfg', undef, sub {
+ my $data = cfs_read_file('jobs.cfg');
+
+ die "Job '$id' already exists\n"
+ if $data->{ids}->{$id};
+
+ my $plugin = PVE::Job::Registry->lookup('realm-sync');
+ my $opts = $plugin->check_config($id, $param, 1, 1);
+
+ my $realm = $opts->{realm};
+ my $cfg = cfs_read_file('domains.cfg');
+
+ raise_param_exc({ realm => "No such realm '$realm'" })
+ if !defined($cfg->{ids}->{$realm});
+
+ my $realm_type = $cfg->{ids}->{$realm}->{type};
+ raise_param_exc({ realm => "Only LDAP/AD realms can be synced." })
+ if $realm_type ne 'ldap' && $realm_type ne 'ad';
+
+ $data->{ids}->{$id} = $opts;
+
+ cfs_write_file('jobs.cfg', $data);
+ });
+ die "$@" if ($@);
+
+ return undef;
+ }});
+
+__PACKAGE__->register_method({
+ name => 'update_job',
+ path => '{id}',
+ method => 'PUT',
+ protected => 1,
+ description => "Update realm-sync job definition.",
+ permissions => {
+ description => "'Realm.AllocateUser' on '/access/realm/<realm>' and "
+ ." 'User.Modify' permissions to '/access/groups/'.",
+ check => [ 'and',
+ ['perm', '/access/realm/{realm}', ['Realm.AllocateUser']],
+ ['perm', '/access/groups', ['User.Modify']],
+ ],
+ },
+ parameters => PVE::Jobs::RealmSync->updateSchema(),
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $id = extract_param($param, 'id');
+ my $delete = extract_param($param, 'delete');
+ if ($delete) {
+ $delete = [PVE::Tools::split_list($delete)];
+ }
+
+ cfs_lock_file('jobs.cfg', undef, sub {
+ my $jobs = cfs_read_file('jobs.cfg');
+
+ die "no options specified\n" if !scalar(keys %$param);
+
+ my $plugin = PVE::Job::Registry->lookup('realm-sync');
+ my $opts = $plugin->check_config($id, $param, 0, 1);
+
+ my $job = $jobs->{ids}->{$id};
+ die "no such realm-sync job\n" if !$job || $job->{type} ne 'realm-sync';
+
+ my $options = $plugin->options();
+ PVE::SectionConfig::delete_from_config($job, $options, $opts, $delete);
+
+ $job->{$_} = $param->{$_} for keys $param->%*;
+
+ cfs_write_file('jobs.cfg', $jobs);
+
+ return;
+ });
+ die "$@" if ($@);
+ }});
+
+
+__PACKAGE__->register_method({
+ name => 'delete_job',
+ path => '{id}',
+ method => 'DELETE',
+ description => "Delete realm-sync job definition.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ protected => 1,
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ id => {
+ type => 'string',
+ format => 'pve-configid',
+ },
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $id = $param->{id};
+
+ cfs_lock_file('jobs.cfg', undef, sub {
+ my $jobs = cfs_read_file('jobs.cfg');
+
+ if (!defined($jobs->{ids}->{$id}) || $jobs->{ids}->{$id}->{type} ne 'realm-sync') {
+ raise_param_exc({ id => "No such job '$id'" });
+ }
+ delete $jobs->{ids}->{$id};
+
+ cfs_write_file('jobs.cfg', $jobs);
+ PVE::Jobs::RealmSync::save_state($id, undef);
+ });
+ die "$@" if $@;
+
+ return undef;
+ }});
+
+1;
diff --git a/src/PVE/API2/Makefile b/src/PVE/API2/Makefile
index 2817f48..7d2be93 100644
--- a/src/PVE/API2/Makefile
+++ b/src/PVE/API2/Makefile
@@ -2,6 +2,7 @@
API2_SOURCES= \
AccessControl.pm \
Domains.pm \
+ RealmSync.pm \
ACL.pm \
Role.pm \
Group.pm \
@@ -9,6 +10,9 @@ API2_SOURCES= \
TFA.pm \
OpenId.pm
+SUBDIRS=AccessControl
+
.PHONY: install
install:
for i in ${API2_SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/API2/$$i; done
+ set -e && for i in ${SUBDIRS}; do ${MAKE} -C $$i $@; done
diff --git a/src/PVE/Jobs/Makefile b/src/PVE/Jobs/Makefile
new file mode 100644
index 0000000..9eed1b2
--- /dev/null
+++ b/src/PVE/Jobs/Makefile
@@ -0,0 +1,6 @@
+SOURCES=RealmSync.pm
+
+.PHONY: install
+install: ${SOURCES}
+ install -d -m 0755 ${DESTDIR}${PERLDIR}/PVE/Jobs
+ for i in ${SOURCES}; do install -D -m 0644 $$i ${DESTDIR}${PERLDIR}/PVE/Jobs/$$i; done
diff --git a/src/PVE/Jobs/RealmSync.pm b/src/PVE/Jobs/RealmSync.pm
new file mode 100644
index 0000000..3a518cc
--- /dev/null
+++ b/src/PVE/Jobs/RealmSync.pm
@@ -0,0 +1,201 @@
+package PVE::Jobs::RealmSync;
+
+use strict;
+use warnings;
+
+use JSON;
+use POSIX qw(ENOENT);
+
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Cluster;
+use PVE::CalendarEvent;
+use PVE::Tools;
+
+use PVE::API2::Domains;
+
+use base qw(PVE::Job::Registry);
+
+sub type {
+ return 'realm-sync';
+}
+
+my $props = get_standard_option('realm-sync-options', {
+ realm => get_standard_option('realm'),
+});
+
+sub properties {
+ return $props;
+}
+
+sub options {
+ my $options = {
+ enabled => { optional => 1 },
+ schedule => {},
+ comment => { optional => 1 },
+ scope => {},
+ };
+ for my $opt (keys %$props) {
+ next if defined($options->{$opt});
+ # ignore legacy props from realm-sync schema
+ next if $opt eq 'full' || $opt eq 'purge';
+ if ($props->{$opt}->{optional}) {
+ $options->{$opt} = { optional => 1 };
+ } else {
+ $options->{$opt} = {};
+ }
+ }
+ $options->{realm}->{fixed} = 1;
+
+ return $options;
+}
+
+sub decode_value {
+ my ($class, $type, $key, $value) = @_;
+ return $value;
+}
+
+sub encode_value {
+ my ($class, $type, $key, $value) = @_;
+ return $value;
+}
+
+sub createSchema {
+ my ($class, $skip_type) = @_;
+
+ my $schema = $class->SUPER::createSchema($skip_type);
+
+ my $opts = $class->options();
+ for my $opt (keys $schema->{properties}->%*) {
+ next if defined($opts->{$opt}) || $opt eq 'id';
+ delete $schema->{properties}->{$opt};
+ }
+
+ return $schema;
+}
+
+sub updateSchema {
+ my ($class, $skip_type) = @_;
+ my $schema = $class->SUPER::updateSchema($skip_type);
+
+ my $opts = $class->options();
+ for my $opt (keys $schema->{properties}->%*) {
+ next if defined($opts->{$opt});
+ next if $opt eq 'id' || $opt eq 'delete';
+ delete $schema->{properties}->{$opt};
+ }
+
+ return $schema;
+}
+
+my $statedir = "/etc/pve/priv/jobs";
+
+sub get_state {
+ my ($id) = @_;
+
+ mkdir $statedir;
+ my $statefile = "$statedir/realm-sync-$id.json";
+ my $raw = eval { PVE::Tools::file_get_contents($statefile) } // '';
+
+ my $state = ($raw =~ m/^(\{.*\})$/) ? decode_json($1) : {};
+
+ return $state;
+}
+
+sub save_state {
+ my ($id, $state) = @_;
+
+ mkdir $statedir;
+ my $statefile = "$statedir/realm-sync-$id.json";
+
+ if (defined($state)) {
+ PVE::Tools::file_set_contents($statefile, encode_json($state));
+ } else {
+ unlink $statefile or $! == ENOENT or die "could not delete state for $id - $!\n";
+ }
+
+ return undef;
+}
+
+sub run {
+ my ($class, $conf, $id, $schedule) = @_;
+
+ for my $opt (keys %$conf) {
+ delete $conf->{$opt} if !defined($props->{$opt});
+ }
+
+ my $realm = $conf->{realm};
+
+ # cluster synced
+ my $now = time();
+ my $nodename = PVE::INotify::nodename();
+
+ # check statefile in pmxcfs if we should start
+ my $shouldrun = PVE::Cluster::cfs_lock_domain('realm-sync', undef, sub {
+ my $members = PVE::Cluster::get_members();
+
+ my $state = get_state($id);
+ my $last_node = $state->{node} // $nodename;
+ my $last_upid = $state->{upid};
+ my $last_time = $state->{time};
+
+ my $last_node_online = $last_node eq $nodename || ($members->{$last_node} // {})->{online};
+
+ if (defined($last_upid)) {
+ # first check if the next run is scheduled
+ if (my $parsed = PVE::Tools::upid_decode($last_upid, 1)) {
+ my $cal_spec = PVE::CalendarEvent::parse_calendar_event($schedule);
+ my $next_sync = PVE::CalendarEvent::compute_next_event($cal_spec, $parsed->{starttime});
+ return 0 if !defined($next_sync) || $now < $next_sync; # not yet its (next) turn
+ }
+ # check if still running and node is online
+ my $tasks = PVE::Cluster::get_tasklist();
+ for my $task (@$tasks) {
+ next if $task->{upid} ne $last_upid;
+ last if defined($task->{endtime}); # it's already finished
+ last if !$last_node_online; # it's not finished and the node is offline
+ return 0; # not finished and online
+ }
+ } elsif (defined($last_time) && ($last_time+60) > $now && $last_node_online) {
+ # another node started this job in the last 60 seconds and is still online
+ return 0;
+ }
+
+ # any of the following conditions should be true here:
+ # * it was started on another node but that node is offline now
+ # * it was started but either too long ago, or with an error
+ # * the started task finished
+
+ save_state($id, {
+ node => $nodename,
+ time => $now,
+ });
+ return 1;
+ });
+ die $@ if $@;
+
+ if ($shouldrun) {
+ my $upid = eval { PVE::API2::Domains->sync($conf) };
+ my $err = $@;
+ PVE::Cluster::cfs_lock_domain('realm-sync', undef, sub {
+ if ($err && !$upid) {
+ save_state($id, {
+ node => $nodename,
+ time => $now,
+ error => $err,
+ });
+ die "$err\n";
+ }
+
+ save_state($id, {
+ node => $nodename,
+ upid => $upid,
+ });
+ });
+ die $@ if $@;
+ return $upid;
+ }
+
+ return "OK"; # all other cases should not run the sync on this node
+}
+
+1;
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index c839d8f..dfc5314 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -8,3 +8,4 @@ install:
install -D -m 0644 TokenConfig.pm ${DESTDIR}${PERLDIR}/PVE/TokenConfig.pm
make -C API2 install
make -C CLI install
+ make -C Jobs install
--
2.30.2
^ permalink raw reply [flat|nested] 17+ messages in thread* [pve-devel] applied: [PATCH access-control v3 2/2] add realm-sync plugin for jobs and CRUD api for realm-sync-jobs
2023-01-17 11:46 ` [pve-devel] [PATCH access-control v3 2/2] add realm-sync plugin for jobs and CRUD api for realm-sync-jobs Dominik Csapak
@ 2023-06-07 8:38 ` Thomas Lamprecht
0 siblings, 0 replies; 17+ messages in thread
From: Thomas Lamprecht @ 2023-06-07 8:38 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
Am 17/01/2023 um 12:46 schrieb Dominik Csapak:
> to be able to define automated jobs that sync ldap/ad
>
> The jobs plugin contains special handling when no node is given, since
> we only want it to run on a single node when that triggers. For that,
> we save a statefile in /etc/pve/priv/jobs/ which contains the
> node/time/upid of the node that runs the job. The first node that
> is able to lock the realm (via cfs_lock_domain) "wins" and may
> sync from the ldap.
>
> in case a specific node was selected, this is omitted and the Jobs
> handling will not let it run on other nodes anyway
>
> the API part is our usual sectionconfig CRUD api, but specialized
> for the specific type of job.
>
> the api will be at /cluster/jobs/realm-sync
> (this must be done in pve-manager)
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> src/PVE/API2/AccessControl/Makefile | 6 +
> src/PVE/API2/AccessControl/RealmSync.pm | 292 ++++++++++++++++++++++++
> src/PVE/API2/Makefile | 4 +
> src/PVE/Jobs/Makefile | 6 +
> src/PVE/Jobs/RealmSync.pm | 201 ++++++++++++++++
> src/PVE/Makefile | 1 +
> 6 files changed, 510 insertions(+)
> create mode 100644 src/PVE/API2/AccessControl/Makefile
> create mode 100644 src/PVE/API2/AccessControl/RealmSync.pm
> create mode 100644 src/PVE/Jobs/Makefile
> create mode 100644 src/PVE/Jobs/RealmSync.pm
>
>
applied, moving the API from AccessControl namespace to Jobs one, as it's confusing
if it's located in PVE::API2::AccessControl::, but isn't mounted there.
Some small code/import cleanups thrown in, thanks!
^ permalink raw reply [flat|nested] 17+ messages in thread
* [pve-devel] [PATCH widget-toolkit v3 1/1] RealmComboBox: add custom store filters for callers
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (2 preceding siblings ...)
2023-01-17 11:46 ` [pve-devel] [PATCH access-control v3 2/2] add realm-sync plugin for jobs and CRUD api for realm-sync-jobs Dominik Csapak
@ 2023-01-17 11:46 ` Dominik Csapak
2023-03-14 14:26 ` [pve-devel] applied: " Thomas Lamprecht
2023-01-17 11:46 ` [pve-devel] [PATCH manager v3 1/4] Jobs: include existing types in state file regex for deletion Dominik Csapak
` (6 subsequent siblings)
10 siblings, 1 reply; 17+ messages in thread
From: Dominik Csapak @ 2023-01-17 11:46 UTC (permalink / raw)
To: pve-devel
so that a user can filter the underlying store, e.g. for type
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/form/RealmComboBox.js | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/src/form/RealmComboBox.js b/src/form/RealmComboBox.js
index 5f61687..c57b52d 100644
--- a/src/form/RealmComboBox.js
+++ b/src/form/RealmComboBox.js
@@ -6,7 +6,12 @@ Ext.define('Proxmox.form.RealmComboBox', {
xclass: 'Ext.app.ViewController',
init: function(view) {
- view.store.on('load', this.onLoad, view);
+ let store = view.getStore();
+ if (view.storeFilter) {
+ store.setFilters(view.storeFilter);
+ }
+ store.on('load', this.onLoad, view);
+ store.load();
},
onLoad: function(store, records, success) {
@@ -27,6 +32,9 @@ Ext.define('Proxmox.form.RealmComboBox', {
},
},
+ // define custom filters for the underlying store
+ storeFilter: undefined,
+
fieldLabel: gettext('Realm'),
name: 'realm',
queryMode: 'local',
@@ -52,6 +60,6 @@ Ext.define('Proxmox.form.RealmComboBox', {
store: {
model: 'pmx-domains',
- autoLoad: true,
+ autoLoad: false,
},
});
--
2.30.2
^ permalink raw reply [flat|nested] 17+ messages in thread* [pve-devel] [PATCH manager v3 1/4] Jobs: include existing types in state file regex for deletion
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (3 preceding siblings ...)
2023-01-17 11:46 ` [pve-devel] [PATCH widget-toolkit v3 1/1] RealmComboBox: add custom store filters for callers Dominik Csapak
@ 2023-01-17 11:46 ` Dominik Csapak
2023-01-17 11:46 ` [pve-devel] [PATCH manager v3 2/4] Jobs: add RealmSync Plugin and register it Dominik Csapak
` (5 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: Dominik Csapak @ 2023-01-17 11:46 UTC (permalink / raw)
To: pve-devel
otherwise, we cannot correctly match types that contain a hyphen,
since the id itself can also contain those.
creating a regex where the first part is the concrete allowed
types followed by a hyphen + id can also match those.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
PVE/Jobs.pm | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/PVE/Jobs.pm b/PVE/Jobs.pm
index 86ce9f693..70cb48212 100644
--- a/PVE/Jobs.pm
+++ b/PVE/Jobs.pm
@@ -324,7 +324,10 @@ sub synchronize_job_states_with_config {
}
}
- PVE::Tools::dir_glob_foreach($state_dir, '(.*?)-(.*).json', sub {
+ my $valid_types = PVE::Job::Registry->lookup_types();
+ my $type_regex = join("|", $valid_types->@*);
+
+ PVE::Tools::dir_glob_foreach($state_dir, "(${type_regex})-(.*).json", sub {
my ($path, $type, $id) = @_;
if (!defined($data->{ids}->{$id})) {
--
2.30.2
^ permalink raw reply [flat|nested] 17+ messages in thread* [pve-devel] [PATCH manager v3 2/4] Jobs: add RealmSync Plugin and register it
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (4 preceding siblings ...)
2023-01-17 11:46 ` [pve-devel] [PATCH manager v3 1/4] Jobs: include existing types in state file regex for deletion Dominik Csapak
@ 2023-01-17 11:46 ` Dominik Csapak
2023-01-17 11:46 ` [pve-devel] [PATCH manager v3 3/4] api: add realm-sync crud api to /cluster/jobs Dominik Csapak
` (4 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: Dominik Csapak @ 2023-01-17 11:46 UTC (permalink / raw)
To: pve-devel
so that realmsync jobs get executed
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
PVE/Jobs.pm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/PVE/Jobs.pm b/PVE/Jobs.pm
index 70cb48212..bd3233323 100644
--- a/PVE/Jobs.pm
+++ b/PVE/Jobs.pm
@@ -7,9 +7,11 @@ use JSON;
use PVE::Cluster qw(cfs_lock_file cfs_read_file cfs_register_file);
use PVE::Job::Registry;
use PVE::Jobs::VZDump;
+use PVE::Jobs::RealmSync;
use PVE::Tools;
PVE::Jobs::VZDump->register();
+PVE::Jobs::RealmSync->register();
PVE::Job::Registry->init();
cfs_register_file(
--
2.30.2
^ permalink raw reply [flat|nested] 17+ messages in thread* [pve-devel] [PATCH manager v3 3/4] api: add realm-sync crud api to /cluster/jobs
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (5 preceding siblings ...)
2023-01-17 11:46 ` [pve-devel] [PATCH manager v3 2/4] Jobs: add RealmSync Plugin and register it Dominik Csapak
@ 2023-01-17 11:46 ` Dominik Csapak
2023-01-17 11:46 ` [pve-devel] [PATCH manager v3 4/4] ui: add Realm Sync panel Dominik Csapak
` (3 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: Dominik Csapak @ 2023-01-17 11:46 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
PVE/API2/Cluster/Jobs.pm | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/PVE/API2/Cluster/Jobs.pm b/PVE/API2/Cluster/Jobs.pm
index 8166333dc..9a59d7f31 100644
--- a/PVE/API2/Cluster/Jobs.pm
+++ b/PVE/API2/Cluster/Jobs.pm
@@ -5,9 +5,15 @@ use warnings;
use PVE::RESTHandler;
use PVE::CalendarEvent;
+use PVE::API2::AccessControl::RealmSync;
use base qw(PVE::RESTHandler);
+__PACKAGE__->register_method ({
+ subclass => "PVE::API2::AccessControl::RealmSync",
+ path => 'realm-sync',
+});
+
__PACKAGE__->register_method({
name => 'index',
path => '',
@@ -35,6 +41,7 @@ __PACKAGE__->register_method({
code => sub {
return [
{ subdir => 'schedule-analyze' },
+ { subdir => 'realm-sync' },
];
}});
--
2.30.2
^ permalink raw reply [flat|nested] 17+ messages in thread* [pve-devel] [PATCH manager v3 4/4] ui: add Realm Sync panel
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (6 preceding siblings ...)
2023-01-17 11:46 ` [pve-devel] [PATCH manager v3 3/4] api: add realm-sync crud api to /cluster/jobs Dominik Csapak
@ 2023-01-17 11:46 ` Dominik Csapak
2023-03-07 8:06 ` [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (2 subsequent siblings)
10 siblings, 0 replies; 17+ messages in thread
From: Dominik Csapak @ 2023-01-17 11:46 UTC (permalink / raw)
To: pve-devel
a typical CRUD panel for adding/editing/removing realm sync jobs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/dc/Config.js | 7 +
www/manager6/dc/RealmSyncJob.js | 364 ++++++++++++++++++++++++++++++++
3 files changed, 372 insertions(+)
create mode 100644 www/manager6/dc/RealmSyncJob.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 05afeda40..02ba06d84 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -166,6 +166,7 @@ JSSRC= \
dc/MetricServerView.js \
dc/UserTagAccessEdit.js \
dc/RegisteredTagsEdit.js \
+ dc/RealmSyncJob.js \
lxc/CmdMenu.js \
lxc/Config.js \
lxc/CreateWizard.js \
diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
index 13ded12e8..72a9bec13 100644
--- a/www/manager6/dc/Config.js
+++ b/www/manager6/dc/Config.js
@@ -140,6 +140,13 @@ Ext.define('PVE.dc.Config', {
iconCls: 'fa fa-address-book-o',
itemId: 'domains',
},
+ {
+ xtype: 'pveRealmSyncJobView',
+ title: gettext('Realm Sync'),
+ groups: ['permissions'],
+ iconCls: 'fa fa-refresh',
+ itemId: 'realmsyncjobs',
+ },
{
xtype: 'pveHAStatus',
title: 'HA',
diff --git a/www/manager6/dc/RealmSyncJob.js b/www/manager6/dc/RealmSyncJob.js
new file mode 100644
index 000000000..5a903ef7d
--- /dev/null
+++ b/www/manager6/dc/RealmSyncJob.js
@@ -0,0 +1,364 @@
+Ext.define('PVE.dc.RealmSyncJobView', {
+ extend: 'Ext.grid.Panel',
+ alias: 'widget.pveRealmSyncJobView',
+
+ stateful: true,
+ stateId: 'grid-realmsyncjobs',
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ addRealmSyncJob: function(button) {
+ let me = this;
+ Ext.create(`PVE.dc.RealmSyncJobEdit`, {
+ autoShow: true,
+ listeners: {
+ destroy: () => me.reload(),
+ },
+ });
+ },
+
+ editRealmSyncJob: function() {
+ let me = this;
+ let view = me.getView();
+ let selection = view.getSelection();
+ if (!selection || selection.length < 1) {
+ return;
+ }
+
+ Ext.create(`PVE.dc.RealmSyncJobEdit`, {
+ jobid: selection[0].data.id,
+ autoShow: true,
+ listeners: {
+ destroy: () => me.reload(),
+ },
+ });
+ },
+
+ reload: function() {
+ this.getView().getStore().load();
+ },
+ },
+
+ store: {
+ autoLoad: true,
+ id: 'realm-syncs',
+ proxy: {
+ type: 'proxmox',
+ url: '/api2/json/cluster/jobs/realm-sync',
+ },
+ },
+
+ columns: [
+ {
+ header: gettext('Enabled'),
+ width: 80,
+ dataIndex: 'enabled',
+ xtype: 'checkcolumn',
+ sortable: true,
+ disabled: true,
+ disabledCls: 'x-item-enabled',
+ stopSelection: false,
+ },
+ {
+ text: gettext('Name'),
+ flex: 1,
+ dataIndex: 'id',
+ hidden: true,
+ },
+ {
+ text: gettext('Realm'),
+ width: 200,
+ dataIndex: 'realm',
+ },
+ {
+ header: gettext('Schedule'),
+ width: 150,
+ dataIndex: 'schedule',
+ },
+ {
+ text: gettext('Next Run'),
+ dataIndex: 'next-run',
+ width: 150,
+ renderer: PVE.Utils.render_next_event,
+ },
+ {
+ header: gettext('Comment'),
+ dataIndex: 'comment',
+ renderer: Ext.htmlEncode,
+ sorter: (a, b) => (a.data.comment || '').localeCompare(b.data.comment || ''),
+ flex: 1,
+ },
+ ],
+
+ tbar: [
+ {
+ text: gettext('Add'),
+ handler: 'addRealmSyncJob',
+ },
+ {
+ text: gettext('Edit'),
+ xtype: 'proxmoxButton',
+ handler: 'editRealmSyncJob',
+ disabled: true,
+ },
+ {
+ xtype: 'proxmoxStdRemoveButton',
+ baseurl: `/api2/extjs/cluster/jobs/realm-sync`,
+ callback: 'reload',
+ },
+ ],
+
+ listeners: {
+ itemdblclick: 'editRealmSyncJob',
+ },
+
+ initComponent: function() {
+ var me = this;
+
+ me.callParent();
+
+ Proxmox.Utils.monStoreErrors(me, me.getStore());
+ },
+});
+
+Ext.define('PVE.dc.RealmSyncJobEdit', {
+ extend: 'Proxmox.window.Edit',
+ mixins: ['Proxmox.Mixin.CBind'],
+
+ subject: gettext('Realm Sync Job'),
+ onlineHelp: 'pveum_ldap_sync',
+
+ // don't focus the schedule field on edit
+ defaultFocus: 'field[name=id]',
+
+ cbindData: function() {
+ let me = this;
+ me.isCreate = !me.jobid;
+ me.jobid = me.jobid || "";
+ let url = '/api2/extjs/cluster/jobs/realm-sync';
+ me.url = me.jobid ? `${url}/${me.jobid}` : url;
+ me.method = me.isCreate ? 'POST' : 'PUT';
+ if (!me.isCreate) {
+ me.subject = `${me.subject}: ${me.jobid}`;
+ }
+ return {};
+ },
+
+ submitUrl: function(url, values) {
+ return this.isCreate ? `${url}/${values.id}` : url;
+ },
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ updateDefaults: function(_field, newValue) {
+ let me = this;
+ // only update on create
+ if (!me.getView().isCreate) {
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/access/domains/${newValue}`,
+ success: function(response) {
+ // first reset the fields to their default
+ ['acl', 'entry', 'properties'].forEach(opt => {
+ me.lookup(`remove-vanished-${opt}`)?.setValue(false);
+ });
+ me.lookup('enable-new')?.setValue('1');
+ me.lookup('scope')?.setValue(undefined);
+
+ let options = response?.result?.data?.['sync-defaults-options'];
+ if (options) {
+ let parsed = PVE.Parser.parsePropertyString(options);
+ if (parsed['remove-vanished']) {
+ let opts = parsed['remove-vanished'].split(';');
+ for (const opt of opts) {
+ me.lookup(`remove-vanished-${opt}`)?.setValue(true);
+ }
+ delete parsed['remove-vanished'];
+ }
+ for (const [name, value] of Object.entries(parsed)) {
+ me.lookup(name)?.setValue(value);
+ }
+ }
+ },
+ });
+ },
+ },
+
+ items: [
+ {
+ xtype: 'inputpanel',
+
+ cbind: {
+ isCreate: '{isCreate}',
+ },
+
+ onGetValues: function(values) {
+ let me = this;
+
+ let vanished_opts = [];
+ ['acl', 'entry', 'properties'].forEach((prop) => {
+ if (values[`remove-vanished-${prop}`]) {
+ vanished_opts.push(prop);
+ }
+ delete values[`remove-vanished-${prop}`];
+ });
+
+ if (!values.id && me.isCreate) {
+ values.id = 'realmsync-' + Ext.data.identifier.Uuid.Global.generate().slice(0, 13);
+ }
+
+ if (vanished_opts.length > 0) {
+ values['remove-vanished'] = vanished_opts.join(';');
+ } else {
+ values['remove-vanished'] = 'none';
+ }
+
+ PVE.Utils.delete_if_default(values, 'node', '');
+
+ if (me.isCreate) {
+ delete values.delete; // on create we cannot delete values
+ }
+
+ return values;
+ },
+
+ column1: [
+ {
+ xtype: 'pmxDisplayEditField',
+ editConfig: {
+ xtype: 'pmxRealmComboBox',
+ storeFilter: rec => rec.data.type === 'ldap' || rec.data.type === 'ad',
+ },
+ cbind: {
+ editable: '{isCreate}',
+ },
+ listeners: {
+ change: 'updateDefaults',
+ },
+ fieldLabel: gettext('Realm'),
+ name: 'realm',
+ reference: 'realm',
+ },
+ {
+ xtype: 'pveCalendarEvent',
+ fieldLabel: gettext('Schedule'),
+ allowBlank: false,
+ name: 'schedule',
+ reference: 'schedule',
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ fieldLabel: gettext('Enable'),
+ name: 'enabled',
+ reference: 'enabled',
+ uncheckedValue: 0,
+ defaultValue: 1,
+ checked: true,
+ },
+ ],
+
+ column2: [
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'scope',
+ reference: 'scope',
+ fieldLabel: gettext('Scope'),
+ value: '',
+ emptyText: gettext('No default available'),
+ deleteEmpty: false,
+ allowBlank: false,
+ comboItems: [
+ ['users', gettext('Users')],
+ ['groups', gettext('Groups')],
+ ['both', gettext('Users and Groups')],
+ ],
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ value: '1',
+ deleteEmpty: false,
+ allowBlank: false,
+ comboItems: [
+ ['1', Proxmox.Utils.yesText],
+ ['0', Proxmox.Utils.noText],
+ ],
+ name: 'enable-new',
+ reference: 'enable-new',
+ fieldLabel: gettext('Enable new'),
+ },
+ ],
+
+ columnB: [
+ {
+ xtype: 'fieldset',
+ title: gettext('Remove Vanished Options'),
+ items: [
+ {
+ xtype: 'proxmoxcheckbox',
+ fieldLabel: gettext('ACL'),
+ name: 'remove-vanished-acl',
+ reference: 'remove-vanished-acl',
+ boxLabel: gettext('Remove ACLs of vanished users and groups.'),
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ fieldLabel: gettext('Entry'),
+ name: 'remove-vanished-entry',
+ reference: 'remove-vanished-entry',
+ boxLabel: gettext('Remove vanished user and group entries.'),
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ fieldLabel: gettext('Properties'),
+ name: 'remove-vanished-properties',
+ reference: 'remove-vanished-properties',
+ boxLabel: gettext('Remove vanished properties from synced users.'),
+ },
+ ],
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'comment',
+ fieldLabel: gettext('Job Comment'),
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext('Description of the job'),
+ },
+ },
+ {
+ xtype: 'displayfield',
+ reference: 'defaulthint',
+ value: gettext('Default sync options can be set by editing the realm.'),
+ userCls: 'pmx-hint',
+ hidden: true,
+ },
+ ],
+ },
+ ],
+
+ initComponent: function() {
+ let me = this;
+ me.callParent();
+ if (me.jobid) {
+ me.load({
+ success: function(response, options) {
+ let values = response.result.data;
+
+ if (values['remove-vanished']) {
+ let opts = values['remove-vanished'].split(';');
+ for (const opt of opts) {
+ values[`remove-vanished-${opt}`] = 1;
+ }
+ }
+ me.down('inputpanel').setValues(values);
+ },
+ });
+ }
+ },
+});
--
2.30.2
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (7 preceding siblings ...)
2023-01-17 11:46 ` [pve-devel] [PATCH manager v3 4/4] ui: add Realm Sync panel Dominik Csapak
@ 2023-03-07 8:06 ` Dominik Csapak
2023-05-03 7:35 ` Dominik Csapak
2023-06-07 9:59 ` Thomas Lamprecht
10 siblings, 0 replies; 17+ messages in thread
From: Dominik Csapak @ 2023-03-07 8:06 UTC (permalink / raw)
To: pve-devel
any comment here? IMO the series is in good shape, maybe
it could take a bit more testing (from someone besides me)
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (8 preceding siblings ...)
2023-03-07 8:06 ` [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
@ 2023-05-03 7:35 ` Dominik Csapak
2023-06-07 9:59 ` Thomas Lamprecht
10 siblings, 0 replies; 17+ messages in thread
From: Dominik Csapak @ 2023-05-03 7:35 UTC (permalink / raw)
To: pve-devel
any comment to the rest of the series (access-control 2/2 and manager)?
it still applies
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs
2023-01-17 11:46 [pve-devel] [PATCH common/access-control/wt/manager v3] add realm sync jobs Dominik Csapak
` (9 preceding siblings ...)
2023-05-03 7:35 ` Dominik Csapak
@ 2023-06-07 9:59 ` Thomas Lamprecht
10 siblings, 0 replies; 17+ messages in thread
From: Thomas Lamprecht @ 2023-06-07 9:59 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
Am 17/01/2023 um 12:46 schrieb Dominik Csapak:
> pve-manager:
>
> Dominik Csapak (4):
> Jobs: include existing types in state file regex for deletion
> Jobs: add RealmSync Plugin and register it
> api: add realm-sync crud api to /cluster/jobs
> ui: add Realm Sync panel
>
> PVE/API2/Cluster/Jobs.pm | 7 +
> PVE/Jobs.pm | 7 +-
> www/manager6/Makefile | 1 +
> www/manager6/dc/Config.js | 7 +
> www/manager6/dc/RealmSyncJob.js | 364 ++++++++++++++++++++++++++++++++
> 5 files changed, 385 insertions(+), 1 deletion(-)
> create mode 100644 www/manager6/dc/RealmSyncJob.js
>
Now applied those too finally, thanks!
Some thoughts:
* would clarify that a existing LDAP/AD realm is required via some UX change in the
add/edit widnow, maybe:
- add an empty text in the field
- disable the "scope" field until an realm is selected, as otherwise the invalid
state is slightly confusing.
* Merge this into realm, as panel at the bottom there.
Having many realms is rather the exception, the two built-ins plus one or maybe
two external ones (e.g., a LDAP and a OIDC) are probably enough for most setups.
That means we got a lot of "free" reals^W space estate in the existing realm panel,
putting that to better use and avoiding a tree entry might maybe improve cognitive
load imposed by our UI minimally (or at least not increase it).
* A "Run now" button is missing?
^ permalink raw reply [flat|nested] 17+ messages in thread