Re: [pdm-devel] [PATCH 1/2] schema2rust: extract pattern validation from nested API schema fields

[Hannes Laimer <h.laimer@proxmox.com>]

On 10/15/25 14:32, Wolfgang Bumiller wrote:
> The patch itself works, but there are 2 issues:
> 
> The currently generated regexes (luckily?) all produce regexes which are
> 1. anchored
> 2. compatible with the regex crate
> 
> With your patch, a lot of them will not satisfy both.
> 
> For 1:
> The PVE schema code generally anchors the regexes at *use* time, which
> is less than ideal and we could probably move towards including the
> anchors in the regexes itself (unless we specifically need them
> unanchored for some reason?)
> In the rust code we could not do this. The regex crate does not expose
> this and requires them to be anchored explicitly within the regex.
> 
> We could also include `^` and `$` in the generator, but it'll be a bit
> redundant (but won't hurt).
> 
> But for 2:
> I'm afraid what we need to do is find a way to "whitelist" which regexes
> get generated. We could produce TODO comments for the rest.
> 
> I went ahead and pushed a commit which produces tests for the regexes.
> If you go ahead and run `cargo` test after regenerating with this patch,
> you'll see the issue.
> 

Thanks for taking a look! Hmm, I don't think we have patterns that rely
on PCRE specific features, i.e. we could maybe convert those without
losing semantic meaning for validation.

I'm not super familiar with PCRE and its intricacies, but I don't think
we have pattern that couldn't be represented otherwise.

Some things conversion would have to do:
- {,N} -> {0,N}
- in rust we shouldn't need flag-resets, (I think)
- (?^i:...) -> (?i:...)
- ...

Not sure if it would be feasible to try to integrate some kind of
"conversion" into the generator itself. Of the patterns used in the PVE
API spec, I think this would mostly be syntactic things. BUT I did
probably miss some things :P also, is there a reason we would not want
this assuming converting patterns "on-the-fly" in the generator is
feasible?


> On Wed, Oct 01, 2025 at 05:47:16PM +0200, Hannes Laimer wrote:
>> Previously, pattern validation was only extracted from top-level string fields,
>> leaving nested pattern fields and array item patterns unprocessed. This change
>> enables pattern extraction from nested schema structures, ensuring regex
>> validation is generated for all pattern fields regardless of their depth.
>> The generated Rust code now includes const_regex! definitions and
>> ApiStringFormat::Pattern references for previously ignored pattern fields.
>>
>> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
>> ---
>>   pve-api-types/generator-lib/Schema2Rust.pm | 27 +++++++++++++++++++---
>>   1 file changed, 24 insertions(+), 3 deletions(-)
>>
>> diff --git a/pve-api-types/generator-lib/Schema2Rust.pm b/pve-api-types/generator-lib/Schema2Rust.pm
>> index 0f3efc0a..794c7abe 100644
>> --- a/pve-api-types/generator-lib/Schema2Rust.pm
>> +++ b/pve-api-types/generator-lib/Schema2Rust.pm
>> @@ -206,6 +206,25 @@ my sub print_struct : prototype($$$) {
>>   
>>       my @arrays;
>>   
>> +    # Collect array fields
>> +    for my $field (sort keys $def->{fields}->%*) {
>> +        my $field_def = $def->{fields}->{$field};
>> +        if ($field_def->{kind} eq 'array-field') {
>> +            push @arrays, $field_def;
>> +        }
>> +    }
>> +
>> +    # Add regex constants from array fields to the main struct's API object
>> +    for my $array (@arrays) {
>> +        my $type_name = $array->{array_type_name};
>> +        next if $done_array_types->{$type_name};
>> +        if ($API && $array->{api}->{-regexes}) {
>> +            for my $name (keys %{$array->{api}->{-regexes}}) {
>> +                $def->{api}->{-regexes}->{$name} = $array->{api}->{-regexes}->{$name};
>> +            }
>> +        }
>> +    }
>> +
>>       print_api_string($out, $def->{api}, 'struct', $def->{name});
>>   
>>       if (length($def->{description})) {
>> @@ -217,9 +236,6 @@ my sub print_struct : prototype($$$) {
>>       print {$out} "pub struct $def->{name} {\n";
>>       for my $field (sort keys $def->{fields}->%*) {
>>           my $field_def = $def->{fields}->{$field};
>> -        if ($field_def->{kind} eq 'array-field') {
>> -            push @arrays, $field_def;
>> -        }
>>   
>>           my $name = $field_def->{name};
>>           my $rust_name = $field_def->{rust_name};
>> @@ -1097,6 +1113,11 @@ my sub string_type : prototype($$$$) {
>>           # if (my $kind = $fmt->{kind}) {
>>           #     $api_props->{format_fixme} = '"LIST TYPE"';
>>           # }
>> +    } elsif (defined(my $pattern = delete $schema->{pattern})) {
>> +        # Handle pattern field directly from schema if no format was specified
>> +        my $re_name = namify_const(${name_hint}, 're');
>> +        $api_props->{-regexes}->{$re_name} = $pattern;
>> +        $api_props->{format} = "&ApiStringFormat::Pattern(&$re_name)";
>>       }
>>   
>>       return 'String';
>> -- 
>> 2.47.3
>>
>>
>>
>> _______________________________________________
>> pdm-devel mailing list
>> pdm-devel@lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
>>
>>



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel