all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Ellmenreich <t.ellmenreich@proxmox.com>
To: pve-devel@lists.proxmox.com
Cc: Thomas Ellmenreich <t.ellmenreich@proxmox.com>
Subject: [PATCH common v2 00/11] pem parser: refactor parsing to cover more edge cases
Date: Wed, 18 Nov 2026 11:31:14 +0100	[thread overview]
Message-ID: <20261118103127.165100-1-t.ellmenreich@proxmox.com> (raw)

After the previous series: [0], was already applied, I realized that the
parsing ignores a specific edgecase. After that, I was also pointed to the
following thread [1], which includes another edgecase.

Trying to fix these edgecases, I have decided to rework the PEM parsing so
that it cleans incoming input, then parses it in a stricter manner, and
finally outputs a more standardized PEM at the end.

Change Overview:
---------------

I have tried to split up all of the new changes into separate commits to make
them more easily reversible but also better understandable. But here is a
summary of said changes:

 * Introduction of a ANY_LABEL constant instead of recreating 'qr/.*?/'
   everywhere. This new constant also avoids matching a next header.
 * 'pem-string' format calls 'check_pem' with multiple, since PEM chains
   should also be valid.
 * Force BEGIN and END lines to start at the beginning of the line and
   either end with the end of the line or the string.
 * Make the validation of single PEM entries stricter so that it adheres
   more to RFC 7468 [2].
 * Add a utility to not only strip leading, but also following text, so that
   subroutines that are expected to return a single item, but are handed a
   chain, can strip everything except the first item.
 * Parsing a chain now adds the cleaned and validated entries to a list,
   before joining them with newlines instead of adding them to a string.
 * Decoding a PEM to a DER is quicker to avoid parsing invalid PEM's.
 * Refactored and expanded the tests to cover more cases.
 * Added CRLF support by simply replacing them all with newlines at the
   beginning of the 'check_pem', 'split_pem' and 'der_to_pem' subroutines.

Changelog:
---------

 * v1 -> v2:

   The first version only contained the usage of the ANY_LABEL for the
   'pem_to_der' subroutine, as well as the check that BEGIN and END should be
   on their own lines. Through the further edge case mentioned in the
   forum post [1], I decided to take a more complete but more invasive
   approach.

Open Questions:
--------------

 * Integration tests with openssl

   I started working on some integration tests that work directly with
   openssl to generate PEM's, which are then validated by our library. In
   the end, I decided to drop the patch in favour of gathering some opinions
   first. My approach would have been to generate some edgecase PEM's with
   openssl and commit them, the Perl code could then be run against these
   PEM's to validate the stability of its functionality. Otherwise, we could
   also make the tests depend on openssl directly, and thus generate a way
   greater number of PEM's on the fly, to then test against.

 * Normalisation of content:

   One more improvement that could be made is to pass the base64 content
   through the decoder and encoder, which would lead it to be normalized. I
   personally find this unnecessary, but it could be one more thing to make
   the output of the 'check_pem' function more consistent.

Notes for the Maintainer:
------------------------

I have split up the series into a number of different commits to make
them easier to understand, but I would recommend to doublecheck before
dropping an inbetween commit, as not every commit makes sense by themselves.
That said, I have made sure that all commits pass all tests and can be built.

[0]: https://lore.proxmox.com/all/20260805141114.190477-1-t.ellmenreich@proxmox.com/T/#u
[1]: https://forum.proxmox.com/threads/custom-certificate-pem-with-comments-or-key-pem-with-a-section-ec-parameters-fail-to-upload.187526/#post-875517
[2]: https://www.rfc-editor.org/info/rfc7468/


pve-common:

Thomas Ellmenreich (11):
  certificate tests: use q{} for multiline strings.
  pem parser: add constant to match any label
  pem parser: refactor check_pem to use list instead of string
  pem parser: stricter handling of newlines
  certificate formats: enable multiple parsing for pem-string format
  pem parsing: strip all surrounding text from chain entries
  pem parsing: make validation of pem entry stricter
  pem parsing: default splitting to any label
  pem to der: create clearer error boundary
  certificate tests: refactor and exapand testsuite
  pom parsing: allow for CRLF line ending

 src/PVE/Certificate.pm         |  81 ++++--
 test/Makefile                  |   2 +
 test/certificate_utils_test.pl | 355 ++++++++++++++++++++++++++
 test/check_pem_test.pl         | 444 +++++++++++++--------------------
 test/pem_to_der_test.pl        | 172 +++++++++++++
 test/split_pem_test.pl         | 260 +++++++++----------
 6 files changed, 891 insertions(+), 423 deletions(-)
 create mode 100755 test/certificate_utils_test.pl
 create mode 100755 test/pem_to_der_test.pl


Summary over all repositories:
  6 files changed, 891 insertions(+), 423 deletions(-)

-- 
Generated by murpp 0.12.0




             reply	other threads:[~2026-11-18 10:32 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 10:31 Thomas Ellmenreich [this message]
2026-11-18 10:31 ` [PATCH common v2 01/11] certificate tests: use q{} for multiline strings Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 02/11] pem parser: add constant to match any label Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 03/11] pem parser: refactor check_pem to use list instead of string Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 04/11] pem parser: stricter handling of newlines Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 05/11] certificate formats: enable multiple parsing for pem-string format Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 06/11] pem parsing: strip all surrounding text from chain entries Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 07/11] pem parsing: make validation of pem entry stricter Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 08/11] pem parsing: default splitting to any label Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 09/11] pem to der: create clearer error boundary Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 10/11] certificate tests: refactor and exapand testsuite Thomas Ellmenreich
2026-11-18 10:31 ` [PATCH common v2 11/11] pom parsing: allow for CRLF line ending Thomas Ellmenreich

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118103127.165100-1-t.ellmenreich@proxmox.com \
    --to=t.ellmenreich@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal