[pve-devel] [RFC network/ve-rs 0/8] Template-based FRR config generation

* [pve-devel] [RFC network/ve-rs 0/8] Template-based FRR config generation
@ 2025-09-19  9:41 Gabriel Goller
  2025-09-19  9:41 ` [pve-devel] [PATCH ve-rs 1/4] frr: add templates and structs to represent the frr config Gabriel Goller
                   ` (7 more replies)
  0 siblings, 8 replies; 9+ messages in thread
From: Gabriel Goller @ 2025-09-19  9:41 UTC (permalink / raw)
  To: pve-devel

Currently we generate the frr config by having a few rust-structs that look
very similar to the literal frr config -- these structs then implement a custom
trait similar to Display, which serializes the config into a String. This is
not optimal due to multiple reasons, the main one being it's quite verbose and
and the fact that we often mixed Display and FrrDump implementations. This
means that sometimes structs are serialized using Display and sometimes using
FrrDump. There is also the question of granularity, so does a single line
correspond to a single FrrDump implementation? Or does every frr word
correspond to an FrrDump implementation and a random one just adds a '
'?

In order to clean this up a bit, there was an attempt at writing a full-fledged
serde serializer, which takes structs and converts them into blocks and using
nested enums as the options on every line. This turned out to be quite
difficult and involved a lot of magic when creating the config in rust.
Especially the ending statements (e.g. 'exit-address-family' when exiting an
address-family block, but 'exit' when exiting a router block) turned out to be
quite tricky.

The third and probably most popular way (Vyos and Sonic both use jinja
templates as well) to generate frr config is to use templates. So you basically
have rust-structs that look similar to the frr config and implement derive or
implemennt serialize. Then we can use template files, which contain the whole
configuration with every option wrapped in a `if` block and lists/blocks
wrapped in `for` loops. Finally, we can use the rust structs to insert them into
the templates and get the full frr config. The major downside of this approach
is that the frr template files get quite complicated and can be a major source
of config generation errors. A positive thing though, is that we don't need to
model the whole frr config with every option and every property, we can easily
wrap a few options into a single (e.g.) bool, and then maybe split it out later
when we add a ui option for each property.
Here I made the decision to split every protocol into its own template file.
These template files then import some common structs, such as the interfaces and
route-maps, which are used in every protocol. To accommodate for this, I also had
to change the structure of the FrrConfig:
So we no longer have:

config/
├─ interfaces/
│  ├─ openfabric
│  ├─ ospf
├─ routers/
│  ├─ openfabric
│  ├─ ospf

but:

config/
├─ openfabric/
│  ├─ interfaces
│  ├─ routers
├─ ospf/
│  ├─ interfaces
│  ├─ routers

Which plays much nicer with the templates, and IMO still makes sense logically.

I used minijinja for this series, as that seems to be the most used template
engine in the rust ecosystem, even though we mostly seem to use handlebars. I
discussed this we Lukas and Thomas, and we'll decide in the next few weeks if
this is fine, or it's better if we use the same templating engine
everywhere.

The current frr.conf.local is quite broken, so in order to introduce a new and
(a bit) saner option to write custom frr config, add a directory where the user
can specific custom template files to customize the frr config generated by the
sdn stack. Having a templating engine is very powerful here, as the user could
write something like this:

    interface {{ interface_name }}
    {% if interface_name == "ens19" %}
     ip network point-to-point
    {% endif %}

Obviously if you don't know what you're doing, you can break everything, so I'd
wouldn't advertise this feature too much :)

The pve-network patches are only here to make the generated frr config nicer
(removing duplicated "!" comments) and fixing/improving the tests.

ve-rs:

Gabriel Goller (4):
  frr: add templates and structs to represent the frr config
  sdn-types: forward serialize to display for NET
  ve-config: fabrics: use new proxmox-frr structs to generate frr config
  tests: always prepend the frr delimiter/comment "!" to the block

 proxmox-frr/Cargo.toml                        |   4 +
 proxmox-frr/debian/control                    |  14 +
 proxmox-frr/examples/route_map.rs             |  70 +++++
 proxmox-frr/src/lib.rs                        | 166 ++++--------
 proxmox-frr/src/openfabric.rs                 |  15 +-
 proxmox-frr/src/ospf.rs                       |  37 +--
 proxmox-frr/src/route_map.rs                  | 108 ++------
 proxmox-frr/src/serializer.rs                 | 256 +++++-------------
 proxmox-frr/templates/access_list.jinja       |   6 +
 proxmox-frr/templates/fabricd.jinja           |  36 +++
 proxmox-frr/templates/interface.jinja         |   4 +
 proxmox-frr/templates/ospfd.jinja             |  27 ++
 proxmox-frr/templates/route_map.jinja         |  11 +
 proxmox-sdn-types/src/net.rs                  |   4 +-
 proxmox-ve-config/src/sdn/fabric/frr.rs       | 145 +++++-----
 proxmox-ve-config/src/sdn/frr.rs              |  11 +-
 .../fabric__openfabric_default_pve.snap       |   2 +-
 .../fabric__openfabric_default_pve1.snap      |   2 +-
 .../fabric__openfabric_dualstack_pve.snap     |   8 +-
 .../fabric__openfabric_ipv6_only_pve.snap     |   2 +-
 .../fabric__openfabric_multi_fabric_pve1.snap |   2 +-
 .../snapshots/fabric__ospf_default_pve.snap   |   2 +-
 .../snapshots/fabric__ospf_default_pve1.snap  |   2 +-
 .../fabric__ospf_multi_fabric_pve1.snap       |   2 +-
 24 files changed, 426 insertions(+), 510 deletions(-)
 create mode 100644 proxmox-frr/examples/route_map.rs
 create mode 100644 proxmox-frr/templates/access_list.jinja
 create mode 100644 proxmox-frr/templates/fabricd.jinja
 create mode 100644 proxmox-frr/templates/interface.jinja
 create mode 100644 proxmox-frr/templates/ospfd.jinja
 create mode 100644 proxmox-frr/templates/route_map.jinja

network:

Gabriel Goller (4):
  sdn: remove duplicate comment line '!' in frr config
  sdn: add trailing newline in frr config
  sdn: tests: add missing comment '!' in frr config
  tests: use Test::Differences to make test assertions

 debian/control                                |  1 +
 src/PVE/Network/SDN/Frr.pm                    |  3 +-
 src/test/run_test_dns.pl                      | 15 ++++-----
 src/test/run_test_ipams.pl                    | 13 ++++----
 src/test/run_test_subnets.pl                  | 31 ++++++++++---------
 src/test/run_test_vnets_blackbox.pl           | 17 +++++-----
 src/test/run_test_zones.pl                    |  5 +--
 .../expected_controller_config                |  3 +-
 .../expected_controller_config                |  3 +-
 .../evpn/ebgp/expected_controller_config      |  3 +-
 .../ebgp_loopback/expected_controller_config  |  3 +-
 .../evpn/exitnode/expected_controller_config  |  3 +-
 .../expected_controller_config                |  3 +-
 .../expected_controller_config                |  3 +-
 .../exitnode_snat/expected_controller_config  |  3 +-
 .../expected_controller_config                |  3 +-
 .../evpn/ipv4/expected_controller_config      |  3 +-
 .../evpn/ipv4ipv6/expected_controller_config  |  3 +-
 .../expected_controller_config                |  3 +-
 .../evpn/ipv6/expected_controller_config      |  3 +-
 .../ipv6underlay/expected_controller_config   |  3 +-
 .../evpn/isis/expected_controller_config      |  3 +-
 .../isis_loopback/expected_controller_config  |  3 +-
 .../expected_controller_config                |  3 +-
 .../expected_controller_config                |  3 +-
 .../multiplezones/expected_controller_config  |  3 +-
 .../expected_controller_config                |  5 ++-
 .../ospf_fabric/expected_controller_config    |  5 ++-
 .../evpn/rt_import/expected_controller_config |  3 +-
 .../evpn/vxlanport/expected_controller_config |  3 +-
 30 files changed, 70 insertions(+), 88 deletions(-)

Summary over all repositories:
  54 files changed, 496 insertions(+), 598 deletions(-)

-- 
Generated by git-murpp 0.8.0

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

^ permalink raw reply	[flat|nested] 9+ messages in thread