[pve-devel] [PATCH pve-network 00/38] add subnet plugin

[Alexandre Derumier <aderumier@odiso.com>]

This patch series add basic subnets managements.
(need pve-cluster V5 patch series)

Subnets will be use for multiple things:
- defined gateway ip on vnets
- enable snat on a subnet
- add cloudnit|dhcp default network configs (gateway, static routes, nameservers,searchdomain,....)
- add ipam management. (ip registrations to external ipam, for vm|ct ip management)
- add dns registration. (reverse dns for subnet, and searchdomain regisration in external dns like powerdns )
- ...


Already implemented:

- gateway option is implemented:

It's currently replace ip management on vnets for layer3 plugins,
through the option "gateway".

If the option gateway is defined, for layer3 plugins (evpn && simple),
the ip will be used for the vnet.

A vnet can have multiple subnets, with multiples ipv4/ipv6.

- ipam
- dns registration


Changelog v2:

- add ipams plugins. Currently netbox && phpipam.
- the subnet && the subnet gateway are registered to ipam
- add/del/find_next_free ip are implemented, so it should be easy to use them in qemu && lxc config.

Changelog v3:

- add an internal ipam plugin

Changelog v4:

- fix pveipam plugin find_free_ip
- detect ipv4/ipv6 in find_free_ip

Changelog v5:

- add vnets add_ip,del_ip,... should be ok for use in lxc/qemuserver

Changelog v6:

- add dns plugins
- internal ipam fixes
- rework vnet-subnet association
- fixes && cleanup

Changelog v7:

- add snat to simple|evpn plugin
- cleanup subnet options
- fix reversedns

Changelog v8:

- move subnet api to /sdn/vnet/<vnet>/subnet
- make ipam non optionnal && use pve ipam as default
- don't allow subnets on vlanaware vnet
- fixes && cleanup

Changelog v9:

- write running config on commit in /etc/pve/sdn/.running-config
  and use it as source for config generation,status

Changelog v10:

- move ipams/dns options from subnets to zone
- add hostname/description to ipam db
- cleanup && bugfix

Changelog v11:
- vnets: remove unused hash:diff
- evpn plugin improvements


Alexandre Derumier (38):
  vnets: add subnets
  add subnets verifications hooks
  zones: simple|evpn: add gateway ip from subnets to vnet
  zone: add vnet_update_hook
  vnets: subnets: use cidr
  subnet: fix on_delete_hook
  api2: subnet create: convert cidr to subnetid
  api2: increase version on apply/reload only
  add ipams plugins
  add pve internal ipam plugin
  vnets: find_free_ip : add ipversion detection
  vnets: add add_ip
  vnets: add del_ip + rework add_ip/find_free_ip
  add dns plugin
  Fix vnet gateway for routed setup + /32 pointopoint subnet
  ipam : pveplugin : fix find_next_free_ip
  add vnet to subnets && remove subnetlist from vnet
  zones: evpn|simple: add snat iptables rules
  subnet: disable route option for now and add dns domain format
  dns: fix reverse dns
  subnets: move api to /sdn/vnet/<vnet>/subnets && make vnet option not
    optionnal
  zones: evpn : fix raise exception
  subnet: make ipam not optionnal and use pve ipam as default
  don't allow subnets on vlanware vnet
  generate sdn/.running-config on apply
  api: add running/pending zones/vnets/subnets/controllers
  small bugfixes
  move dns options from subnets to zone
  move ipam option from subnet to zone
  subnets/ipam: allow same subnet on different zones
  vnets: allow duplicate tags in differents zones
  ipam: verify api access  on create/update
  ipam: add hostname/description to ipam db
  update documentation
  vnets: remove unused hash:diff
  zones: auto find controller
  sdn: controllers : add ebgp support
  update test documentation

 PVE/API2/Network/SDN.pm                   |  18 +-
 PVE/API2/Network/SDN/Controllers.pm       |  59 ++++-
 PVE/API2/Network/SDN/Dns.pm               | 242 ++++++++++++++++++++
 PVE/API2/Network/SDN/Ipams.pm             | 248 +++++++++++++++++++++
 PVE/API2/Network/SDN/Makefile             |   2 +-
 PVE/API2/Network/SDN/Subnets.pm           | 120 ++++++++--
 PVE/API2/Network/SDN/Vnets.pm             |  80 ++++++-
 PVE/API2/Network/SDN/Zones.pm             | 101 ++++++++-
 PVE/Network/SDN.pm                        | 102 +++++++--
 PVE/Network/SDN/Controllers.pm            |  47 ++--
 PVE/Network/SDN/Controllers/EvpnPlugin.pm |  21 +-
 PVE/Network/SDN/Controllers/Plugin.pm     |   1 +
 PVE/Network/SDN/Dns.pm                    |  57 +++++
 PVE/Network/SDN/Dns/Makefile              |   8 +
 PVE/Network/SDN/Dns/Plugin.pm             | 118 ++++++++++
 PVE/Network/SDN/Dns/PowerdnsPlugin.pm     | 255 ++++++++++++++++++++++
 PVE/Network/SDN/Ipams.pm                  |  70 ++++++
 PVE/Network/SDN/Ipams/Makefile            |   8 +
 PVE/Network/SDN/Ipams/NetboxPlugin.pm     | 189 ++++++++++++++++
 PVE/Network/SDN/Ipams/PVEPlugin.pm        | 204 +++++++++++++++++
 PVE/Network/SDN/Ipams/PhpIpamPlugin.pm    | 217 ++++++++++++++++++
 PVE/Network/SDN/Ipams/Plugin.pm           | 131 +++++++++++
 PVE/Network/SDN/Makefile                  |   4 +-
 PVE/Network/SDN/SubnetPlugin.pm           | 124 +++++++----
 PVE/Network/SDN/Subnets.pm                | 238 +++++++++++++++++++-
 PVE/Network/SDN/VnetPlugin.pm             |  37 ++--
 PVE/Network/SDN/Vnets.pm                  |  92 +++++++-
 PVE/Network/SDN/Zones.pm                  |  51 +++--
 PVE/Network/SDN/Zones/EvpnPlugin.pm       |  98 +++++++--
 PVE/Network/SDN/Zones/FaucetPlugin.pm     |   4 +
 PVE/Network/SDN/Zones/Plugin.pm           |  16 +-
 PVE/Network/SDN/Zones/QinQPlugin.pm       |  26 ++-
 PVE/Network/SDN/Zones/SimplePlugin.pm     |  73 ++++++-
 PVE/Network/SDN/Zones/VlanPlugin.pm       |  28 ++-
 PVE/Network/SDN/Zones/VxlanPlugin.pm      |  30 ++-
 debian/control                            |   2 +
 test/documentation.txt                    |  33 ++-
 test/generateconfig.pl                    |   5 +-
 38 files changed, 2942 insertions(+), 217 deletions(-)
 create mode 100644 PVE/API2/Network/SDN/Dns.pm
 create mode 100644 PVE/API2/Network/SDN/Ipams.pm
 create mode 100644 PVE/Network/SDN/Dns.pm
 create mode 100644 PVE/Network/SDN/Dns/Makefile
 create mode 100644 PVE/Network/SDN/Dns/Plugin.pm
 create mode 100644 PVE/Network/SDN/Dns/PowerdnsPlugin.pm
 create mode 100644 PVE/Network/SDN/Ipams.pm
 create mode 100644 PVE/Network/SDN/Ipams/Makefile
 create mode 100644 PVE/Network/SDN/Ipams/NetboxPlugin.pm
 create mode 100644 PVE/Network/SDN/Ipams/PVEPlugin.pm
 create mode 100644 PVE/Network/SDN/Ipams/PhpIpamPlugin.pm
 create mode 100644 PVE/Network/SDN/Ipams/Plugin.pm

-- 
2.20.1