[pve-devel] [PATCH cluster/manager/network 0/3] remove legacy ipam and mac db from observed files

[pve-devel] [PATCH cluster/manager/network 0/3] remove legacy ipam and mac db from observed files

[Stefan Hanreich]

The IPAM and MAC DB files have been moved in PVE 8.3. Remove the fallback code
that handles that transitional state with PVE 9. Leave the pve-network
postinstall script for now and potentially replace it in PVE 10 with a script
that removes the legacy files. Even foolhardy users can then upgrade from PVE
8.2 to PVE 9 directly without any IPAM breakage.

pve-cluster:

Stefan Hanreich (1):
  pmxcfs: remove legacy ipam db and mac cache files

 src/PVE/Cluster.pm  | 2 --
 src/pmxcfs/status.c | 2 --
 2 files changed, 4 deletions(-)


pve-manager:

Stefan Hanreich (1):
  pve8to9: check for legacy ipam db or mac cache files

 PVE/CLI/pve8to9.pm | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)


pve-network:

Stefan Hanreich (1):
  ipam: remove fallback code for ipam.db and mac cache

 src/PVE/Network/SDN/Ipams.pm           | 30 +-------------------------
 src/PVE/Network/SDN/Ipams/PVEPlugin.pm | 28 +-----------------------
 2 files changed, 2 insertions(+), 56 deletions(-)


Summary over all repositories:
  5 files changed, 45 insertions(+), 60 deletions(-)

-- 
Generated by git-murpp 0.8.0

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-cluster 1/1] pmxcfs: remove legacy ipam db and mac cache files

[Stefan Hanreich]

Those files get migrated with pve-network >= 0.9.9 - so we do not need
to observe them anymore via pmxcfs.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/Cluster.pm  | 2 --
 src/pmxcfs/status.c | 2 --
 2 files changed, 4 deletions(-)

diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index 9380241..e829687 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -61,8 +61,6 @@ my $observed = {
     'priv/tfa.cfg' => 1,
     'priv/token.cfg' => 1,
     'priv/acme/plugins.cfg' => 1,
-    'priv/ipam.db' => 1, # TODO: replaced by sdn/ipam-pve-db.json, remove in PVE 9+
-    'priv/macs.db' => 1, # TODO: replaced by sdn/mac-cache.json, remove in PVE 9+
     '/qemu-server/' => 1,
     '/openvz/' => 1,
     '/lxc/' => 1,
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index a00e793..d07ab33 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -88,8 +88,6 @@ static memdb_change_t memdb_change_array[] = {
     {.path = "priv/acme/plugins.cfg"},
     {.path = "priv/tfa.cfg"},
     {.path = "priv/token.cfg"},
-    {.path = "priv/ipam.db"}, // TODO: replaced by sdn/ipam-pve-db.json remove with PVE 9 or later
-    {.path = "priv/macs.db"}, // TODO: replaced by sdn/mac-cache.json remove with PVE 9 or later
     {.path = "datacenter.cfg"},
     {.path = "vzdump.cron"},
     {.path = "vzdump.conf"},
-- 
2.47.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-manager 1/1] pve8to9: check for legacy ipam db or mac cache files

[Stefan Hanreich]

Check if there are any not-yet-migrated ipam / mac cache files in
pmxcfs. Those should have been migrated over in the pve-network
postinst, but if something went wrong during this process we can
explicitly notify users here again to avoid any unpleasant suprises
after the upgrade.

If all nodes are on PVE 9, it is safe to delete the legacy files,
print a notice informing the users.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 PVE/CLI/pve8to9.pm | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/PVE/CLI/pve8to9.pm b/PVE/CLI/pve8to9.pm
index 834432b95..5f7c21af9 100644
--- a/PVE/CLI/pve8to9.pm
+++ b/PVE/CLI/pve8to9.pm
@@ -2016,6 +2016,48 @@ sub check_virtual_guests {
     check_qemu_machine_versions();
 }
 
+my $LEGACY_IPAM_DB = "/etc/pve/priv/ipam.db";
+my $NEW_IPAM_DB = "/etc/pve/sdn/pve-ipam-state.json";
+
+my $LEGACY_MAC_DB = "/etc/pve/priv/macs.db";
+my $NEW_MAC_DB = "/etc/pve/sdn/mac-cache.json";
+
+sub check_legacy_ipam_files {
+    log_info("Checking for IPAM DB files that have not yet been migrated.");
+
+    if (-e $LEGACY_IPAM_DB) {
+        if (-e $NEW_IPAM_DB) {
+            log_notice(
+                "Found leftover legacy IPAM DB file in $LEGACY_IPAM_DB.\n"
+                . "This file can be deleted AFTER upgrading ALL nodes to PVE 9."
+            );
+        } else {
+            log_fail(
+                "Found IPAM DB file in $LEGACY_IPAM_DB that has not been migrated!\n"
+                . "File needs to be migrated to $NEW_IPAM_DB before upgrading. Update pve-network to the newest version."
+            );
+        }
+    } else {
+        log_pass("No legacy IPAM DB found.");
+    }
+
+    if (-e $LEGACY_MAC_DB) {
+        if (-e $NEW_MAC_DB) {
+            log_notice(
+                "Found leftover legacy MAC DB file in $LEGACY_MAC_DB.\n"
+                . "This file can be deleted AFTER upgrading ALL nodes to PVE 9."
+            );
+        } else {
+            log_fail(
+                "Found MAC DB file in $LEGACY_MAC_DB that has not been migrated!\n"
+                . "File needs to be migrated to $NEW_MAC_DB before upgrading. Update pve-network to the newest version."
+            );
+        }
+    } else {
+        log_pass("No legacy MAC DB found.");
+    }
+}
+
 sub check_misc {
     print_header("MISCELLANEOUS CHECKS");
     my $ssh_config = eval { PVE::Tools::file_get_contents('/root/.ssh/config') };
@@ -2110,6 +2152,7 @@ sub check_misc {
     check_legacy_backup_job_options();
     check_lvm_autoactivation();
     check_rrd_migration();
+    check_legacy_ipam_files();
 }
 
 my sub colored_if {
-- 
2.47.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-network 1/1] ipam: remove fallback code for ipam.db and mac cache

[Stefan Hanreich]

With the upgrade to PVE 9, those files should have been successfully
migrated on all nodes, so we do not need to fallback on the old files
anymore.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/Network/SDN/Ipams.pm           | 30 +-------------------------
 src/PVE/Network/SDN/Ipams/PVEPlugin.pm | 28 +-----------------------
 2 files changed, 2 insertions(+), 56 deletions(-)

diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
index 7ec0f06..00aa20c 100644
--- a/src/PVE/Network/SDN/Ipams.pm
+++ b/src/PVE/Network/SDN/Ipams.pm
@@ -21,35 +21,7 @@ PVE::Network::SDN::Ipams::PhpIpamPlugin->register();
 PVE::Network::SDN::Ipams::Plugin->init();
 
 my $macdb_filename = "sdn/mac-cache.json";
-my $macdb_filename_legacy = 'priv/macs.db';
-
-cfs_register_file(
-    $macdb_filename,
-    sub {
-        my ($filename, $data) = @_;
-        if (defined($data)) {
-            return json_reader($filename, $data);
-        } else {
-            # TODO: remove legacy cache file handling with PVE 9+ after ensuring all call sites got
-            # switched over.
-            return cfs_read_file($macdb_filename_legacy);
-        }
-    },
-    sub {
-        my ($filename, $data) = @_;
-        # TODO: remove below with PVE 9+, add a pve8to9 check to allow doing so.
-        if (-e $macdb_filename_legacy && -e $macdb_filename) {
-            # only clean-up if we succeeded to write the new path at least once
-            unlink $macdb_filename_legacy
-                or $!{ENOENT}
-                or warn "failed to unlink legacy MAC cache - $!\n";
-        }
-        return json_writer->($filename, $data);
-    },
-);
-
-# drop reading $macdb_filename_legacy with PVE 9+ - for now do not write it anymore.
-cfs_register_file($macdb_filename_legacy, \&json_reader, undef);
+cfs_register_file($macdb_filename, \&json_reader, \&json_writer);
 
 sub json_reader {
     my ($filename, $data) = @_;
diff --git a/src/PVE/Network/SDN/Ipams/PVEPlugin.pm b/src/PVE/Network/SDN/Ipams/PVEPlugin.pm
index 6764d79..d2aba1e 100644
--- a/src/PVE/Network/SDN/Ipams/PVEPlugin.pm
+++ b/src/PVE/Network/SDN/Ipams/PVEPlugin.pm
@@ -14,37 +14,11 @@ use Digest::SHA;
 use base('PVE::Network::SDN::Ipams::Plugin');
 
 my $ipamdb_file = "sdn/pve-ipam-state.json";
-my $ipamdb_file_legacy = "priv/ipam.db";
 
 PVE::Cluster::cfs_register_file(
     $ipamdb_file,
-    sub {
-        my ($filename, $data) = @_;
-        if (defined($data)) {
-            return PVE::Network::SDN::Ipams::PVEPlugin->parse_config($filename, $data);
-        } else {
-            # TODO: remove legacy state file handling with PVE 9+ after ensuring all call sites got
-            # switched over.
-            return cfs_read_file($ipamdb_file_legacy);
-        }
-    },
-    sub {
-        my ($filename, $data) = @_;
-        # TODO: remove below with PVE 9+, add a pve8to9 check to allow doing so.
-        if (-e $ipamdb_file_legacy && -e $ipamdb_file) {
-            # only clean-up if we succeeded to write the new path at least once
-            unlink $ipamdb_file_legacy
-                or $!{ENOENT}
-                or warn "failed to unlink legacy IPAM DB - $!\n";
-        }
-        return PVE::Network::SDN::Ipams::PVEPlugin->write_config($filename, $data);
-    },
-);
-
-PVE::Cluster::cfs_register_file(
-    $ipamdb_file_legacy,
     sub { PVE::Network::SDN::Ipams::PVEPlugin->parse_config(@_); },
-    undef, # no writer for legacy file, all must go to the new file.
+    sub { PVE::Network::SDN::Ipams::PVEPlugin->write_config(@_); },
 );
 
 sub type {
-- 
2.47.2


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-cluster 1/1] pmxcfs: remove legacy ipam db and mac cache files

[Thomas Lamprecht]

On Fri, 01 Aug 2025 14:10:25 +0200, Stefan Hanreich wrote:
> Those files get migrated with pve-network >= 0.9.9 - so we do not need
> to observe them anymore via pmxcfs.
> 
> 

Applied, thanks!

[1/1] pmxcfs: remove legacy ipam db and mac cache files
      commit: 8c12940a66e84ace6c428f4ddbda547d862316b0


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-network 1/1] ipam: remove fallback code for ipam.db and mac cache

[Thomas Lamprecht]

On Fri, 01 Aug 2025 14:10:27 +0200, Stefan Hanreich wrote:
> With the upgrade to PVE 9, those files should have been successfully
> migrated on all nodes, so we do not need to fallback on the old files
> anymore.
> 
> 

Applied, thanks!

[1/1] ipam: remove fallback code for ipam.db and mac cache
      commit: 3245a20d7c501e524b4899b923dea66b9475faff


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH pve-manager 1/1] pve8to9: check for legacy ipam db or mac cache files

[Thomas Lamprecht]

On Fri, 01 Aug 2025 14:10:26 +0200, Stefan Hanreich wrote:
> Check if there are any not-yet-migrated ipam / mac cache files in
> pmxcfs. Those should have been migrated over in the pve-network
> postinst, but if something went wrong during this process we can
> explicitly notify users here again to avoid any unpleasant suprises
> after the upgrade.
> 
> If all nodes are on PVE 9, it is safe to delete the legacy files,
> print a notice informing the users.
> 
> [...]

Applied, thanks!

[1/1] pve8to9: check for legacy ipam db or mac cache files
      commit: 17488fc091be297e2732a66be387b839a333e36b


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel