[PVE-User] Change Ceph Public Network

[PVE-User] Change Ceph Public Network

[clayj]

I have Promox and Ceph running on 5 servers.  

 

I currently have 2 NICs in each server, a 1Gb NIC that's configured as the
Ceph "public network", 192.168.75.0/24; and a 2.5Gb NIC that's configured as
the Ceph "cluster network, 192.168.70.0/24.   The .75 net is also what
connects the servers to the "rest of the world", including some Synology NAS
servers.

 

I have a 1Tb SSD drive on 4 of the 5 servers configured as  Ceph OSDs.

 

I'd like to get better performance out of Ceph, so I added a 3rd 2Gb NIC to
each server, 192.168.71.0/24.

 

I took down all my VMs; and in ceph.conf, changed the mon entries and the
public_network entries from 192.168.75.xx to 192.168.71.xx; then did a
"restart ceph-target" on each of the Proxmox servers.

 

That was a complete disaster!   Ceph never came back up and I had to reverse
the process in order to restore.

 

Is there a "correct" way to do this; or did I just miss a step.  Is there a
config file for the OSDs that I need to change?

 

Thanks in advance!

 

Clay Jackson

clayj@n7qnm.net <mailto:clayj@n7qnm.net> 

 

_______________________________________________
pve-user mailing list
pve-user@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] Change Ceph Public Network

[David der Nederlanden | ITTY via pve-user]

[-- Attachment #1: Type: message/rfc822, Size: 13098 bytes --]

From: David der Nederlanden | ITTY <david@itty.nl>
To: "clayj@n7qnm.net" <clayj@n7qnm.net>, Proxmox VE user list <pve-user@lists.proxmox.com>
Subject: Re: [PVE-User] Change Ceph Public Network
Date: Mon, 7 Oct 2024 21:48:04 +0000
Message-ID: <a7a5afb9-8c89-48ee-8130-c41ba633ef9e@email.android.com>

Hi Clay,

This can be accomplished at runtime if done right,
take a look at this thread I opened a while ago.

https://forum.proxmox.com/threads/ceph-changing-public-network.119116/

In the meantime other users did such a migration succesfully including myself.

Kind regards
David der Nederlanden

On 7 Oct 2024 23:44, clayj@n7qnm.net wrote:
I have Promox and Ceph running on 5 servers.



I currently have 2 NICs in each server, a 1Gb NIC that's configured as the
Ceph "public network", 192.168.75.0/24; and a 2.5Gb NIC that's configured as
the Ceph "cluster network, 192.168.70.0/24.   The .75 net is also what
connects the servers to the "rest of the world", including some Synology NAS
servers.



I have a 1Tb SSD drive on 4 of the 5 servers configured as  Ceph OSDs.



I'd like to get better performance out of Ceph, so I added a 3rd 2Gb NIC to
each server, 192.168.71.0/24.



I took down all my VMs; and in ceph.conf, changed the mon entries and the
public_network entries from 192.168.75.xx to 192.168.71.xx; then did a
"restart ceph-target" on each of the Proxmox servers.



That was a complete disaster!   Ceph never came back up and I had to reverse
the process in order to restore.



Is there a "correct" way to do this; or did I just miss a step.  Is there a
config file for the OSDs that I need to change?



Thanks in advance!



Clay Jackson

clayj@n7qnm.net <mailto:clayj@n7qnm.net>



_______________________________________________
pve-user mailing list
pve-user@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user



[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
pve-user mailing list
pve-user@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user