From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Friedrich Weber <f.weber@proxmox.com>
Subject: [pve-devel] applied: [PATCH ceph master 1/3] fix #5213: ceph-osd postinst: add patch to avoid connection freezes
Date: Thu, 15 Feb 2024 14:16:38 +0100 [thread overview]
Message-ID: <2e541a87-4996-4630-b4bf-87f8308463b5@proxmox.com> (raw)
In-Reply-To: <20240215094056.66233-2-f.weber@proxmox.com>
Am 15/02/2024 um 10:40 schrieb Friedrich Weber:
> Assume there is an open TCP connection to a VM, and ceph-osd is
> installed/upgraded on the host on which the PVE firewall is active.
> Currently, ceph-osd postinst reloads all sysctl settings. Thus,
> installing/upgrading ceph-osd will set the sysctl setting
> `net.bridge.bridge-nf-call-iptables` to 0. The PVE firewall will flip
> the setting back to 1 in its next iteration (in <10 seconds). But
> while the setting is 0, conntrack will not see packets of the existing
> TCP connection. When the setting is flipped back to 1, conntrack will
> see packets again, but may consider the seq/ack numbers of new packets
> out-of-window, mark them as invalid and drop them. This will freeze
> the TCP connection.
>
> To avoid this, add a patch that modifies the ceph-osd postinst to only
> apply settings from the sysctl settings file shipped with ceph-osd,
> and only apply them on fresh install. As the ceph-osd sysctl settings
> do not set `net.bridge.bridge-nf-call-iptables`, this will avoid the
> temporary flip to 0 when installing/upgrading ceph-osd.
>
> Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
> ---
> ...t-avoid-reloading-all-sysctl-setting.patch | 47 +++++++++++++++++++
> patches/series | 1 +
> 2 files changed, 48 insertions(+)
> create mode 100644 patches/0015-ceph-osd-postinst-avoid-reloading-all-sysctl-setting.patch
>
>
applied, thanks!
as talked off-list, ceph is really not trying to reduce confusion potential
doing things like:
install -D -m 644 etc/sysctl/90-ceph-osd.conf $(DESTDIR)/etc/sysctl.d/30-ceph-osd.conf
I.e., having it checked in as 90-... but installing it as 30-..
And while I think the argument for "admin could have overrides that this
affects", which you mentioned that Fabian brought up off-list, is fine,
but is just as true on initial installation.
What might be better is one (or some) of:
- do nothing, just install the file and be done, a reboot sorts this out
sooner or later anyway.
- a script that checks if there are any overrides and only sets it up if
there are none else warns.
- just warns visible in general if lower values are detected.
- drop our odd disabling of the `net.bridge.bridge-nf-call-iptables`
next prev parent reply other threads:[~2024-02-15 13:16 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-15 9:40 [pve-devel] [PATCH ceph master+quincy-stable-8 0/3] fix #5213: avoid connection freezes when installing/upgrading ceph-osd Friedrich Weber
2024-02-15 9:40 ` [pve-devel] [PATCH ceph master 1/3] fix #5213: ceph-osd postinst: add patch to avoid connection freezes Friedrich Weber
2024-02-15 13:16 ` Thomas Lamprecht [this message]
2024-02-16 13:54 ` [pve-devel] applied: " Friedrich Weber
2024-02-15 9:40 ` [pve-devel] [PATCH ceph quincy-stable-8 2/3] " Friedrich Weber
2024-02-15 13:17 ` [pve-devel] applied: " Thomas Lamprecht
2024-02-15 9:40 ` [pve-devel] [PATCH ceph master 3/3] buildsys: add check for changed ceph-osd sysctl settings Friedrich Weber
2024-02-15 13:20 ` Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2e541a87-4996-4630-b4bf-87f8308463b5@proxmox.com \
--to=t.lamprecht@proxmox.com \
--cc=f.weber@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox