[pve-devel] [PATCH lxc 0/2] fix apparmor rules and improve cgroupv2 experience

[pve-devel] [PATCH lxc 0/2] fix apparmor rules and improve cgroupv2 experience

[Stoiko Ivanov]

This patchset addresses 2 minor inconveniences I ran into, while running my
host with 'systemd.unified_cgroup_hierarchy=1':

* apparmor mount denies for '/proc/sys/kernel/random/boot_id' (this happens
  irrespective of the cgroup-layout
* having to add
  `lxc.init.cmd: /lib/systemd/systemd systemd.unified_cgroup_hierarchy=1`
  to all my container configs (for debian and arch containers at least
  alpine runs without issues) - see [0] for a discussion of the topic

While investigating this I noticed that the fixes for both issues were already
on upstream/master (with one small other fix in between) - so instead of
cherry-picking both patches I fast-forwarded to the last needed commit.
Glad to resend with the patches cherry-picked and added to our patchqueue.

I would probably submit the apparmor fix upstream (after a quick check by
another set of eyes :)

[0] https://github.com/lxc/lxc/issues/3183

Stoiko Ivanov (2):
  update lxc to include fixes for cgroupv2 setups
  apparmor: add rule for allowing remount of boot_id

 ...apparmor-Allow-ro-remount-of-boot_id.patch | 26 +++++++++++++++++++
 debian/patches/series                         |  1 +
 lxc                                           |  2 +-
 3 files changed, 28 insertions(+), 1 deletion(-)
 create mode 100644 debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch

-- 
2.20.1

[pve-devel] [PATCH lxc 1/2] update lxc to include fixes for cgroupv2 setups

[Stoiko Ivanov]

This commit fast-forwards 7 commits from upstream/master. The first commit
(partially) fixes a missing apparmor rule for /proc/sys/kernel/random/boot_id)

The last commit fixes running containers in pure cgroupv2 environments (by
premounting cgroup2).

It contains one other fix for a netlink bug, which I haven't seen in our
support channels, thus assume limited potential for regressions.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 lxc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lxc b/lxc
index 538337e..4547e73 160000
--- a/lxc
+++ b/lxc
@@ -1 +1 @@
-Subproject commit 538337ee9dc5ca385cc8d9b6faaac1575c014a1b
+Subproject commit 4547e73e3e1c7f7a9fc88da6ac3276d99df1c5ec
-- 
2.20.1

[pve-devel] [PATCH lxc 2/2] apparmor: add rule for allowing remount of boot_id

[Stoiko Ivanov]

commit 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 in upstream only partially
fixes the apparmor deny for mounting boot_id (used for example for identifying
different boots with `journalctl`) inside the container.

Tested by editing the profile and replacing it disregarding the cache:
`apparmor_parser -W -T -r /etc/apparmor.d/usr.bin.lxc-start`

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
---
 ...apparmor-Allow-ro-remount-of-boot_id.patch | 26 +++++++++++++++++++
 debian/patches/series                         |  1 +
 2 files changed, 27 insertions(+)
 create mode 100644 debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch

diff --git a/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
new file mode 100644
index 0000000..fefc586
--- /dev/null
+++ b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
@@ -0,0 +1,26 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Stoiko Ivanov <s.ivanov@proxmox.com>
+Date: Wed, 22 Jul 2020 12:17:24 +0200
+Subject: [PATCH lxc] apparmor: Allow ro remount of boot_id
+
+The rule added in 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 did not cover all
+necessary mount calls for /proc/sys/kernel/random/boot_id
+(in src/lxc/conf.c: lxc_setup_boot_id) - the ro remount is missing.
+
+Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
+---
+ config/apparmor/abstractions/start-container.in | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/config/apparmor/abstractions/start-container.in b/config/apparmor/abstractions/start-container.in
+index 9998f1121..9f64c2727 100644
+--- a/config/apparmor/abstractions/start-container.in
++++ b/config/apparmor/abstractions/start-container.in
+@@ -22,6 +22,7 @@
+   mount -> /var/lib/lxc/{**,},
+ 
+   mount /dev/.lxc-boot-id -> /proc/sys/kernel/random/boot_id,
++  mount options=(ro, nosuid, nodev, noexec, remount, bind) -> /proc/sys/kernel/random/boot_id,
+ 
+   # required for some pre-mount hooks
+   mount fstype=overlayfs,
diff --git a/debian/patches/series b/debian/patches/series
index ee20ef5..f588081 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -1,3 +1,4 @@
 pve/0001-PVE-Config-lxc.service-start-after-a-potential-syslo.patch
 pve/0002-PVE-Config-deny-rw-mounting-of-sys-and-proc.patch
 pve/0003-PVE-Config-attach-always-use-getent.patch
+pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
-- 
2.20.1

Re: [pve-devel] [PATCH lxc 2/2] apparmor: add rule for allowing remount of boot_id

[Thomas Lamprecht]

On 22.07.20 13:05, Stoiko Ivanov wrote:
> commit 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 in upstream only partially
> fixes the apparmor deny for mounting boot_id (used for example for identifying
> different boots with `journalctl`) inside the container.
> 
> Tested by editing the profile and replacing it disregarding the cache:
> `apparmor_parser -W -T -r /etc/apparmor.d/usr.bin.lxc-start`
> 

was this proposed to upstream as pull request? Did not found it on the
LXC GitHub page.

> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> ---
>  ...apparmor-Allow-ro-remount-of-boot_id.patch | 26 +++++++++++++++++++
>  debian/patches/series                         |  1 +
>  2 files changed, 27 insertions(+)
>  create mode 100644 debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> 
> diff --git a/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> new file mode 100644
> index 0000000..fefc586
> --- /dev/null
> +++ b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> @@ -0,0 +1,26 @@
> +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
> +From: Stoiko Ivanov <s.ivanov@proxmox.com>
> +Date: Wed, 22 Jul 2020 12:17:24 +0200
> +Subject: [PATCH lxc] apparmor: Allow ro remount of boot_id
> +
> +The rule added in 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 did not cover all
> +necessary mount calls for /proc/sys/kernel/random/boot_id
> +(in src/lxc/conf.c: lxc_setup_boot_id) - the ro remount is missing.
> +
> +Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> +---
> + config/apparmor/abstractions/start-container.in | 1 +
> + 1 file changed, 1 insertion(+)
> +
> +diff --git a/config/apparmor/abstractions/start-container.in b/config/apparmor/abstractions/start-container.in
> +index 9998f1121..9f64c2727 100644
> +--- a/config/apparmor/abstractions/start-container.in
> ++++ b/config/apparmor/abstractions/start-container.in
> +@@ -22,6 +22,7 @@
> +   mount -> /var/lib/lxc/{**,},
> + 
> +   mount /dev/.lxc-boot-id -> /proc/sys/kernel/random/boot_id,
> ++  mount options=(ro, nosuid, nodev, noexec, remount, bind) -> /proc/sys/kernel/random/boot_id,
> + 
> +   # required for some pre-mount hooks
> +   mount fstype=overlayfs,
> diff --git a/debian/patches/series b/debian/patches/series
> index ee20ef5..f588081 100644
> --- a/debian/patches/series
> +++ b/debian/patches/series
> @@ -1,3 +1,4 @@
>  pve/0001-PVE-Config-lxc.service-start-after-a-potential-syslo.patch
>  pve/0002-PVE-Config-deny-rw-mounting-of-sys-and-proc.patch
>  pve/0003-PVE-Config-attach-always-use-getent.patch
> +pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> 

Re: [pve-devel] [PATCH lxc 2/2] apparmor: add rule for allowing remount of boot_id

[Stoiko Ivanov]

On Wed, 22 Jul 2020 13:51:19 +0200
Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:

> On 22.07.20 13:05, Stoiko Ivanov wrote:
> > commit 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 in upstream only partially
> > fixes the apparmor deny for mounting boot_id (used for example for identifying
> > different boots with `journalctl`) inside the container.
> > 
> > Tested by editing the profile and replacing it disregarding the cache:
> > `apparmor_parser -W -T -r /etc/apparmor.d/usr.bin.lxc-start`
> >   
> 
> was this proposed to upstream as pull request? Did not found it on the
> LXC GitHub page.

sorry my phrasing in the cover-letter was misleading: I want to make a
pull request upstream for this patch, after somebody else sanity-checks it
-> if it looks ok to you - I'll open the PR.


> 
> > Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> > ---
> >  ...apparmor-Allow-ro-remount-of-boot_id.patch | 26 +++++++++++++++++++
> >  debian/patches/series                         |  1 +
> >  2 files changed, 27 insertions(+)
> >  create mode 100644 debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> > 
> > diff --git a/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> > new file mode 100644
> > index 0000000..fefc586
> > --- /dev/null
> > +++ b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> > @@ -0,0 +1,26 @@
> > +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
> > +From: Stoiko Ivanov <s.ivanov@proxmox.com>
> > +Date: Wed, 22 Jul 2020 12:17:24 +0200
> > +Subject: [PATCH lxc] apparmor: Allow ro remount of boot_id
> > +
> > +The rule added in 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 did not cover all
> > +necessary mount calls for /proc/sys/kernel/random/boot_id
> > +(in src/lxc/conf.c: lxc_setup_boot_id) - the ro remount is missing.
> > +
> > +Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> > +---
> > + config/apparmor/abstractions/start-container.in | 1 +
> > + 1 file changed, 1 insertion(+)
> > +
> > +diff --git a/config/apparmor/abstractions/start-container.in b/config/apparmor/abstractions/start-container.in
> > +index 9998f1121..9f64c2727 100644
> > +--- a/config/apparmor/abstractions/start-container.in
> > ++++ b/config/apparmor/abstractions/start-container.in
> > +@@ -22,6 +22,7 @@
> > +   mount -> /var/lib/lxc/{**,},
> > + 
> > +   mount /dev/.lxc-boot-id -> /proc/sys/kernel/random/boot_id,
> > ++  mount options=(ro, nosuid, nodev, noexec, remount, bind) -> /proc/sys/kernel/random/boot_id,
> > + 
> > +   # required for some pre-mount hooks
> > +   mount fstype=overlayfs,
> > diff --git a/debian/patches/series b/debian/patches/series
> > index ee20ef5..f588081 100644
> > --- a/debian/patches/series
> > +++ b/debian/patches/series
> > @@ -1,3 +1,4 @@
> >  pve/0001-PVE-Config-lxc.service-start-after-a-potential-syslo.patch
> >  pve/0002-PVE-Config-deny-rw-mounting-of-sys-and-proc.patch
> >  pve/0003-PVE-Config-attach-always-use-getent.patch
> > +pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> >   
> 

Re: [pve-devel] [PATCH lxc 2/2] apparmor: add rule for allowing remount of boot_id

[Thomas Lamprecht]

On 22.07.20 13:59, Stoiko Ivanov wrote:
> On Wed, 22 Jul 2020 13:51:19 +0200
> Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:
> 
>> On 22.07.20 13:05, Stoiko Ivanov wrote:
>>> commit 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 in upstream only partially
>>> fixes the apparmor deny for mounting boot_id (used for example for identifying
>>> different boots with `journalctl`) inside the container.
>>>
>>> Tested by editing the profile and replacing it disregarding the cache:
>>> `apparmor_parser -W -T -r /etc/apparmor.d/usr.bin.lxc-start`
>>>   
>>
>> was this proposed to upstream as pull request? Did not found it on the
>> LXC GitHub page.
> 
> sorry my phrasing in the cover-letter was misleading: I want to make a
> pull request upstream for this patch, after somebody else sanity-checks it
> -> if it looks ok to you - I'll open the PR.
> 

Haha, and I wanted the reverse: get upstream to review it with their
in-depth knowledge so that I can rely on that check ;-P

> 
>>
>>> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
>>> ---
>>>  ...apparmor-Allow-ro-remount-of-boot_id.patch | 26 +++++++++++++++++++
>>>  debian/patches/series                         |  1 +
>>>  2 files changed, 27 insertions(+)
>>>  create mode 100644 debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
>>>
>>> diff --git a/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
>>> new file mode 100644
>>> index 0000000..fefc586
>>> --- /dev/null
>>> +++ b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
>>> @@ -0,0 +1,26 @@
>>> +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
>>> +From: Stoiko Ivanov <s.ivanov@proxmox.com>
>>> +Date: Wed, 22 Jul 2020 12:17:24 +0200
>>> +Subject: [PATCH lxc] apparmor: Allow ro remount of boot_id
>>> +
>>> +The rule added in 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 did not cover all
>>> +necessary mount calls for /proc/sys/kernel/random/boot_id
>>> +(in src/lxc/conf.c: lxc_setup_boot_id) - the ro remount is missing.
>>> +
>>> +Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
>>> +---
>>> + config/apparmor/abstractions/start-container.in | 1 +
>>> + 1 file changed, 1 insertion(+)
>>> +
>>> +diff --git a/config/apparmor/abstractions/start-container.in b/config/apparmor/abstractions/start-container.in
>>> +index 9998f1121..9f64c2727 100644
>>> +--- a/config/apparmor/abstractions/start-container.in
>>> ++++ b/config/apparmor/abstractions/start-container.in
>>> +@@ -22,6 +22,7 @@
>>> +   mount -> /var/lib/lxc/{**,},
>>> + 
>>> +   mount /dev/.lxc-boot-id -> /proc/sys/kernel/random/boot_id,
>>> ++  mount options=(ro, nosuid, nodev, noexec, remount, bind) -> /proc/sys/kernel/random/boot_id,
>>> + 
>>> +   # required for some pre-mount hooks
>>> +   mount fstype=overlayfs,
>>> diff --git a/debian/patches/series b/debian/patches/series
>>> index ee20ef5..f588081 100644
>>> --- a/debian/patches/series
>>> +++ b/debian/patches/series
>>> @@ -1,3 +1,4 @@
>>>  pve/0001-PVE-Config-lxc.service-start-after-a-potential-syslo.patch
>>>  pve/0002-PVE-Config-deny-rw-mounting-of-sys-and-proc.patch
>>>  pve/0003-PVE-Config-attach-always-use-getent.patch
>>> +pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
>>>   
>>
> 

Re: [pve-devel] [PATCH lxc 2/2] apparmor: add rule for allowing remount of boot_id

[Stoiko Ivanov]

On Wed, 22 Jul 2020 14:09:09 +0200
Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:

> On 22.07.20 13:59, Stoiko Ivanov wrote:
> > On Wed, 22 Jul 2020 13:51:19 +0200
> > Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:
> >   
> >> On 22.07.20 13:05, Stoiko Ivanov wrote:  
> >>> commit 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 in upstream only partially
> >>> fixes the apparmor deny for mounting boot_id (used for example for identifying
> >>> different boots with `journalctl`) inside the container.
> >>>
> >>> Tested by editing the profile and replacing it disregarding the cache:
> >>> `apparmor_parser -W -T -r /etc/apparmor.d/usr.bin.lxc-start`
> >>>     
> >>
> >> was this proposed to upstream as pull request? Did not found it on the
> >> LXC GitHub page.  
> > 
> > sorry my phrasing in the cover-letter was misleading: I want to make a
> > pull request upstream for this patch, after somebody else sanity-checks it  
> > -> if it looks ok to you - I'll open the PR.  
> >   
> 
> Haha, and I wanted the reverse: get upstream to review it with their
> in-depth knowledge so that I can rely on that check ;-P
aye - makes sense - https://github.com/lxc/lxc/pull/3495 :)

> 
> >   
> >>  
> >>> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> >>> ---
> >>>  ...apparmor-Allow-ro-remount-of-boot_id.patch | 26 +++++++++++++++++++
> >>>  debian/patches/series                         |  1 +
> >>>  2 files changed, 27 insertions(+)
> >>>  create mode 100644 debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> >>>
> >>> diff --git a/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> >>> new file mode 100644
> >>> index 0000000..fefc586
> >>> --- /dev/null
> >>> +++ b/debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> >>> @@ -0,0 +1,26 @@
> >>> +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
> >>> +From: Stoiko Ivanov <s.ivanov@proxmox.com>
> >>> +Date: Wed, 22 Jul 2020 12:17:24 +0200
> >>> +Subject: [PATCH lxc] apparmor: Allow ro remount of boot_id
> >>> +
> >>> +The rule added in 863845075d3f77d27c91bd9f47d2f8ddc4867bd5 did not cover all
> >>> +necessary mount calls for /proc/sys/kernel/random/boot_id
> >>> +(in src/lxc/conf.c: lxc_setup_boot_id) - the ro remount is missing.
> >>> +
> >>> +Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
> >>> +---
> >>> + config/apparmor/abstractions/start-container.in | 1 +
> >>> + 1 file changed, 1 insertion(+)
> >>> +
> >>> +diff --git a/config/apparmor/abstractions/start-container.in b/config/apparmor/abstractions/start-container.in
> >>> +index 9998f1121..9f64c2727 100644
> >>> +--- a/config/apparmor/abstractions/start-container.in
> >>> ++++ b/config/apparmor/abstractions/start-container.in
> >>> +@@ -22,6 +22,7 @@
> >>> +   mount -> /var/lib/lxc/{**,},
> >>> + 
> >>> +   mount /dev/.lxc-boot-id -> /proc/sys/kernel/random/boot_id,
> >>> ++  mount options=(ro, nosuid, nodev, noexec, remount, bind) -> /proc/sys/kernel/random/boot_id,
> >>> + 
> >>> +   # required for some pre-mount hooks
> >>> +   mount fstype=overlayfs,
> >>> diff --git a/debian/patches/series b/debian/patches/series
> >>> index ee20ef5..f588081 100644
> >>> --- a/debian/patches/series
> >>> +++ b/debian/patches/series
> >>> @@ -1,3 +1,4 @@
> >>>  pve/0001-PVE-Config-lxc.service-start-after-a-potential-syslo.patch
> >>>  pve/0002-PVE-Config-deny-rw-mounting-of-sys-and-proc.patch
> >>>  pve/0003-PVE-Config-attach-always-use-getent.patch
> >>> +pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> >>>     
> >>  
> >   
> 

Re: [pve-devel] [PATCH lxc 0/2] fix apparmor rules and improve cgroupv2 experience

[Thomas Lamprecht]

On 22.07.20 13:05, Stoiko Ivanov wrote:
> This patchset addresses 2 minor inconveniences I ran into, while running my
> host with 'systemd.unified_cgroup_hierarchy=1':
> 
> * apparmor mount denies for '/proc/sys/kernel/random/boot_id' (this happens
>   irrespective of the cgroup-layout
> * having to add
>   `lxc.init.cmd: /lib/systemd/systemd systemd.unified_cgroup_hierarchy=1`
>   to all my container configs (for debian and arch containers at least
>   alpine runs without issues) - see [0] for a discussion of the topic
> 
> While investigating this I noticed that the fixes for both issues were already
> on upstream/master (with one small other fix in between) - so instead of
> cherry-picking both patches I fast-forwarded to the last needed commit.
> Glad to resend with the patches cherry-picked and added to our patchqueue.
> 
> I would probably submit the apparmor fix upstream (after a quick check by
> another set of eyes :)
> 
> [0] https://github.com/lxc/lxc/issues/3183
> 
> Stoiko Ivanov (2):
>   update lxc to include fixes for cgroupv2 setups
>   apparmor: add rule for allowing remount of boot_id
> 
>  ...apparmor-Allow-ro-remount-of-boot_id.patch | 26 +++++++++++++++++++
>  debian/patches/series                         |  1 +
>  lxc                                           |  2 +-
>  3 files changed, 28 insertions(+), 1 deletion(-)
>  create mode 100644 debian/patches/pve/0004-apparmor-Allow-ro-remount-of-boot_id.patch
> 

2/2 got merged into upstream and is available with 4.0.4, could you see
if we can seamlessly update from currently packaged 4.0.3 to 4.0.4?