From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pve-devel-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 3A3601FF168 for <inbox@lore.proxmox.com>; Tue, 18 Mar 2025 10:46:43 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5A2A01A0AB; Tue, 18 Mar 2025 10:46:31 +0100 (CET) Message-ID: <ec981a0e-31a3-40ac-8f7c-26d60c68b693@proxmox.com> Date: Tue, 18 Mar 2025 10:45:56 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta To: Raven King <thekingofravens@disroot.org>, Proxmox VE development discussion <pve-devel@lists.proxmox.com> References: <e0a39815-26cc-4530-9bcd-5e5e2f1f1ae4@disroot.org> Content-Language: en-GB, de-AT From: Thomas Lamprecht <t.lamprecht@proxmox.com> Autocrypt: addr=t.lamprecht@proxmox.com; keydata= xsFNBFsLjcYBEACsaQP6uTtw/xHTUCKF4VD4/Wfg7gGn47+OfCKJQAD+Oyb3HSBkjclopC5J uXsB1vVOfqVYE6PO8FlD2L5nxgT3SWkc6Ka634G/yGDU3ZC3C/7NcDVKhSBI5E0ww4Qj8s9w OQRloemb5LOBkJNEUshkWRTHHOmk6QqFB/qBPW2COpAx6oyxVUvBCgm/1S0dAZ9gfkvpqFSD 90B5j3bL6i9FIv3YGUCgz6Ue3f7u+HsEAew6TMtlt90XV3vT4M2IOuECG/pXwTy7NtmHaBQ7 UJBcwSOpDEweNob50+9B4KbnVn1ydx+K6UnEcGDvUWBkREccvuExvupYYYQ5dIhRFf3fkS4+ wMlyAFh8PQUgauod+vqs45FJaSgTqIALSBsEHKEs6IoTXtnnpbhu3p6XBin4hunwoBFiyYt6 YHLAM1yLfCyX510DFzX/Ze2hLqatqzY5Wa7NIXqYYelz7tXiuCLHP84+sV6JtEkeSUCuOiUY virj6nT/nJK8m0BzdR6FgGtNxp7RVXFRz/+mwijJVLpFsyG1i0Hmv2zTn3h2nyGK/I6yhFNt dX69y5hbo6LAsRjLUvZeHXpTU4TrpN/WiCjJblbj5um5eEr4yhcwhVmG102puTtuCECsDucZ jpKpUqzXlpLbzG/dp9dXFH3MivvfuaHrg3MtjXY1i+/Oxyp5iwARAQABzTNUaG9tYXMgTGFt cHJlY2h0IChBdXRoLTQpIDx0LmxhbXByZWNodEBwcm94bW94LmNvbT7CwY4EEwEIADgWIQQO R4qbEl/pah9K6VrTZCM6gDZWBgUCWwuNxgIbAwULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAK CRDTZCM6gDZWBm/jD/4+6JB2s67eaqoP6x9VGaXNGJPCscwzLuxDTCG90G9FYu29VcXtubH/ bPwsyBbNUQpqTm/s4XboU2qpS5ykCuTjqavrcP33tdkYfGcItj2xMipJ1i3TWvpikQVsX42R G64wovLs/dvpTYphRZkg5DwhgTmy3mRkmofFCTa+//MOcNOORltemp984tWjpR3bUJETNWpF sKGZHa3N4kCNxb7A+VMsJZ/1gN3jbQbQG7GkJtnHlWkw9rKCYqBtWrnrHa4UAvSa9M/XCIAB FThFGqZI1ojdVlv5gd6b/nWxfOPrLlSxbUo5FZ1i/ycj7/24nznW1V4ykG9iUld4uYUY86bB UGSjew1KYp9FmvKiwEoB+zxNnuEQfS7/Bj1X9nxizgweiHIyFsRqgogTvLh403QMSGNSoArk tqkorf1U+VhEncIn4H3KksJF0njZKfilrieOO7Vuot1xKr9QnYrZzJ7m7ZxJ/JfKGaRHXkE1 feMmrvZD1AtdUATZkoeQtTOpMu4r6IQRfSdwm/CkppZXfDe50DJxAMDWwfK2rr2bVkNg/yZI tKLBS0YgRTIynkvv0h8d9dIjiicw3RMeYXyqOnSWVva2r+tl+JBaenr8YTQw0zARrhC0mttu cIZGnVEvQuDwib57QLqMjQaC1gazKHvhA15H5MNxUhwm229UmdH3KM7BTQRbC43GARAAyTkR D6KRJ9Xa2fVMh+6f186q0M3ni+5tsaVhUiykxjsPgkuWXWW9MbLpYXkzX6h/RIEKlo2BGA95 QwG5+Ya2Bo3g7FGJHAkXY6loq7DgMp5/TVQ8phsSv3WxPTJLCBq6vNBamp5hda4cfXFUymsy HsJy4dtgkrPQ/bnsdFDCRUuhJHopnAzKHN8APXpKU6xV5e3GE4LwFsDhNHfH/m9+2yO/trcD txSFpyftbK2gaMERHgA8SKkzRhiwRTt9w5idOfpJVkYRsgvuSGZ0pcD4kLCOIFrer5xXudk6 NgJc36XkFRMnwqrL/bB4k6Pi2u5leyqcXSLyBgeHsZJxg6Lcr2LZ35+8RQGPOw9C0ItmRjtY ZpGKPlSxjxA1WHT2YlF9CEt3nx7c4C3thHHtqBra6BGPyW8rvtq4zRqZRLPmZ0kt/kiMPhTM 8wZAlObbATVrUMcZ/uNjRv2vU9O5aTAD9E5r1B0dlqKgxyoImUWB0JgpILADaT3VybDd3C8X s6Jt8MytUP+1cEWt9VKo4vY4Jh5vwrJUDLJvzpN+TsYCZPNVj18+jf9uGRaoK6W++DdMAr5l gQiwsNgf9372dbMI7pt2gnT5/YdG+ZHnIIlXC6OUonA1Ro/Itg90Q7iQySnKKkqqnWVc+qO9 GJbzcGykxD6EQtCSlurt3/5IXTA7t6sAEQEAAcLBdgQYAQgAIBYhBA5HipsSX+lqH0rpWtNk IzqANlYGBQJbC43GAhsMAAoJENNkIzqANlYGD1sP/ikKgHgcspEKqDED9gQrTBvipH85si0j /Jwu/tBtnYjLgKLh2cjv1JkgYYjb3DyZa1pLsIv6rGnPX9bH9IN03nqirC/Q1Y1lnbNTynPk IflgvsJjoTNZjgu1wUdQlBgL/JhUp1sIYID11jZphgzfDgp/E6ve/8xE2HMAnf4zAfJaKgD0 F+fL1DlcdYUditAiYEuN40Ns/abKs8I1MYx7Yglu3RzJfBzV4t86DAR+OvuF9v188WrFwXCS RSf4DmJ8tntyNej+DVGUnmKHupLQJO7uqCKB/1HLlMKc5G3GLoGqJliHjUHUAXNzinlpE2Vj C78pxpwxRNg2ilE3AhPoAXrY5qED5PLE9sLnmQ9AzRcMMJUXjTNEDxEYbF55SdGBHHOAcZtA kEQKub86e+GHA+Z8oXQSGeSGOkqHi7zfgW1UexddTvaRwE6AyZ6FxTApm8wq8NT2cryWPWTF BDSGB3ujWHMM8ERRYJPcBSjTvt0GcEqnd+OSGgxTkGOdufn51oz82zfpVo1t+J/FNz6MRMcg 8nEC+uKvgzH1nujxJ5pRCBOquFZaGn/p71Yr0oVitkttLKblFsqwa+10Lt6HBxm+2+VLp4Ja 0WZNncZciz3V3cuArpan/ZhhyiWYV5FD0pOXPCJIx7WS9PTtxiv0AOS4ScWEUmBxyhFeOpYa DrEx In-Reply-To: <e0a39815-26cc-4530-9bcd-5e5e2f1f1ae4@disroot.org> X-SPAM-LEVEL: Spam detection results: 0 AWL -0.040 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [pve-devel] Proposal For Podman Container Support X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/> List-Post: <mailto:pve-devel@lists.proxmox.com> List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe> Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com> Hi Raven King, I want to say thanks up-front for trying to improve on of our open source projects and reaching out upfront for doing so, highly appreciated. Am 13.03.25 um 19:03 schrieb Raven King: > This is my first time writing to this mailing list. I have never > contributed to proxmox but I would like to try and write a feature that > allows native container support (not inside an LXC or VM). FWIW, LXC definitively are "native containers", it's less confusing to use application containers (like OCI conform ones) and system containers (running a full distro), as LXC can be used for both, i.e. docker used LXC for a while for isolating application CTs. > My goal would be that you could manage those containers much like > LXC/VMs with similar UI behavior (resource usage views, easy access to > container console, and resource sharing). > Its a large undertaking, and I would probably want to get a little > experience with the proxmox codebase first. I think we should take a step back and not focus on integrating podman too much, but rather about adding support directly in our existing container toolkit. Actually I'm pondering about adding support for the OCI runtime spec [0] and maybe also the OCI image spec [1] over the last years and had some light talks with developers about that, so I think you and I agree on the end goal already, but not on the road or method to get there. [0]: https://github.com/opencontainers/runtime-spec [1]: https://github.com/opencontainers/image-spec > > *Why do this?* > 1. It is parroted by users frequently. Just look up "run docker in > proxmox" and you will see dozens. > 2. It would add a major use case to proxmox. > 3. For me personally, it removes a major pain point of using > proxmox, which is setting up an LXC to then share resources with to then > setup a docker image to then share resources with. > Or using docker directly and tearing my hair out as it > magically breaks all my proxmox network config. The why's are all fine, the reason it does not exist yet is not because we saw no reason, but rather because there are good workarounds that can be used and correctly implementing this into our container runtime while ensuring as much as possible is shared with our existing implementation. > > *Why Podman?* > 1. Easy enough to use. > 2. Packaging. The support in debian is straightforward and won't > confuse anyone. This means the project won't have to maintain podman > itself in any way. > 3. Security. Podman needs limited privileges to operate compared to > docker. This makes it easier to mesh with things such as user accounts. > 4. Interop. It easily goes to/from kubernetes, which can help in > some enterprise use cases. Also doesn't interact in ways that break > existing pve config mechanisms. > > *What does podman offer an LXC doesn't?* > 1. Easy deployment, you can just pull images that someone prebuilt > for a purpose, including most docker images. > 2. Directly sharing a host directory (not a whole drive) such as > single zfs datastore. While achievable in LXC, you have to do a bunch of > user mapping and the setup is rather involved. You're mixing things here, LXC is a lower-level technology, it simply does not care about image management and certainly does not limit PVE on sharing CT and/or host directories at all. LXC rather provide building blocks that can definitively be used to support these things. > *What drawbacks have I considered? > *1. Using privileged ports in a podman container is a little tricky > without root. Proxmox mostly runs as root though, so this is really only > a problem for secondary users. > 2. I will take a lot of work to ensure the networking works in a way > consistent with other networking in proxmox. > 3. Increase support burden as users who aren't entirely familiar with > docker/podman containers ask questions that could be answered through > research. > 4. Some services people might want to run, such as nginx proxy manager, > are gonna be very hard to use in this way due to number 1. IMO above are not that significant but rather minor/mid-level technical hurdles that can be overcome, the biggest drawbacks of using podman directly are IMO rather: - two CT stack we need to support, ours and a third-party one - dependency on third party developers and a programming language (Go) we do not use at all in any of our projects. - while great software, it does not align _that_ well with Proxmox VE's ways of things, thus a clean and good integration that feels native to PVE, and not just tacked on, is IMO rather hard to do. That all means quite some high permanent maintenance cost, which is what would have to bear, so I'm rather opposed to it, at least without some concrete plan of someone having intermediate+ experience with not just working with PVE but also internal development, as otherwise it's IMO too hard to ensure above concerns are unfounded or not relevant. In general, I'd highly favor a native implementation that we have control over and can neatly integrate into Proxmox VE and all its features (SDN, backup, HA that gets extended by orchestration ...), and given that there are specifications for what we want to support here, that should be (hopefully) doable without extreme effort, and ideally with not much more effort than integrating complete solutions like podman, at least if one also minds the long term maintenance cost. > I am writing to the mailing list before even beginning on this endeavor > to get several questions answered: > > 1. Do y'all have any general tips and pointers about navigating and > working with the proxmox codebase? There is some basic info here: https://pve.proxmox.com/wiki/Developer_Documentation Definitively does not cover all the things though, but basic patch handling should be described. > 2. Where is a good list to grab bugs to get familiar with proxmox > structure and what functionality is where? I have some hardware, but I > am not capable of testing stuff like multi-gpu setups. > I see > https://bugzilla.proxmox.com/describecomponents.cgi?product=pve but > there is a lot of components to proxmox. I have a hard time picking a > spot to start. We try to add a "start-contributing" tag to some issue requests, i.e.: https://bugzilla.proxmox.com/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=UNDECIDED&bug_status=REOPENED&bug_status=MORE%20INFO%20NEEDED&bug_status=POSTPONED&list_id=5791&longdesc=\btag%3A\s*start-contributing&longdesc_type=regexp&query_format=advanced&resolution=--- This can be filtered for the "container" component, albeit I did not ensured the three issues that come up are valid: https://bugzilla.proxmox.com/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=MORE%20INFO%20NEEDED&bug_status=UNDECIDED&bug_status=POSTPONED&component=Container%2FLXC&list_id=50251&longdesc=%5Cbtag%3A%5Cs%2Astart-contributing&longdesc_type=regexp&order=Importance&product=pve&query_format=advanced&resolution=--- > 3. Are there any major drawbacks to container support that need > consideration? Just to ensure we speak of the same: Containers are already supported in general, albeit our runtime that wraps LXC and co is targeting system containers, not application ones. > 4. Are there specific drawbacks to podman that need consideration? See above. > 5. Anything else I am overlooking with this idea? The way I'd get started is evaluating the OCI specs, pve-container and potentially also existing runtimes that implement the OCI specs. For then implementing the spec and integrating that into pve-container we would definitively open, even lightly encouraging, doing so in rust, maybe at least the lower level building blocks for understanding/parsing formats as defined in the OCI specs. Then use perlmod [2] to integrate that rust modules into the existing pve-container Perl based code. You could also just stay in perl, that would be fine for us, but in general we try to use rust if possible for new (bigger) features and also find that it provides us with a lot of guarantees and modern language features to make one lives easier in the long term. [2]: https://git.proxmox.com/?p=perlmod.git;a=blob;f=README.md;hb=HEAD That's naturally a lot to ask for a new contributor, but if it was easy it would have been already done, and we simply want to favor native and well integrated solutions to avoid external dependencies, of which drawbacks we had to deal with too much in the past, so we're pretty much set on that part. FWIW, I directly CC'd one developer I talked lightly over adding OCI support to PVE, maybe he got some time to think over this and spearhead the initial effort. If you're still interested into tackling this we naturally will try to help you on any specific question, I still appreciate you wanting to move this forward, but I also wanted to manage expectations, as this might be quite the task, especially for one not accustomed to our project and its code basis. best regards Thomas _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel