From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 8E7D61FF187 for ; Mon, 8 Sep 2025 17:15:43 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 6DA49161B0; Mon, 8 Sep 2025 17:15:44 +0200 (CEST) Message-ID: <1ce0be28-beda-4a3b-a8be-858b41cca332@proxmox.com> Date: Mon, 8 Sep 2025 17:15:39 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: pve-devel@lists.proxmox.com References: <20250709123435.64796-1-f.schauer@proxmox.com> Content-Language: en-US From: Filip Schauer In-Reply-To: <20250709123435.64796-1-f.schauer@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1757344517390 X-SPAM-LEVEL: Spam detection results: 0 AWL -1.638 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LOTSOFHASH 0.25 Emails with lots of hash-like gibberish RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLACK 3 Contains an URL listed in the URIBL blacklist [plugin.pm] Subject: Re: [pve-devel] superseded: [PATCH container/docs/lxcfs/manager/proxmox{, -perl-rs}/storage v3 00/13] support OCI images as container templates X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" Superseded by: https://lore.proxmox.com/pve-devel/20250908150224.155373-1-f.schauer@proxmox.com/ On 09/07/2025 14:34, Filip Schauer wrote: > Add basic support for OCI (Open Container Initiative) images [0] as > container templates. > > An OCI image can be obtained from a registry like Docker Hub. This patch > series does not implement the OCI Distribution Spec, so this requires > external tools. > > Either using Docker: > > ``` > $ docker pull httpd > $ docker save httpd > httpd.tar > ``` > > Or using Podman: > When using Podman, the format needs to be explicitly specified, > otherwise it defaults to docker-archive. > > ``` > $ podman pull httpd > $ podman save --format=oci-archive httpd > httpd.tar > ``` > > Or using skopeo: > > ``` > $ skopeo copy docker://httpd:latest oci-archive:httpd.tar:latest > ``` > > The tarball can be uploaded to a storage as a container template and > then used during container creation. It is automatically detected that > the container template is an OCI image. The resulting container still > uses the existing LXC framework. > > # Dependencies: > > To be able to build `proxmox-oci`, the `oci-spec` crate is required as a > dependency. A patch from Christoph [1] packages the `oci-spec` crate as > a deb package. Alternatively if the `oci-spec` crate is not yet > packaged, it can be downloaded from crates.io. > > Here is a little script to download the `oci-spec` crate along with its > dependencies: > > ```sh > download_crate() { > CRATE_NAME=$1 > CRATE_VERSION=$2 > CRATE_SHA256=$3 > > wget https://crates.io/api/v1/crates/$CRATE_NAME/$CRATE_VERSION/download > > COMPUTED_SHA256=$(sha256sum download | awk '{ print $1 }') > if [ "$COMPUTED_SHA256" != "$CRATE_SHA256" ]; then > echo "Checksum mismatch"; exit 1 > fi > > tar -xf download > rm download > mv $CRATE_NAME-$CRATE_VERSION /usr/share/cargo/registry/ > echo "{\"package\":\"$CRATE_SHA256\",\"files\":{}}" > /usr/share/cargo/registry/$CRATE_NAME-$CRATE_VERSION/.cargo-checksum.json > } > > download_crate strsim 0.11.1 7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f > download_crate ident_case 1.0.1 b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39 > download_crate darling_macro 0.20.11 fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead > download_crate darling_core 0.20.11 0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e > download_crate darling 0.20.11 fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee > download_crate proc-macro-error-attr2 2.0.0 96de42df36bb9bba5542fe9f1a054b8cc87e172759a1868aa05c1f3acc89dfc5 > download_crate derive_builder_core 0.20.2 2d5bcf7b024d6835cfb3d473887cd966994907effbe9227e8c8219824d06c4e8 > download_crate thiserror-impl 2.0.0 22efd00f33f93fa62848a7cab956c3d38c8d43095efda1decfc2b3a5dc0b8972 > download_crate rustversion 1.0.20 eded382c5f5f786b989652c49544c4877d9f015cc22e145a5ea8ea66c2921cd2 > download_crate heck 0.5.0 2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea > download_crate proc-macro-error2 2.0.1 11ec05c52be0a07b08061f7dd003e7d7092e0472bc731b4af7bb1ef876109802 > download_crate derive_builder_macro 0.20.2 ab63b0e2bf4d5928aff72e83a7dace85d7bba5fe12dcc3c5a572d78caffd3f3c > download_crate thiserror 2.0.0 15291287e9bff1bc6f9ff3409ed9af665bec7a5fc8ac079ea96be07bca0e2668 > download_crate strum_macros 0.27.1 c77a8c5abcaf0f9ce05d62342b7d298c346515365c36b673df4ebe3ced01fde8 > download_crate strum 0.27.1 f64def088c51c9510a8579e3c5d67c65349dcf755e5479ad3d010aa6454e2c32 > download_crate getset 0.1.5 f3586f256131df87204eb733da72e3d3eb4f343c639f4b7be279ac7c48baeafe > download_crate derive_builder 0.20.2 507dfb09ea8b7fa618fcf76e953f4f5e192547945816d5358edffe39f6f94947 > download_crate oci-spec 0.8.1 57e9beda9d92fac7bf4904c34c83340ef1024159faee67179a04e0277523da33 > ``` > > Since librust-oci-spec-dev is in the proxmox-oci/debian/control file, a > dummy package needs to be installed, so dpkg-checkbuilddeps does not > complain. > > dummy_librust_oci_spec.equivs: > > ``` > Package: librust-oci-spec-dev > Version: 0.8.1 > Provides: librust-oci-spec-0.8+default-dev (= 0.8.1-1) > ``` > > ``` > $ equivs-build dummy_librust_oci_spec.equivs > $ dpkg -i ./librust-oci-spec-dev_0.8.1_all.deb > ``` > > # Build & install order: > > OCI image support: > 1. proxmox > 2. proxmox-perl-rs > 3. lxcfs > 4. pve-container > > .tar container template support: > 1. pve-storage > 2. pve-manager > > In no particular order: > * pve-docs > > FYI: I am on vacation. If anyone wants make changes to my patches, you > are free to do so. I will be back on the 28th. > > [0] https://github.com/opencontainers/image-spec/blob/main/spec.md > [1] https://lore.proxmox.com/pve-devel/20250606103719.533030-2-c.heiss@proxmox.com/ > > Changed since v2: > * lxcfs: lxc.mount.hook: override env variables from container config > * pve-container: rebase onto newest master (5a8b3f962f16) and re-format > with proxmox-perltidy > * pve-container: check whether archive is an OCI image before trying to > parse it as one > * pve-container: add an "ipmanagehost" property to pct.conf to indicate > that network interface IP configuration should be handled by the host. > * pve-container: manage_dhclient: add a FIXME comment regarding the > AppArmor profile: "use a profile that confines writes to > /var/lib/lxc/$vmid and rootfs" > * pve-container: kill_dhclients: untaint pid from pidfile > * pve-container: fix manage_dhclient called with 'stop' instead of > 'start' for IPv6 when container is started > * pve-docs: add OCI image docs > * proxmox-perl-rs: rebase onto newest master (3809f1229602) > * proxmox-perl-rs: forward all errors to Perl > * proxmox-perl-rs: remove oci-spec dependency > * pve-manager: rebase onto newest master (84b22751f211) and re-format > * proxmox: io: introduce RangeReader for bounded reads > * proxmox: oci: remove reachable unwraps & refactor code > * proxmox: oci: increase hasher buffer size from 4096 to 32768 (matching > internal sha2::Digest buffering) > * proxmox: oci: preserve permissions and xattrs during rootfs extraction > * proxmox: oci: handle whiteouts & opaque whiteouts > * pve-storage: Modify VZTMPL_EXT_RE_1 regex to put "tar" into capture > group when matching on a .tar file. > > Changed since v1: > * Fix entrypoint command missing Cmd > * Set lxc.signal.halt according to StopSignal (Fixes container shutdown) > * setup: Ensure that both /etc/systemd/network and > /etc/systemd/system-preset exist before writing files into them. > * ui: storage upload: accept *.tar files as vztmpl > * proxmox-perl-rs: rebase on latest master (3d9806cb3c7f) > * proxmox-perl-rs: add new dependencies to debian/control > * proxmox-oci: refactor errors and use `thiserror` to avoid boilerplate > > proxmox: > > Filip Schauer (2): > io: introduce RangeReader for bounded reads > add proxmox-oci crate > > Cargo.toml | 1 + > proxmox-io/src/lib.rs | 3 + > proxmox-io/src/range_reader.rs | 94 ++++++++++ > proxmox-oci/Cargo.toml | 22 +++ > proxmox-oci/debian/changelog | 5 + > proxmox-oci/debian/control | 45 +++++ > proxmox-oci/debian/debcargo.toml | 7 + > proxmox-oci/src/lib.rs | 283 +++++++++++++++++++++++++++++++ > proxmox-oci/src/oci_tar_image.rs | 145 ++++++++++++++++ > 9 files changed, 605 insertions(+) > create mode 100644 proxmox-io/src/range_reader.rs > create mode 100644 proxmox-oci/Cargo.toml > create mode 100644 proxmox-oci/debian/changelog > create mode 100644 proxmox-oci/debian/control > create mode 100644 proxmox-oci/debian/debcargo.toml > create mode 100644 proxmox-oci/src/lib.rs > create mode 100644 proxmox-oci/src/oci_tar_image.rs > > > proxmox-perl-rs: > > Filip Schauer (1): > add Perl mapping for OCI container image parser/extractor > > pve-rs/Cargo.toml | 1 + > pve-rs/Makefile | 1 + > pve-rs/debian/control | 1 + > pve-rs/src/bindings/mod.rs | 3 +++ > pve-rs/src/bindings/oci.rs | 21 +++++++++++++++++++++ > 5 files changed, 27 insertions(+) > create mode 100644 pve-rs/src/bindings/oci.rs > > > pve-container: > > Filip Schauer (6): > add support for OCI images as container templates > config: add entrypoint parameter > configure static IP in LXC config for custom entrypoint > setup: debian: create /etc/network path if missing > setup: recursively mkdir /etc/systemd/{network,system-preset} > manage DHCP for containers with custom entrypoint > > src/PVE/API2/LXC.pm | 100 ++++++++++++++++++++++++++++++---- > src/PVE/LXC.pm | 106 ++++++++++++++++++++++++++++++++++-- > src/PVE/LXC/Config.pm | 31 ++++++++++- > src/PVE/LXC/Setup/Base.pm | 3 +- > src/PVE/LXC/Setup/Debian.pm | 1 + > 5 files changed, 224 insertions(+), 17 deletions(-) > > > lxcfs: > > Filip Schauer (1): > lxc.mount.hook: override env variables from container config > > .../patches/reset-path-to-host-defaults.patch | 37 +++++++++++++++++++ > debian/patches/series | 1 + > 2 files changed, 38 insertions(+) > create mode 100644 debian/patches/reset-path-to-host-defaults.patch > > > pve-storage: > > Filip Schauer (1): > allow .tar container templates > > src/PVE/Storage.pm | 2 +- > src/PVE/Storage/Plugin.pm | 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > > > pve-manager: > > Filip Schauer (1): > ui: storage upload: accept *.tar files as vztmpl > > www/manager6/window/UploadToStorage.js | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > > pve-docs: > > Filip Schauer (1): > ct: add OCI image docs > > pct.adoc | 72 +++++++++++++++++++++++++++++++++++++++++++++++++------- > 1 file changed, 64 insertions(+), 8 deletions(-) > > > Summary over all repositories: > 25 files changed, 961 insertions(+), 28 deletions(-) > _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel