[pve-devel] [PATCH container/proxmox{, -perl-rs}/storage 0/9] support OCI images as container templates

[pve-devel] [PATCH container/proxmox{, -perl-rs}/storage 0/9] support OCI images as container templates

[Filip Schauer]

Add basic support for OCI (Open Container Initiative) images [0] as
container templates.

An OCI image can be for example obtained from Docker Hub:

```
$ docker pull httpd
$ docker save httpd > httpd.tar
```

The tarball can be uploaded to a storage as a container template and
then used during container creation. It is automatically detected that
the container template is an OCI image. The resulting container still
uses the existing LXC framework.

# Dependencies:

Since the `oci-spec` crate is not in any Debian repository at the time
of writing, it needs to be downloaded from crates.io, in order to be
able to build `proxmox-oci`.

Here is a little script to download the `oci-spec` crate along with its
dependencies:

```sh
download_crate() {
    CRATE_NAME=$1
    CRATE_VERSION=$2
    CRATE_SHA256=$3

    wget https://crates.io/api/v1/crates/$CRATE_NAME/$CRATE_VERSION/download

    COMPUTED_SHA256=$(sha256sum download | awk '{ print $1 }')
    if [ "$COMPUTED_SHA256" != "$CRATE_SHA256" ]; then
        echo "Checksum mismatch"; exit 1
    fi

    tar -xf download
    rm download
    mv $CRATE_NAME-$CRATE_VERSION /usr/share/cargo/registry/
    echo "{\"package\":\"$CRATE_SHA256\",\"files\":{}}" > /usr/share/cargo/registry/$CRATE_NAME-$CRATE_VERSION/.cargo-checksum.json
}

download_crate strsim 0.11.1 7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f
download_crate ident_case 1.0.1 b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39
download_crate darling_macro 0.20.11 fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead
download_crate darling_core 0.20.11 0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e
download_crate darling 0.20.11 fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee
download_crate proc-macro-error-attr2 2.0.0 96de42df36bb9bba5542fe9f1a054b8cc87e172759a1868aa05c1f3acc89dfc5
download_crate derive_builder_core 0.20.2 2d5bcf7b024d6835cfb3d473887cd966994907effbe9227e8c8219824d06c4e8
download_crate thiserror-impl 2.0.0 22efd00f33f93fa62848a7cab956c3d38c8d43095efda1decfc2b3a5dc0b8972
download_crate rustversion 1.0.20 eded382c5f5f786b989652c49544c4877d9f015cc22e145a5ea8ea66c2921cd2
download_crate heck 0.5.0 2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea
download_crate proc-macro-error2 2.0.1 11ec05c52be0a07b08061f7dd003e7d7092e0472bc731b4af7bb1ef876109802
download_crate derive_builder_macro 0.20.2 ab63b0e2bf4d5928aff72e83a7dace85d7bba5fe12dcc3c5a572d78caffd3f3c
download_crate thiserror 2.0.0 15291287e9bff1bc6f9ff3409ed9af665bec7a5fc8ac079ea96be07bca0e2668
download_crate strum_macros 0.27.1 c77a8c5abcaf0f9ce05d62342b7d298c346515365c36b673df4ebe3ced01fde8
download_crate strum 0.27.1 f64def088c51c9510a8579e3c5d67c65349dcf755e5479ad3d010aa6454e2c32
download_crate getset 0.1.5 f3586f256131df87204eb733da72e3d3eb4f343c639f4b7be279ac7c48baeafe
download_crate derive_builder 0.20.2 507dfb09ea8b7fa618fcf76e953f4f5e192547945816d5358edffe39f6f94947
download_crate oci-spec 0.8.1 57e9beda9d92fac7bf4904c34c83340ef1024159faee67179a04e0277523da33
```

Since librust-oci-spec-dev is in the proxmox-oci/debian/control file, a
dummy package needs to be installed, so dpkg-checkbuilddeps does not
complain.

dummy_librust_oci_spec.equivs:

```
Package: librust-oci-spec-dev
Version: 0.8.1
Provides: librust-oci-spec-0.8+default-dev (= 0.8.1-1)
```

```
$ equivs-build dummy_librust_oci_spec.equivs
$ dpkg -i ./librust-oci-spec-dev_0.8.1_all.deb
```

# Build & install order:

1. proxmox
2. proxmox-perl-rs
3. pve-container
*  pve-storage (no particular order there)

[0] https://github.com/opencontainers/image-spec/blob/main/spec.md

proxmox:

Filip Schauer (1):
  add proxmox-oci crate

 Cargo.toml                       |   1 +
 proxmox-oci/Cargo.toml           |  21 ++++
 proxmox-oci/debian/changelog     |   5 +
 proxmox-oci/debian/control       |  45 ++++++++
 proxmox-oci/debian/debcargo.toml |   7 ++
 proxmox-oci/src/lib.rs           | 165 +++++++++++++++++++++++++++++
 proxmox-oci/src/oci_tar_image.rs | 173 +++++++++++++++++++++++++++++++
 7 files changed, 417 insertions(+)
 create mode 100644 proxmox-oci/Cargo.toml
 create mode 100644 proxmox-oci/debian/changelog
 create mode 100644 proxmox-oci/debian/control
 create mode 100644 proxmox-oci/debian/debcargo.toml
 create mode 100644 proxmox-oci/src/lib.rs
 create mode 100644 proxmox-oci/src/oci_tar_image.rs


proxmox-perl-rs:

Filip Schauer (1):
  add Perl mapping for OCI container image parser

 pve-rs/Cargo.toml |  2 ++
 pve-rs/Makefile   |  1 +
 pve-rs/src/lib.rs |  1 +
 pve-rs/src/oci.rs | 20 ++++++++++++++++++++
 4 files changed, 24 insertions(+)
 create mode 100644 pve-rs/src/oci.rs


pve-storage:

Filip Schauer (1):
  allow .tar container templates

 src/PVE/Storage.pm        | 2 +-
 src/PVE/Storage/Plugin.pm | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)


pve-container:

Filip Schauer (6):
  config: whitelist lxc.init.cwd
  add support for OCI images as container templates
  config: add entrypoint parameter
  configure static IP in LXC config for custom entrypoint
  setup: debian: create /etc/network path if missing
  manage DHCP for containers with custom entrypoint

 src/PVE/API2/LXC.pm         | 53 ++++++++++++++++++++--
 src/PVE/LXC.pm              | 88 ++++++++++++++++++++++++++++++++++---
 src/PVE/LXC/Config.pm       | 19 +++++++-
 src/PVE/LXC/Setup/Debian.pm |  1 +
 4 files changed, 152 insertions(+), 9 deletions(-)


Summary over all repositories:
  17 files changed, 595 insertions(+), 11 deletions(-)

-- 
Generated by git-murpp 0.6.0


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH proxmox 1/9] add proxmox-oci crate

[Filip Schauer]

This crate can parse and extract an OCI image bundled as a tar archive.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 Cargo.toml                       |   1 +
 proxmox-oci/Cargo.toml           |  21 ++++
 proxmox-oci/debian/changelog     |   5 +
 proxmox-oci/debian/control       |  45 ++++++++
 proxmox-oci/debian/debcargo.toml |   7 ++
 proxmox-oci/src/lib.rs           | 165 +++++++++++++++++++++++++++++
 proxmox-oci/src/oci_tar_image.rs | 173 +++++++++++++++++++++++++++++++
 7 files changed, 417 insertions(+)
 create mode 100644 proxmox-oci/Cargo.toml
 create mode 100644 proxmox-oci/debian/changelog
 create mode 100644 proxmox-oci/debian/control
 create mode 100644 proxmox-oci/debian/debcargo.toml
 create mode 100644 proxmox-oci/src/lib.rs
 create mode 100644 proxmox-oci/src/oci_tar_image.rs

diff --git a/Cargo.toml b/Cargo.toml
index 71763c5a..6f20ab18 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -25,6 +25,7 @@ members = [
     "proxmox-metrics",
     "proxmox-network-api",
     "proxmox-notify",
+    "proxmox-oci",
     "proxmox-openid",
     "proxmox-product-config",
     "proxmox-rest-server",
diff --git a/proxmox-oci/Cargo.toml b/proxmox-oci/Cargo.toml
new file mode 100644
index 00000000..77545b03
--- /dev/null
+++ b/proxmox-oci/Cargo.toml
@@ -0,0 +1,21 @@
+[package]
+name = "proxmox-oci"
+description = "OCI image parsing and extraction"
+version = "0.1.0"
+
+authors.workspace = true
+edition.workspace = true
+exclude.workspace = true
+homepage.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version.workspace = true
+
+[dependencies]
+flate2.workspace = true
+oci-spec = "0.8.1"
+sha2 = "0.10"
+tar.workspace = true
+zstd.workspace = true
+
+proxmox-io.workspace = true
diff --git a/proxmox-oci/debian/changelog b/proxmox-oci/debian/changelog
new file mode 100644
index 00000000..754d06c1
--- /dev/null
+++ b/proxmox-oci/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-oci (0.1.0-1) bookworm; urgency=medium
+
+  * Initial release.
+
+ -- Proxmox Support Team <support@proxmox.com>  Mon, 28 Apr 2025 12:34:56 +0200
diff --git a/proxmox-oci/debian/control b/proxmox-oci/debian/control
new file mode 100644
index 00000000..b2317c8e
--- /dev/null
+++ b/proxmox-oci/debian/control
@@ -0,0 +1,45 @@
+Source: rust-proxmox-oci
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.82) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-flate2-1+default-dev <!nocheck>,
+ librust-oci-spec-0.8+default-dev (>= 0.8.1-~~) <!nocheck>,
+ librust-proxmox-io-1+default-dev (>= 1.1.0-~~) <!nocheck>,
+ librust-sha2-0.10+default-dev <!nocheck>,
+ librust-tar-0.4+default-dev <!nocheck>,
+ librust-zstd-0.12+bindgen-dev <!nocheck>,
+ librust-zstd-0.12+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.0
+Vcs-Git: git://git.proxmox.com/git/proxmox.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: proxmox-oci
+Rules-Requires-Root: no
+
+Package: librust-proxmox-oci-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-flate2-1+default-dev,
+ librust-oci-spec-0.8+default-dev (>= 0.8.1-~~),
+ librust-proxmox-io-1+default-dev (>= 1.1.0-~~),
+ librust-sha2-0.10+default-dev,
+ librust-tar-0.4+default-dev,
+ librust-zstd-0.12+bindgen-dev,
+ librust-zstd-0.12+default-dev
+Provides:
+ librust-proxmox-oci+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0-dev (= ${binary:Version}),
+ librust-proxmox-oci-0+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1.0-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1.0+default-dev (= ${binary:Version})
+Description: OCI image parsing and extraction - Rust source code
+ Source code for Debianized Rust crate "proxmox-oci"
diff --git a/proxmox-oci/debian/debcargo.toml b/proxmox-oci/debian/debcargo.toml
new file mode 100644
index 00000000..b7864cdb
--- /dev/null
+++ b/proxmox-oci/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox.git"
diff --git a/proxmox-oci/src/lib.rs b/proxmox-oci/src/lib.rs
new file mode 100644
index 00000000..57bd48b4
--- /dev/null
+++ b/proxmox-oci/src/lib.rs
@@ -0,0 +1,165 @@
+use flate2::read::GzDecoder;
+use oci_spec::image::{Arch, Config, ImageConfiguration, ImageManifest, MediaType};
+use oci_spec::OciSpecError;
+use oci_tar_image::OciTarImage;
+use sha2::{Digest, Sha256};
+use std::collections::HashMap;
+use std::fs::File;
+use std::io::{Read, Seek};
+use std::path::PathBuf;
+use std::str::FromStr;
+use tar::Archive;
+
+pub mod oci_tar_image;
+
+/// Build a mapping from uncompressed layer digests (as found in the image config's `rootfs.diff_ids`)
+/// to their corresponding compressed-layer digests (i.e. the filenames under `blobs/<algorithm>/<digest>`)
+fn build_layer_map<R: Read + Seek>(
+    oci_tar_image: &mut OciTarImage<R>,
+    image_manifest: &ImageManifest,
+) -> HashMap<oci_spec::image::Digest, oci_spec::image::Descriptor> {
+    let mut layer_mapping = HashMap::new();
+
+    for layer in image_manifest.layers() {
+        match layer.media_type() {
+            MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => {
+                layer_mapping.insert(layer.digest().clone(), layer.clone());
+            }
+            MediaType::ImageLayerGzip
+            | MediaType::ImageLayerNonDistributableGzip
+            | MediaType::ImageLayerZstd
+            | MediaType::ImageLayerNonDistributableZstd => {
+                let compressed_blob = oci_tar_image.open_blob(layer.digest()).unwrap();
+                let mut decoder: Box<dyn Read> = match layer.media_type() {
+                    MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
+                        Box::new(GzDecoder::new(compressed_blob))
+                    }
+                    MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
+                        Box::new(zstd::Decoder::new(compressed_blob).unwrap())
+                    }
+                    _ => unreachable!(),
+                };
+                let mut hasher = Sha256::new();
+                let mut buf = proxmox_io::boxed::zeroed(4096);
+
+                loop {
+                    let bytes_read = decoder.read(&mut buf).unwrap();
+                    if bytes_read == 0 {
+                        break;
+                    }
+
+                    hasher.update(&buf[..bytes_read]);
+                }
+
+                let uncompressed_digest =
+                    oci_spec::image::Sha256Digest::from_str(&format!("{:x}", hasher.finalize()))
+                        .unwrap();
+
+                layer_mapping.insert(uncompressed_digest.into(), layer.clone());
+            }
+            _ => (),
+        }
+    }
+
+    layer_mapping
+}
+
+pub enum ProxmoxOciError {
+    NotAnOciImage,
+    OciSpecError(OciSpecError),
+}
+
+impl From<OciSpecError> for ProxmoxOciError {
+    fn from(oci_spec_err: OciSpecError) -> Self {
+        Self::OciSpecError(oci_spec_err)
+    }
+}
+
+impl From<std::io::Error> for ProxmoxOciError {
+    fn from(io_err: std::io::Error) -> Self {
+        Self::OciSpecError(io_err.into())
+    }
+}
+
+pub fn parse_oci_image(
+    oci_tar_path: &str,
+    rootfs_path: &str,
+) -> Result<Option<Config>, ProxmoxOciError> {
+    let oci_tar_file = File::open(oci_tar_path)?;
+    let mut oci_tar_image = match OciTarImage::new(oci_tar_file) {
+        Ok(oci_tar_image) => oci_tar_image,
+        Err(_) => return Err(ProxmoxOciError::NotAnOciImage),
+    };
+    let image_manifest = oci_tar_image.image_manifest(&Arch::Amd64)?;
+
+    let image_config_descriptor = image_manifest.config();
+
+    if image_config_descriptor.media_type() != &MediaType::ImageConfig {
+        return Err(ProxmoxOciError::OciSpecError(OciSpecError::Other(
+            "ImageConfig has wrong media type".into(),
+        )));
+    }
+
+    let image_config_file = oci_tar_image.open_blob(image_config_descriptor.digest())?;
+    let image_config = ImageConfiguration::from_reader(image_config_file)?;
+
+    extract_oci_image_rootfs(
+        &mut oci_tar_image,
+        &image_manifest,
+        &image_config,
+        rootfs_path.into(),
+    )?;
+
+    Ok(image_config.config().clone())
+}
+
+fn extract_oci_image_rootfs<R: Read + Seek>(
+    oci_tar_image: &mut OciTarImage<R>,
+    image_manifest: &ImageManifest,
+    image_config: &ImageConfiguration,
+    target_path: PathBuf,
+) -> oci_spec::Result<()> {
+    if !target_path.exists() {
+        return Err(OciSpecError::Other(
+            "Rootfs destination path not found".into(),
+        ));
+    }
+
+    let layer_map = build_layer_map(oci_tar_image, image_manifest);
+
+    for layer in image_config.rootfs().diff_ids() {
+        let layer_digest = oci_spec::image::Digest::from_str(layer)?;
+
+        let layer_descriptor = match layer_map.get(&layer_digest) {
+            Some(descriptor) => descriptor,
+            None => {
+                return Err(OciSpecError::Other(
+                    "Unknown layer digest found in rootfs.diff_ids".into(),
+                ));
+            }
+        };
+
+        let layer_file = oci_tar_image.open_blob(layer_descriptor.digest())?;
+
+        let tar_file: Box<dyn Read> = match layer_descriptor.media_type() {
+            MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => Box::new(layer_file),
+            MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
+                Box::new(GzDecoder::new(layer_file))
+            }
+            MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
+                Box::new(zstd::Decoder::new(layer_file)?)
+            }
+            _ => {
+                return Err(OciSpecError::Other(
+                    "Encountered invalid media type for rootfs layer".into(),
+                ));
+            }
+        };
+
+        let mut archive = Archive::new(tar_file);
+        archive.set_preserve_ownerships(true);
+        archive.unpack(&target_path)?;
+    }
+
+    Ok(())
+}
diff --git a/proxmox-oci/src/oci_tar_image.rs b/proxmox-oci/src/oci_tar_image.rs
new file mode 100644
index 00000000..02199c75
--- /dev/null
+++ b/proxmox-oci/src/oci_tar_image.rs
@@ -0,0 +1,173 @@
+use oci_spec::image::{Arch, Digest, ImageIndex, ImageManifest, MediaType};
+use oci_spec::OciSpecError;
+use std::cmp::min;
+use std::collections::HashMap;
+use std::io::{Read, Seek, SeekFrom};
+use std::path::PathBuf;
+use tar::Archive;
+
+#[derive(Clone)]
+pub struct TarEntry {
+    offset: u64,
+    size: usize,
+}
+
+pub struct TarEntryReader<'a, R: Read + Seek> {
+    tar_entry: TarEntry,
+    reader: &'a mut R,
+    position: u64,
+}
+
+impl<'a, R: Read + Seek> TarEntryReader<'a, R> {
+    pub fn new(tar_entry: TarEntry, reader: &'a mut R) -> Self {
+        Self {
+            tar_entry,
+            reader,
+            position: 0,
+        }
+    }
+}
+
+impl<R: Read + Seek> Read for TarEntryReader<'_, R> {
+    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
+        let max_read = min(buf.len(), self.tar_entry.size - self.position as usize);
+        let limited_buf = &mut buf[..max_read];
+        self.reader
+            .seek(SeekFrom::Start(self.tar_entry.offset + self.position))?;
+        let bytes_read = self.reader.read(limited_buf)?;
+        self.position += bytes_read as u64;
+
+        Ok(bytes_read)
+    }
+}
+
+impl<R: Read + Seek> Seek for TarEntryReader<'_, R> {
+    fn seek(&mut self, pos: SeekFrom) -> std::io::Result<u64> {
+        self.position = match pos {
+            SeekFrom::Start(position) => min(position, self.tar_entry.size as u64),
+            SeekFrom::End(offset) => {
+                if offset > self.tar_entry.size as i64 {
+                    return Err(std::io::Error::new(
+                        std::io::ErrorKind::NotSeekable,
+                        "Tried to seek before the beginning of the file",
+                    ));
+                }
+
+                (if offset <= 0 {
+                    self.tar_entry.size
+                } else {
+                    self.tar_entry.size - offset as usize
+                }) as u64
+            }
+            SeekFrom::Current(offset) => {
+                if (self.position as i64 + offset) < 0 {
+                    return Err(std::io::Error::new(
+                        std::io::ErrorKind::NotSeekable,
+                        "Tried to seek before the beginning of the file",
+                    ));
+                }
+
+                min(self.position + offset as u64, self.tar_entry.size as u64)
+            }
+        };
+
+        Ok(self.position)
+    }
+}
+
+struct TarArchive<R: Read + Seek> {
+    reader: R,
+    entries: HashMap<PathBuf, TarEntry>,
+}
+
+impl<R: Read + Seek> TarArchive<R> {
+    pub fn new(reader: R) -> std::io::Result<Self> {
+        let mut archive = Archive::new(reader);
+        let entries = archive.entries_with_seek()?;
+        let mut entries_index = HashMap::new();
+
+        for entry in entries {
+            let entry = entry?;
+            let offset = entry.raw_file_position();
+            let size = entry.size() as usize;
+            let path = entry.path()?.into_owned();
+            let tar_entry = TarEntry { offset, size };
+            entries_index.insert(path, tar_entry);
+        }
+
+        Ok(Self {
+            reader: archive.into_inner(),
+            entries: entries_index,
+        })
+    }
+
+    pub fn get_inner_file(
+        &mut self,
+        inner_file_path: PathBuf,
+    ) -> std::io::Result<TarEntryReader<R>> {
+        let tar_entry = match self.entries.get(&inner_file_path) {
+            Some(tar_entry) => tar_entry,
+            None => {
+                return Err(std::io::Error::new(
+                    std::io::ErrorKind::NotFound,
+                    "File not found in archive",
+                ));
+            }
+        };
+
+        Ok(TarEntryReader::new(tar_entry.clone(), &mut self.reader))
+    }
+}
+
+pub struct OciTarImage<R: Read + Seek> {
+    archive: TarArchive<R>,
+    image_index: ImageIndex,
+}
+
+impl<R: Read + Seek> OciTarImage<R> {
+    pub fn new(reader: R) -> oci_spec::Result<Self> {
+        let mut archive = TarArchive::new(reader)?;
+        let index_file = archive.get_inner_file("index.json".into())?;
+        let image_index = ImageIndex::from_reader(index_file)?;
+
+        Ok(Self {
+            archive,
+            image_index,
+        })
+    }
+
+    pub fn image_index(&self) -> &ImageIndex {
+        &self.image_index
+    }
+
+    pub fn open_blob(&mut self, digest: &Digest) -> std::io::Result<TarEntryReader<R>> {
+        self.archive.get_inner_file(get_blob_path(digest))
+    }
+
+    pub fn image_manifest(&mut self, architecture: &Arch) -> oci_spec::Result<ImageManifest> {
+        let image_manifest_descriptor = match self.image_index.manifests().iter().find(|&x| {
+            x.media_type() == &MediaType::ImageManifest
+                && x.platform()
+                    .as_ref()
+                    .is_none_or(|platform| platform.architecture() == architecture)
+        }) {
+            Some(descriptor) => descriptor,
+            None => {
+                return Err(OciSpecError::Io(std::io::Error::new(
+                    std::io::ErrorKind::NotFound,
+                    "Image manifest not found for architecture",
+                )));
+            }
+        };
+
+        let image_manifest_file = self.open_blob(&image_manifest_descriptor.digest().clone())?;
+
+        ImageManifest::from_reader(image_manifest_file)
+    }
+}
+
+fn get_blob_path(digest: &Digest) -> PathBuf {
+    let algorithm = digest.algorithm().as_ref();
+    let digest = digest.digest();
+    PathBuf::from(format!("blobs/{algorithm}/{digest}"))
+}
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH proxmox-perl-rs 2/9] add Perl mapping for OCI container image parser

[Filip Schauer]

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 pve-rs/Cargo.toml |  2 ++
 pve-rs/Makefile   |  1 +
 pve-rs/src/lib.rs |  1 +
 pve-rs/src/oci.rs | 20 ++++++++++++++++++++
 4 files changed, 24 insertions(+)
 create mode 100644 pve-rs/src/oci.rs

diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index 82ebce4..2ca5f9f 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -21,6 +21,7 @@ hex = "0.4"
 http = "0.2.7"
 libc = "0.2"
 nix = "0.26"
+oci-spec = "0.8.1"
 openssl = "0.10.40"
 serde = "1.0"
 serde_bytes = "0.11"
@@ -45,3 +46,4 @@ proxmox-sys = "0.6"
 proxmox-tfa = { version = "5", features = ["api"] }
 proxmox-time = "2"
 proxmox-ve-config = { version = "0.2.1" }
+proxmox-oci = "0.1.0"
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index 4a5a277..b9c07c0 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -28,6 +28,7 @@ PERLMOD_GENPACKAGE := /usr/lib/perlmod/genpackage.pl \
 
 PERLMOD_PACKAGES := \
 	  PVE::RS::Firewall::SDN \
+	  PVE::RS::OCI \
 	  PVE::RS::OpenId \
 	  PVE::RS::ResourceScheduling::Static \
 	  PVE::RS::TFA
diff --git a/pve-rs/src/lib.rs b/pve-rs/src/lib.rs
index bb979b9..1fe9e77 100644
--- a/pve-rs/src/lib.rs
+++ b/pve-rs/src/lib.rs
@@ -12,6 +12,7 @@ use proxmox_notify::{Config, Notification, Severity};
 mod common;
 
 pub mod firewall;
+pub mod oci;
 pub mod openid;
 
 pub mod bindings;
diff --git a/pve-rs/src/oci.rs b/pve-rs/src/oci.rs
new file mode 100644
index 0000000..3ce06c0
--- /dev/null
+++ b/pve-rs/src/oci.rs
@@ -0,0 +1,20 @@
+#[perlmod::package(name = "PVE::RS::OCI")]
+mod export {
+    use anyhow::{bail, Error};
+    use oci_spec::image::Config;
+    use proxmox_oci::ProxmoxOciError;
+
+    #[export]
+    pub fn parse_oci_image(
+        oci_tar_path: &str,
+        rootfs_path: &str,
+    ) -> Result<Option<Config>, Error> {
+        match proxmox_oci::parse_oci_image(oci_tar_path, rootfs_path) {
+            Ok(config) => Ok(Some(config.unwrap_or_default())),
+            Err(err) => match err {
+                ProxmoxOciError::NotAnOciImage => Ok(None),
+                ProxmoxOciError::OciSpecError(e) => bail!("{e}"),
+            }
+        }
+    }
+}
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH storage 3/9] allow .tar container templates

[Filip Schauer]

This is needed for OCI container images bundled as tar files, as
generated by `docker save`. OCI images do not need additional
compression, since the content is usually compressed already.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/PVE/Storage.pm        | 2 +-
 src/PVE/Storage/Plugin.pm | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index d0a696a..6b03e32 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -110,7 +110,7 @@ PVE::Storage::Plugin->init();
 
 our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
 
-our $VZTMPL_EXT_RE_1 = qr/\.tar\.(gz|xz|zst|bz2)/i;
+our $VZTMPL_EXT_RE_1 = qr/\.tar(?:\.(gz|xz|zst|bz2))?/i;
 
 our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
 
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 4e16420..db24f4a 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1380,7 +1380,7 @@ my $get_subdir_files = sub {
 	} elsif ($tt eq 'vztmpl') {
 	    next if $fn !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
 
-	    $info = { volid => "$sid:vztmpl/$1", format => "t$2" };
+	    $info = { volid => "$sid:vztmpl/$1", format => $2 ? "t$2" : "tar" };
 
 	} elsif ($tt eq 'backup') {
 	    next if $fn !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH container 4/9] config: whitelist lxc.init.cwd

[Filip Schauer]

This parameter allows setting the working directory of the init process
in the container. This can be used by containers created from an OCI
image, that specifies a custom working directory.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/PVE/LXC/Config.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 0740e8c..49067ea 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -641,6 +641,7 @@ my $valid_lxc_conf_keys = {
     'lxc.signal.reboot' => 1,
     'lxc.signal.stop' => 1,
     'lxc.init.cmd' => 1,
+    'lxc.init.cwd' => 1,
     'lxc.pty.max' => 1,
     'lxc.console.logfile' => 1,
     'lxc.console.path' => 1,
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH container 5/9] add support for OCI images as container templates

[Filip Schauer]

This aims to add basic support for the Open Container Initiative image
format according to the specification. [0]

[0] https://github.com/opencontainers/image-spec/blob/main/spec.md

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/PVE/API2/LXC.pm | 53 ++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 50 insertions(+), 3 deletions(-)

diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 5c6ee57..520b926 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -19,9 +19,11 @@ use PVE::Storage;
 use PVE::RESTHandler;
 use PVE::RPCEnvironment;
 use PVE::ReplicationConfig;
+use PVE::RS::OCI;
 use PVE::LXC;
 use PVE::LXC::Create;
 use PVE::LXC::Migrate;
+use PVE::LXC::Namespaces;
 use PVE::GuestHelpers;
 use PVE::VZDump::Plugin;
 use PVE::API2::LXC::Config;
@@ -484,9 +486,54 @@ __PACKAGE__->register_method({
 		eval {
 		    my $rootdir = PVE::LXC::mount_all($vmid, $storage_cfg, $conf, 1);
 		    $bwlimit = PVE::Storage::get_bandwidth_limit('restore', [keys %used_storages], $bwlimit);
-		    print "restoring '$archive' now..\n"
-			if $restore && $archive ne '-';
-		    PVE::LXC::Create::restore_archive($storage_cfg, $archive, $rootdir, $conf, $ignore_unpack_errors, $bwlimit);
+		    my $oci_config;
+
+		    if ($restore && $archive ne '-') {
+			print "restoring '$archive' now..\n";
+		    } else {
+			# Try interpreting the file as an OCI image first.
+			# If it fails, treat it as an LXC template instead.
+			my $archivepath = PVE::Storage::abs_filesystem_path($storage_cfg, $archive);
+			my ($id_map, $root_uid, $root_gid) = PVE::LXC::parse_id_maps($conf);
+			$oci_config = PVE::LXC::Namespaces::run_in_userns(sub {
+			    return PVE::RS::OCI::parse_oci_image($archivepath, $rootdir);
+			}, $id_map);
+		    }
+
+		    if (defined($oci_config)) {
+			# OCI image extracted successfully
+
+			# Set the entrypoint and arguments if specified by the OCI image
+			my @init_cmd = ();
+			push(@init_cmd, $oci_config->{Entrypoint}) if $oci_config->{Entrypoint};
+			push(@init_cmd, $oci_config->{Cmd}) if $oci_config->{Cmd};
+			if (@init_cmd) {
+			    my $init_cmd_str = join(' ', @{ $init_cmd[0] });
+			    push @{$conf->{lxc}}, ['lxc.init.cmd', $init_cmd_str];
+			    # An entrypoint other than /sbin/init breaks the tty console mode.
+			    # This is fixed by setting cmode: console
+			    $conf->{cmode} = 'console';
+			}
+
+			push @{$conf->{lxc}}, ['lxc.init.cwd', $oci_config->{WorkingDir}]
+			    if ($oci_config->{WorkingDir});
+
+			if (my $envs = $oci_config->{Env}) {
+			    for my $env (@{$envs}) {
+				push @{$conf->{lxc}}, ['lxc.environment', $env];
+			    }
+			}
+		    } else {
+			# Not an OCI image, so restore it as an LXC image instead
+			PVE::LXC::Create::restore_archive(
+			    $storage_cfg,
+			    $archive,
+			    $rootdir,
+			    $conf,
+			    $ignore_unpack_errors,
+			    $bwlimit
+			);
+		    }
 
 		    if ($restore) {
 			print "merging backed-up and given configuration..\n";
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH container 6/9] config: add entrypoint parameter

[Filip Schauer]

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/PVE/API2/LXC.pm   |  2 +-
 src/PVE/LXC.pm        |  2 ++
 src/PVE/LXC/Config.pm | 12 ++++++++++++
 3 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 520b926..d4b074a 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -509,7 +509,7 @@ __PACKAGE__->register_method({
 			push(@init_cmd, $oci_config->{Cmd}) if $oci_config->{Cmd};
 			if (@init_cmd) {
 			    my $init_cmd_str = join(' ', @{ $init_cmd[0] });
-			    push @{$conf->{lxc}}, ['lxc.init.cmd', $init_cmd_str];
+			    $conf->{entrypoint} = $init_cmd_str;
 			    # An entrypoint other than /sbin/init breaks the tty console mode.
 			    # This is fixed by setting cmode: console
 			    $conf->{cmode} = 'console';
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 2b9f0cf..b2be27e 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -794,6 +794,8 @@ sub update_lxc_config {
 
     $raw .= "lxc.rootfs.path = $dir/rootfs\n";
 
+    $raw .= "lxc.init.cmd = $conf->{entrypoint}\n" if defined($conf->{entrypoint});
+
     foreach my $k (sort keys %$conf) {
 	next if $k !~ m/^net(\d+)$/;
 	my $ind = $1;
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 49067ea..d7d8b6a 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -591,6 +591,12 @@ my $confdesc = {
 	enum => ['shell', 'console', 'tty'],
 	default => 'tty',
     },
+    entrypoint => {
+	optional => 1,
+	type => 'string',
+	description => "Absolute path from container rootfs to the binary to use as init.",
+	default => '/sbin/init',
+    },
     protection => {
 	optional => 1,
 	type => 'boolean',
@@ -1745,6 +1751,12 @@ sub get_cmode {
     return $conf->{cmode} // $confdesc->{cmode}->{default};
 }
 
+sub get_entrypoint {
+    my ($class, $conf) = @_;
+
+    return $conf->{entrypoint} // $confdesc->{entrypoint}->{default};
+}
+
 sub valid_volume_keys {
     my ($class, $reverse) = @_;
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH container 7/9] configure static IP in LXC config for custom entrypoint

[Filip Schauer]

When a container uses the default `/sbin/init` entrypoint, network
interface configuration is usually managed by processes within the
container. However, containers with a different entrypoint might not
have any internal network management process. Consequently, IP addresses
might not be assigned.

This change ensures that a static IP address is explicitly set in the
LXC config for the container.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/PVE/LXC.pm | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index b2be27e..0131ac3 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -818,6 +818,18 @@ sub update_lxc_config {
 	if ($lxc_major >= 4) {
 	    $raw .= "lxc.net.$ind.script.up = /usr/share/lxc/lxcnetaddbr\n";
 	}
+
+	if (!defined($d->{link_down}) || $d->{link_down} != 1
+	    && PVE::LXC::Config->get_entrypoint($conf) ne "/sbin/init"
+	) {
+	    $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n"
+		if defined($d->{ip}) && $d->{ip} !~ /^(dhcp|manual)$/;
+	    $raw .= "lxc.net.$ind.ipv4.gateway = $d->{gw}\n" if defined($d->{gw});
+	    $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n"
+		if defined($d->{ip6}) && $d->{ip6} !~ /^(auto|dhcp|manual)$/;
+	    $raw .= "lxc.net.$ind.ipv6.gateway = $d->{gw6}\n" if defined($d->{gw6});
+	    $raw .= "lxc.net.$ind.flags = up\n";
+	}
     }
 
     my $had_cpuset = 0;
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH container 8/9] setup: debian: create /etc/network path if missing

[Filip Schauer]

This prevents an error during Debian container setup when the
/etc/network directory is missing.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/PVE/LXC/Setup/Debian.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/PVE/LXC/Setup/Debian.pm b/src/PVE/LXC/Setup/Debian.pm
index 8f7f524..b607c44 100644
--- a/src/PVE/LXC/Setup/Debian.pm
+++ b/src/PVE/LXC/Setup/Debian.pm
@@ -418,6 +418,7 @@ sub setup_network {
 	              $interfaces;
     }
 
+    $self->ct_make_path('/etc/network');
     $self->ct_file_set_contents($filename, $interfaces);
 }
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH container 9/9] manage DHCP for containers with custom entrypoint

[Filip Schauer]

Containers that do not use the default `/sbin/init` entrypoint may lack
in‑container network management. A previous commit already handles
static IP addresses. Now this commit also handles DHCP. This is done
using a `dhclient` process for each network interface.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/PVE/LXC.pm        | 74 ++++++++++++++++++++++++++++++++++++++++---
 src/PVE/LXC/Config.pm |  6 +++-
 2 files changed, 74 insertions(+), 6 deletions(-)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 0131ac3..e91b53a 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1004,6 +1004,8 @@ sub vm_stop_cleanup {
 	PVE::Storage::deactivate_volumes($storage_cfg, $vollist);
     };
     warn $@ if $@; # avoid errors - just warn
+
+    kill_dhclients($vmid, '*') if (PVE::LXC::Config->get_entrypoint($conf) ne "/sbin/init");
 }
 
 sub net_tap_plug : prototype($$) {
@@ -1189,6 +1191,34 @@ sub get_interfaces {
     return $res;
 }
 
+sub manage_dhclient {
+    my ($action, $vmid, $ipversion, $eth, $rootdir) = @_;
+
+    File::Path::make_path("/var/lib/lxc/$vmid/hook") if $action eq 'start';
+    my $pidfile = "/var/lib/lxc/$vmid/hook/dhclient$ipversion-$eth.pid";
+    my $leasefile = "/var/lib/lxc/$vmid/hook/dhclient$ipversion-$eth.leases";
+    my $scriptfile = '/usr/share/lxc/hooks/dhclient-script';
+    PVE::Tools::run_command([
+	'lxc-attach', '-n', $vmid, '-s', 'NETWORK|UTSNAME', '--',
+	'aa-exec', '-p', 'unconfined',
+	'/sbin/dhclient', $action eq 'start' ? '-1' : '-r', "-$ipversion",
+	'-pf', $pidfile, '-lf', $leasefile, '-e', "ROOTFS=$rootdir", '-sf', $scriptfile, $eth
+    ]);
+}
+
+sub kill_dhclients {
+    my ($vmid, $eth) = @_;
+
+    foreach my $pidfile (glob("/var/lib/lxc/$vmid/hook/dhclient*-$eth.pid")) {
+	my $pid = eval { file_get_contents($pidfile) };
+	if (!$@) {
+	    chomp $pid;
+	    kill 9, $pid if ($pid =~ m/^\d+$/);
+	    unlink($pidfile);
+	}
+    }
+}
+
 sub update_ipconfig {
     my ($vmid, $conf, $opt, $eth, $newnet, $rootdir) = @_;
 
@@ -1223,11 +1253,21 @@ sub update_ipconfig {
 
 	# step 1: add new IP, if this fails we cancel
 	my $is_real_ip = ($newip && $newip !~ /^(?:auto|dhcp|manual)$/);
-	if ($change_ip && $is_real_ip) {
-	    eval { &$ipcmd($family_opt, 'addr', 'add', $newip, 'dev', $eth); };
-	    if (my $err = $@) {
-		warn $err;
-		return;
+	if ($change_ip) {
+	    if (PVE::LXC::Config->get_entrypoint($conf) ne "/sbin/init") {
+		if ($newip && $newip eq 'dhcp') {
+		    manage_dhclient('start', $vmid, $ipversion, $eth, $rootdir);
+		} elsif ($oldip && $oldip eq 'dhcp') {
+		    manage_dhclient('stop', $vmid, $ipversion, $eth, $rootdir);
+		}
+	    }
+
+	    if ($is_real_ip) {
+		eval { &$ipcmd($family_opt, 'addr', 'add', $newip, 'dev', $eth); };
+		if (my $err = $@) {
+		    warn $err;
+		    return;
+		}
 	    }
 	}
 
@@ -2707,6 +2747,30 @@ sub vm_start {
     }
     PVE::GuestHelpers::exec_hookscript($conf, $vmid, 'post-start');
 
+    my @dhcpv4_interfaces = ();
+    my @dhcpv6_interfaces = ();
+    foreach my $k (sort keys %$conf) {
+	next if $k !~ m/^net(\d+)$/;
+	my $d = PVE::LXC::Config->parse_lxc_network($conf->{$k});
+	push @dhcpv4_interfaces, $d->{name} if $d->{ip} && $d->{ip} eq 'dhcp';
+	push @dhcpv6_interfaces, $d->{name} if $d->{ip6} && $d->{ip6} eq 'dhcp';
+    }
+
+    my $pid = PVE::LXC::find_lxc_pid($vmid);
+    my $rootdir = "/proc/$pid/root";
+
+    if (PVE::LXC::Config->get_entrypoint($conf) ne "/sbin/init") {
+	foreach my $eth (@dhcpv4_interfaces) {
+	    eval { manage_dhclient('start', $vmid, 4, $eth, $rootdir) };
+	    PVE::RESTEnvironment::log_warn("DHCP failed - $@") if $@;
+	}
+
+	foreach my $eth (@dhcpv6_interfaces) {
+	    eval { manage_dhclient('stop', $vmid, 6, $eth, $rootdir) };
+	    PVE::RESTEnvironment::log_warn("DHCP failed - $@") if $@;
+	}
+    }
+
     return;
 }
 
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index d7d8b6a..854e711 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -1490,9 +1490,13 @@ sub vmconfig_hotplug_pending {
 		$cgroup->change_cpu_shares(undef);
 	    } elsif ($opt =~ m/^net(\d)$/) {
 		my $netid = $1;
+		my $net = PVE::LXC::Config->parse_lxc_network($conf->{$opt});
+		if (PVE::LXC::Config->get_entrypoint($conf) ne "/sbin/init") {
+		    PVE::LXC::kill_dhclients($vmid, $net->{name});
+		}
+
 		PVE::Network::veth_delete("veth${vmid}i$netid");
 		if ($have_sdn) {
-		    my $net = PVE::LXC::Config->parse_lxc_network($conf->{$opt});
 		    print "delete ips from $opt\n";
 		    eval { PVE::Network::SDN::Vnets::del_ips_from_mac($net->{bridge}, $net->{hwaddr}, $conf->{hostname}) };
 		    warn $@ if $@;
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH proxmox 1/9] add proxmox-oci crate

[Christoph Heiss]

Some comments inline.

On Tue May 20, 2025 at 2:42 PM CEST, Filip Schauer wrote:
> [..]
> diff --git a/proxmox-oci/src/lib.rs b/proxmox-oci/src/lib.rs
> new file mode 100644
> index 00000000..57bd48b4
> --- /dev/null
> +++ b/proxmox-oci/src/lib.rs
> @@ -0,0 +1,165 @@
> [..]
> +/// Build a mapping from uncompressed layer digests (as found in the image config's `rootfs.diff_ids`)
> +/// to their corresponding compressed-layer digests (i.e. the filenames under `blobs/<algorithm>/<digest>`)
> +fn build_layer_map<R: Read + Seek>(
> +    oci_tar_image: &mut OciTarImage<R>,
> +    image_manifest: &ImageManifest,
> +) -> HashMap<oci_spec::image::Digest, oci_spec::image::Descriptor> {
> +    let mut layer_mapping = HashMap::new();
> +
> +    for layer in image_manifest.layers() {
> +        match layer.media_type() {
> +            MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => {
> +                layer_mapping.insert(layer.digest().clone(), layer.clone());
> +            }
> +            MediaType::ImageLayerGzip
> +            | MediaType::ImageLayerNonDistributableGzip
> +            | MediaType::ImageLayerZstd
> +            | MediaType::ImageLayerNonDistributableZstd => {
> +                let compressed_blob = oci_tar_image.open_blob(layer.digest()).unwrap();
> +                let mut decoder: Box<dyn Read> = match layer.media_type() {
> +                    MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
> +                        Box::new(GzDecoder::new(compressed_blob))
> +                    }
> +                    MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
> +                        Box::new(zstd::Decoder::new(compressed_blob).unwrap())
> +                    }
> +                    _ => unreachable!(),
> +                };
> +                let mut hasher = Sha256::new();
> +                let mut buf = proxmox_io::boxed::zeroed(4096);
> +
> +                loop {
> +                    let bytes_read = decoder.read(&mut buf).unwrap();
> +                    if bytes_read == 0 {
> +                        break;
> +                    }
> +
> +                    hasher.update(&buf[..bytes_read]);
> +                }
> +
> +                let uncompressed_digest =
> +                    oci_spec::image::Sha256Digest::from_str(&format!("{:x}", hasher.finalize()))
> +                        .unwrap();

For this variant, the hashing part could be extracted into a separate
function and then the Gzip/Zstd can be proper independent match arms.
E.g. each having the decoder construction, calling the layer hashing
function and then `.insert()` mapping below.

Has the nice side-effect of avoiding things like the `unreachable!()`
above, too.

> +
> +                layer_mapping.insert(uncompressed_digest.into(), layer.clone());
> +            }
> +            _ => (),
> +        }
> +    }
> +
> +    layer_mapping
> +}
> +
> +pub enum ProxmoxOciError {
> +    NotAnOciImage,
> +    OciSpecError(OciSpecError),
> +}

Since this is only used by `parse_oci_image()`, rename it to something
like ParseError and/or ExtractError, depending on the below.

> +
> +impl From<OciSpecError> for ProxmoxOciError {
> +    fn from(oci_spec_err: OciSpecError) -> Self {
> +        Self::OciSpecError(oci_spec_err)
> +    }
> +}
> +
> +impl From<std::io::Error> for ProxmoxOciError {
> +    fn from(io_err: std::io::Error) -> Self {
> +        Self::OciSpecError(io_err.into())
> +    }
> +}
> +
> +pub fn parse_oci_image(

Seeing as this function not only parses, but also extract the contents
at the same time, this should be named something like
`parse_and_extract_image()`?

(The `_oci` is implied through the crate name already anyway.)

> +    oci_tar_path: &str,
> +    rootfs_path: &str,
> +) -> Result<Option<Config>, ProxmoxOciError> {
> +    let oci_tar_file = File::open(oci_tar_path)?;
> +    let mut oci_tar_image = match OciTarImage::new(oci_tar_file) {
> +        Ok(oci_tar_image) => oci_tar_image,
> +        Err(_) => return Err(ProxmoxOciError::NotAnOciImage),

I'd also separate the `OciSpecError::Io(_)` variant here, possible into
a separate `ProxmoxOciError::IoError` variant. Callers might want to
handle I/O errors other than e.g. actual parsing errors.

> +    };
> +    let image_manifest = oci_tar_image.image_manifest(&Arch::Amd64)?;
> +
> +    let image_config_descriptor = image_manifest.config();
> +
> +    if image_config_descriptor.media_type() != &MediaType::ImageConfig {
> +        return Err(ProxmoxOciError::OciSpecError(OciSpecError::Other(
> +            "ImageConfig has wrong media type".into(),
> +        )));

Can simply be separate variant, e.g. `ProxmoxOciError::WrongMediaType`

> +    }
> +
> +    let image_config_file = oci_tar_image.open_blob(image_config_descriptor.digest())?;
> +    let image_config = ImageConfiguration::from_reader(image_config_file)?;
> +
> +    extract_oci_image_rootfs(
> +        &mut oci_tar_image,
> +        &image_manifest,
> +        &image_config,
> +        rootfs_path.into(),
> +    )?;
> +
> +    Ok(image_config.config().clone())
> +}
> +
> +fn extract_oci_image_rootfs<R: Read + Seek>(
> +    oci_tar_image: &mut OciTarImage<R>,
> +    image_manifest: &ImageManifest,
> +    image_config: &ImageConfiguration,
> +    target_path: PathBuf,
> +) -> oci_spec::Result<()> {
> +    if !target_path.exists() {
> +        return Err(OciSpecError::Other(
> +            "Rootfs destination path not found".into(),
> +        ));
> +    }
> +
> +    let layer_map = build_layer_map(oci_tar_image, image_manifest);
> +
> +    for layer in image_config.rootfs().diff_ids() {
> +        let layer_digest = oci_spec::image::Digest::from_str(layer)?;
> +
> +        let layer_descriptor = match layer_map.get(&layer_digest) {
> +            Some(descriptor) => descriptor,
> +            None => {
> +                return Err(OciSpecError::Other(
> +                    "Unknown layer digest found in rootfs.diff_ids".into(),
> +                ));
> +            }
> +        };
> +
> +        let layer_file = oci_tar_image.open_blob(layer_descriptor.digest())?;
> +
> +        let tar_file: Box<dyn Read> = match layer_descriptor.media_type() {
> +            MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => Box::new(layer_file),
> +            MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
> +                Box::new(GzDecoder::new(layer_file))
> +            }
> +            MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
> +                Box::new(zstd::Decoder::new(layer_file)?)
> +            }
> +            _ => {
> +                return Err(OciSpecError::Other(
> +                    "Encountered invalid media type for rootfs layer".into(),
> +                ));

Again I'd just introduce a separate error variant or even re-use
`ProxmoxOciError::WrongMediaType`, seems appropriate too.

> +            }
> +        };
> +
> +        let mut archive = Archive::new(tar_file);
> +        archive.set_preserve_ownerships(true);
> +        archive.unpack(&target_path)?;
> +    }
> +
> +    Ok(())
> +}
> diff --git a/proxmox-oci/src/oci_tar_image.rs b/proxmox-oci/src/oci_tar_image.rs
> new file mode 100644
> index 00000000..02199c75
> --- /dev/null
> +++ b/proxmox-oci/src/oci_tar_image.rs
> @@ -0,0 +1,173 @@
> [..]
> +pub struct OciTarImage<R: Read + Seek> {
> +    archive: TarArchive<R>,
> +    image_index: ImageIndex,
> +}
> +
> +impl<R: Read + Seek> OciTarImage<R> {
> [..]
> +    pub fn image_manifest(&mut self, architecture: &Arch) -> oci_spec::Result<ImageManifest> {
> +        let image_manifest_descriptor = match self.image_index.manifests().iter().find(|&x| {
> +            x.media_type() == &MediaType::ImageManifest
> +                && x.platform()
> +                    .as_ref()
> +                    .is_none_or(|platform| platform.architecture() == architecture)
> +        }) {
> +            Some(descriptor) => descriptor,
> +            None => {
> +                return Err(OciSpecError::Io(std::io::Error::new(
> +                    std::io::ErrorKind::NotFound,
> +                    "Image manifest not found for architecture",
> +                )));

Doesn't really seem like an I/O error?

> +            }
> +        };
> +
> +        let image_manifest_file = self.open_blob(&image_manifest_descriptor.digest().clone())?;
> +
> +        ImageManifest::from_reader(image_manifest_file)
> +    }
> +}
> +
> +fn get_blob_path(digest: &Digest) -> PathBuf {
> +    let algorithm = digest.algorithm().as_ref();
> +    let digest = digest.digest();
> +    PathBuf::from(format!("blobs/{algorithm}/{digest}"))

The path can also be constructed using the primivites from `PathBuf`
itself, e.g.

  let mut path = PathBuf::new();
  path.push(digest.algorithm().as_ref());
  ..

> +}



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH proxmox-perl-rs 2/9] add Perl mapping for OCI container image parser

[Christoph Heiss]

Needs a rebase on latest master, FWIW.

On Tue May 20, 2025 at 2:42 PM CEST, Filip Schauer wrote:
> Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
> ---
>  pve-rs/Cargo.toml |  2 ++
>  pve-rs/Makefile   |  1 +
>  pve-rs/src/lib.rs |  1 +
>  pve-rs/src/oci.rs | 20 ++++++++++++++++++++
>  4 files changed, 24 insertions(+)
>  create mode 100644 pve-rs/src/oci.rs
>
> diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
> index 82ebce4..2ca5f9f 100644
> --- a/pve-rs/Cargo.toml
> +++ b/pve-rs/Cargo.toml
> @@ -21,6 +21,7 @@ hex = "0.4"
>  http = "0.2.7"
>  libc = "0.2"
>  nix = "0.26"
> +oci-spec = "0.8.1"
>  openssl = "0.10.40"
>  serde = "1.0"
>  serde_bytes = "0.11"
> @@ -45,3 +46,4 @@ proxmox-sys = "0.6"
>  proxmox-tfa = { version = "5", features = ["api"] }
>  proxmox-time = "2"
>  proxmox-ve-config = { version = "0.2.1" }
> +proxmox-oci = "0.1.0"

These two new dependencies are missing their corresponding entries in
debian/control.



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH container 8/9] setup: debian: create /etc/network path if missing

[Christoph Heiss]

On Tue May 20, 2025 at 2:42 PM CEST, Filip Schauer wrote:
> This prevents an error during Debian container setup when the
> /etc/network directory is missing.

Out of curiosity: Is this actually related to the OCI support in this
series or a separate/independent fix?

As this is specific to Debian LXCs.

>
> Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
> ---
>  src/PVE/LXC/Setup/Debian.pm | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/src/PVE/LXC/Setup/Debian.pm b/src/PVE/LXC/Setup/Debian.pm
> index 8f7f524..b607c44 100644
> --- a/src/PVE/LXC/Setup/Debian.pm
> +++ b/src/PVE/LXC/Setup/Debian.pm
> @@ -418,6 +418,7 @@ sub setup_network {
>  	              $interfaces;
>      }
>
> +    $self->ct_make_path('/etc/network');
>      $self->ct_file_set_contents($filename, $interfaces);
>  }
>



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH container 8/9] setup: debian: create /etc/network path if missing

[Filip Schauer]

On 02/06/2025 11:37, Christoph Heiss wrote:
> On Tue May 20, 2025 at 2:42 PM CEST, Filip Schauer wrote:
>> This prevents an error during Debian container setup when the
>> /etc/network directory is missing.
> Out of curiosity: Is this actually related to the OCI support in this
> series or a separate/independent fix?
>
> As this is specific to Debian LXCs.

It is sort of related to OCI support:
I stumbled upon this while testing with Debian based OCI containers that
did not have an /etc/network directory. (e.g. docker.io/httpd)
It seems that so far no Debian LXC template without this directory has
shown up.



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH storage 3/9] allow .tar container templates

[Michael Köppl]

On 5/20/25 14:42, Filip Schauer wrote:
> This is needed for OCI container images bundled as tar files, as
> generated by `docker save`. OCI images do not need additional
> compression, since the content is usually compressed already.
> 
> Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
> ---
>  src/PVE/Storage.pm        | 2 +-
>  src/PVE/Storage/Plugin.pm | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
> index d0a696a..6b03e32 100755
> --- a/src/PVE/Storage.pm
> +++ b/src/PVE/Storage.pm
> @@ -110,7 +110,7 @@ PVE::Storage::Plugin->init();
>  
>  our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
>  
> -our $VZTMPL_EXT_RE_1 = qr/\.tar\.(gz|xz|zst|bz2)/i;
> +our $VZTMPL_EXT_RE_1 = qr/\.tar(?:\.(gz|xz|zst|bz2))?/i;

I think this also needs an update in the UploadToStorage window in
pve-manager. During my testing I was unable to upload a .tar file and
.tar was also not shown as a supported file type in my file selection
dialog. Adding .tar as an accepted extension for the upload dialog made
it work.

>  
>  our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
>  
> diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
> index 4e16420..db24f4a 100644
> --- a/src/PVE/Storage/Plugin.pm
> +++ b/src/PVE/Storage/Plugin.pm
> @@ -1380,7 +1380,7 @@ my $get_subdir_files = sub {
>  	} elsif ($tt eq 'vztmpl') {
>  	    next if $fn !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
>  
> -	    $info = { volid => "$sid:vztmpl/$1", format => "t$2" };
> +	    $info = { volid => "$sid:vztmpl/$1", format => $2 ? "t$2" : "tar" };
>  
>  	} elsif ($tt eq 'backup') {
>  	    next if $fn !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH container/proxmox{, -perl-rs}/storage 0/9] support OCI images as container templates

[Michael Köppl]

Thanks for tackling this! I tested setting up containers based on
various OCI images. Apart from the UI not allowing upload of .tar files
(see my comment on the pve-storage patch), the uploading worked as
expected. I encountered some problems with various images during my
testing. I used docker save to get the .tar files (as per your example).
Tested the following with that setup:

- httpd image (as per your example): worked as expected, was able to
reach the httpd "It works!" page
- redis:latest: connection reset by peer on start, fails to start
- debian:bookworm: `sync_wait: 34 An error occurred in another process
(expected sequence number 7)` on start
- alpine:latest: worked as expected, landed in shell
- fedora:latest: unable to open file
'/etc/systemd/system-preset/00-pve.preset.tmp.85271' - No such file or
directory on create, cannot create container
- ubuntu:latest: `unable to open file
'/etc/systemd/network/eth0.network.tmp.89496' - No such file or directory`
- Supplying an SSH key does not seem to work (tested with alpine OCI image)
- Images with a CMD in their Dockerfile that does not call a shell (i.e.
last line is not CMD ["/bin/bash"]) will not display anything in the
console view. This is not unexpected, but UX-wise it might make sense to
inform users about this in some way (maybe by disabling the console view
and displaying an informational message)

I also tested with .tar files created by podman. Those only worked when
created with --format=oci-archive, otherwise os-release and architecture
can seemingly not be detected during startup, falling back to
'unmanaged' and 'amd64'. Might be worth mentioning in future
documentation for this feature.

On 5/20/25 14:42, Filip Schauer wrote:
> Add basic support for OCI (Open Container Initiative) images [0] as
> container templates.
> 
> An OCI image can be for example obtained from Docker Hub:
> 
> ```
> $ docker pull httpd
> $ docker save httpd > httpd.tar
> ```
> 
> The tarball can be uploaded to a storage as a container template and
> then used during container creation. It is automatically detected that
> the container template is an OCI image. The resulting container still
> uses the existing LXC framework.
> 
> # Dependencies:
> 
> Since the `oci-spec` crate is not in any Debian repository at the time
> of writing, it needs to be downloaded from crates.io, in order to be
> able to build `proxmox-oci`.
> 
> Here is a little script to download the `oci-spec` crate along with its
> dependencies:
> 
> ```sh
> download_crate() {
>     CRATE_NAME=$1
>     CRATE_VERSION=$2
>     CRATE_SHA256=$3
> 
>     wget https://crates.io/api/v1/crates/$CRATE_NAME/$CRATE_VERSION/download
> 
>     COMPUTED_SHA256=$(sha256sum download | awk '{ print $1 }')
>     if [ "$COMPUTED_SHA256" != "$CRATE_SHA256" ]; then
>         echo "Checksum mismatch"; exit 1
>     fi
> 
>     tar -xf download
>     rm download
>     mv $CRATE_NAME-$CRATE_VERSION /usr/share/cargo/registry/
>     echo "{\"package\":\"$CRATE_SHA256\",\"files\":{}}" > /usr/share/cargo/registry/$CRATE_NAME-$CRATE_VERSION/.cargo-checksum.json
> }
> 
> download_crate strsim 0.11.1 7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f
> download_crate ident_case 1.0.1 b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39
> download_crate darling_macro 0.20.11 fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead
> download_crate darling_core 0.20.11 0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e
> download_crate darling 0.20.11 fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee
> download_crate proc-macro-error-attr2 2.0.0 96de42df36bb9bba5542fe9f1a054b8cc87e172759a1868aa05c1f3acc89dfc5
> download_crate derive_builder_core 0.20.2 2d5bcf7b024d6835cfb3d473887cd966994907effbe9227e8c8219824d06c4e8
> download_crate thiserror-impl 2.0.0 22efd00f33f93fa62848a7cab956c3d38c8d43095efda1decfc2b3a5dc0b8972
> download_crate rustversion 1.0.20 eded382c5f5f786b989652c49544c4877d9f015cc22e145a5ea8ea66c2921cd2
> download_crate heck 0.5.0 2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea
> download_crate proc-macro-error2 2.0.1 11ec05c52be0a07b08061f7dd003e7d7092e0472bc731b4af7bb1ef876109802
> download_crate derive_builder_macro 0.20.2 ab63b0e2bf4d5928aff72e83a7dace85d7bba5fe12dcc3c5a572d78caffd3f3c
> download_crate thiserror 2.0.0 15291287e9bff1bc6f9ff3409ed9af665bec7a5fc8ac079ea96be07bca0e2668
> download_crate strum_macros 0.27.1 c77a8c5abcaf0f9ce05d62342b7d298c346515365c36b673df4ebe3ced01fde8
> download_crate strum 0.27.1 f64def088c51c9510a8579e3c5d67c65349dcf755e5479ad3d010aa6454e2c32
> download_crate getset 0.1.5 f3586f256131df87204eb733da72e3d3eb4f343c639f4b7be279ac7c48baeafe
> download_crate derive_builder 0.20.2 507dfb09ea8b7fa618fcf76e953f4f5e192547945816d5358edffe39f6f94947
> download_crate oci-spec 0.8.1 57e9beda9d92fac7bf4904c34c83340ef1024159faee67179a04e0277523da33
> ```
> 
> Since librust-oci-spec-dev is in the proxmox-oci/debian/control file, a
> dummy package needs to be installed, so dpkg-checkbuilddeps does not
> complain.
> 
> dummy_librust_oci_spec.equivs:
> 
> ```
> Package: librust-oci-spec-dev
> Version: 0.8.1
> Provides: librust-oci-spec-0.8+default-dev (= 0.8.1-1)
> ```
> 
> ```
> $ equivs-build dummy_librust_oci_spec.equivs
> $ dpkg -i ./librust-oci-spec-dev_0.8.1_all.deb
> ```
> 
> # Build & install order:
> 
> 1. proxmox
> 2. proxmox-perl-rs
> 3. pve-container
> *  pve-storage (no particular order there)
> 
> [0] https://github.com/opencontainers/image-spec/blob/main/spec.md
> 
> proxmox:
> 
> Filip Schauer (1):
>   add proxmox-oci crate
> 
>  Cargo.toml                       |   1 +
>  proxmox-oci/Cargo.toml           |  21 ++++
>  proxmox-oci/debian/changelog     |   5 +
>  proxmox-oci/debian/control       |  45 ++++++++
>  proxmox-oci/debian/debcargo.toml |   7 ++
>  proxmox-oci/src/lib.rs           | 165 +++++++++++++++++++++++++++++
>  proxmox-oci/src/oci_tar_image.rs | 173 +++++++++++++++++++++++++++++++
>  7 files changed, 417 insertions(+)
>  create mode 100644 proxmox-oci/Cargo.toml
>  create mode 100644 proxmox-oci/debian/changelog
>  create mode 100644 proxmox-oci/debian/control
>  create mode 100644 proxmox-oci/debian/debcargo.toml
>  create mode 100644 proxmox-oci/src/lib.rs
>  create mode 100644 proxmox-oci/src/oci_tar_image.rs
> 
> 
> proxmox-perl-rs:
> 
> Filip Schauer (1):
>   add Perl mapping for OCI container image parser
> 
>  pve-rs/Cargo.toml |  2 ++
>  pve-rs/Makefile   |  1 +
>  pve-rs/src/lib.rs |  1 +
>  pve-rs/src/oci.rs | 20 ++++++++++++++++++++
>  4 files changed, 24 insertions(+)
>  create mode 100644 pve-rs/src/oci.rs
> 
> 
> pve-storage:
> 
> Filip Schauer (1):
>   allow .tar container templates
> 
>  src/PVE/Storage.pm        | 2 +-
>  src/PVE/Storage/Plugin.pm | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> 
> pve-container:
> 
> Filip Schauer (6):
>   config: whitelist lxc.init.cwd
>   add support for OCI images as container templates
>   config: add entrypoint parameter
>   configure static IP in LXC config for custom entrypoint
>   setup: debian: create /etc/network path if missing
>   manage DHCP for containers with custom entrypoint
> 
>  src/PVE/API2/LXC.pm         | 53 ++++++++++++++++++++--
>  src/PVE/LXC.pm              | 88 ++++++++++++++++++++++++++++++++++---
>  src/PVE/LXC/Config.pm       | 19 +++++++-
>  src/PVE/LXC/Setup/Debian.pm |  1 +
>  4 files changed, 152 insertions(+), 9 deletions(-)
> 
> 
> Summary over all repositories:
>   17 files changed, 595 insertions(+), 11 deletions(-)
> 



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH container/proxmox{, -perl-rs}/storage 0/9] support OCI images as container templates

[Christoph Heiss]

Tested this series a bit using the
`docker.io/library/nginx:mainline-{alpine,bookworm}` images, pulled &
exported using `podman`.

Came across the same Problem as Michael, that for `podman save` the
`--format=oci-archive` is required. According to podman-save(1), the
default format is "docker-archive":

  "A tar archive interoperable with docker load(1) (the default)"

Probably the old, Docker-specific image format version? Might make sense
to support that too, if it isn't too different.

The nginx image, both -alpine and -bookworm, fail to start with:

lxc-info: 103: ../src/lxc/af_unix.c: lxc_abstract_unix_recv_fds_iov: 218 Connection reset by peer - Failed to receive response
lxc-info: 103: ../src/lxc/commands.c: lxc_cmd_rsp_recv_fds: 128 Failed to receive file descriptors for command "get_init_pid"

Not sure what's going on there.

Tested further with `ghcr.io/dani-garcia/vaultwarden:1.34.1-alpine`,
that worked w/o problem, being able to access the application on port 80
afterwards.

---

Also, as talked off-list, I sent a debcargo-conf patch for packaging the
`oci-spec` crate [0]. This is only for Trixie (for now), as packaging
for Bookworm would require backporting some dependent crate updates. And
I'm not sure if it's worth seeing the upcoming Trixie release.

But building it on Bookworm also works if you build the `darling`,
`derive-builder`, `strsim` and `zstd-sys` crates too - YMMV tho :^)

[0] https://lore.proxmox.com/pve-devel/20250606103719.533030-2-c.heiss@proxmox.com/

On Tue May 20, 2025 at 2:42 PM CEST, Filip Schauer wrote:
> Add basic support for OCI (Open Container Initiative) images [0] as
> container templates.
>
> An OCI image can be for example obtained from Docker Hub:
>
> ```
> $ docker pull httpd
> $ docker save httpd > httpd.tar
> ```
>
> The tarball can be uploaded to a storage as a container template and
> then used during container creation. It is automatically detected that
> the container template is an OCI image. The resulting container still
> uses the existing LXC framework.
>
> # Dependencies:
>
> Since the `oci-spec` crate is not in any Debian repository at the time
> of writing, it needs to be downloaded from crates.io, in order to be
> able to build `proxmox-oci`.
>
> Here is a little script to download the `oci-spec` crate along with its
> dependencies:
>
> ```sh
> download_crate() {
>     CRATE_NAME=$1
>     CRATE_VERSION=$2
>     CRATE_SHA256=$3
>
>     wget https://crates.io/api/v1/crates/$CRATE_NAME/$CRATE_VERSION/download
>
>     COMPUTED_SHA256=$(sha256sum download | awk '{ print $1 }')
>     if [ "$COMPUTED_SHA256" != "$CRATE_SHA256" ]; then
>         echo "Checksum mismatch"; exit 1
>     fi
>
>     tar -xf download
>     rm download
>     mv $CRATE_NAME-$CRATE_VERSION /usr/share/cargo/registry/
>     echo "{\"package\":\"$CRATE_SHA256\",\"files\":{}}" > /usr/share/cargo/registry/$CRATE_NAME-$CRATE_VERSION/.cargo-checksum.json
> }
>
> download_crate strsim 0.11.1 7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f
> download_crate ident_case 1.0.1 b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39
> download_crate darling_macro 0.20.11 fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead
> download_crate darling_core 0.20.11 0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e
> download_crate darling 0.20.11 fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee
> download_crate proc-macro-error-attr2 2.0.0 96de42df36bb9bba5542fe9f1a054b8cc87e172759a1868aa05c1f3acc89dfc5
> download_crate derive_builder_core 0.20.2 2d5bcf7b024d6835cfb3d473887cd966994907effbe9227e8c8219824d06c4e8
> download_crate thiserror-impl 2.0.0 22efd00f33f93fa62848a7cab956c3d38c8d43095efda1decfc2b3a5dc0b8972
> download_crate rustversion 1.0.20 eded382c5f5f786b989652c49544c4877d9f015cc22e145a5ea8ea66c2921cd2
> download_crate heck 0.5.0 2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea
> download_crate proc-macro-error2 2.0.1 11ec05c52be0a07b08061f7dd003e7d7092e0472bc731b4af7bb1ef876109802
> download_crate derive_builder_macro 0.20.2 ab63b0e2bf4d5928aff72e83a7dace85d7bba5fe12dcc3c5a572d78caffd3f3c
> download_crate thiserror 2.0.0 15291287e9bff1bc6f9ff3409ed9af665bec7a5fc8ac079ea96be07bca0e2668
> download_crate strum_macros 0.27.1 c77a8c5abcaf0f9ce05d62342b7d298c346515365c36b673df4ebe3ced01fde8
> download_crate strum 0.27.1 f64def088c51c9510a8579e3c5d67c65349dcf755e5479ad3d010aa6454e2c32
> download_crate getset 0.1.5 f3586f256131df87204eb733da72e3d3eb4f343c639f4b7be279ac7c48baeafe
> download_crate derive_builder 0.20.2 507dfb09ea8b7fa618fcf76e953f4f5e192547945816d5358edffe39f6f94947
> download_crate oci-spec 0.8.1 57e9beda9d92fac7bf4904c34c83340ef1024159faee67179a04e0277523da33
> ```
>
> Since librust-oci-spec-dev is in the proxmox-oci/debian/control file, a
> dummy package needs to be installed, so dpkg-checkbuilddeps does not
> complain.
>
> dummy_librust_oci_spec.equivs:
>
> ```
> Package: librust-oci-spec-dev
> Version: 0.8.1
> Provides: librust-oci-spec-0.8+default-dev (= 0.8.1-1)
> ```
>
> ```
> $ equivs-build dummy_librust_oci_spec.equivs
> $ dpkg -i ./librust-oci-spec-dev_0.8.1_all.deb
> ```
>
> # Build & install order:
>
> 1. proxmox
> 2. proxmox-perl-rs
> 3. pve-container
> *  pve-storage (no particular order there)
>
> [0] https://github.com/opencontainers/image-spec/blob/main/spec.md
>
> proxmox:
>
> Filip Schauer (1):
>   add proxmox-oci crate
>
>  Cargo.toml                       |   1 +
>  proxmox-oci/Cargo.toml           |  21 ++++
>  proxmox-oci/debian/changelog     |   5 +
>  proxmox-oci/debian/control       |  45 ++++++++
>  proxmox-oci/debian/debcargo.toml |   7 ++
>  proxmox-oci/src/lib.rs           | 165 +++++++++++++++++++++++++++++
>  proxmox-oci/src/oci_tar_image.rs | 173 +++++++++++++++++++++++++++++++
>  7 files changed, 417 insertions(+)
>  create mode 100644 proxmox-oci/Cargo.toml
>  create mode 100644 proxmox-oci/debian/changelog
>  create mode 100644 proxmox-oci/debian/control
>  create mode 100644 proxmox-oci/debian/debcargo.toml
>  create mode 100644 proxmox-oci/src/lib.rs
>  create mode 100644 proxmox-oci/src/oci_tar_image.rs
>
>
> proxmox-perl-rs:
>
> Filip Schauer (1):
>   add Perl mapping for OCI container image parser
>
>  pve-rs/Cargo.toml |  2 ++
>  pve-rs/Makefile   |  1 +
>  pve-rs/src/lib.rs |  1 +
>  pve-rs/src/oci.rs | 20 ++++++++++++++++++++
>  4 files changed, 24 insertions(+)
>  create mode 100644 pve-rs/src/oci.rs
>
>
> pve-storage:
>
> Filip Schauer (1):
>   allow .tar container templates
>
>  src/PVE/Storage.pm        | 2 +-
>  src/PVE/Storage/Plugin.pm | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
>
> pve-container:
>
> Filip Schauer (6):
>   config: whitelist lxc.init.cwd
>   add support for OCI images as container templates
>   config: add entrypoint parameter
>   configure static IP in LXC config for custom entrypoint
>   setup: debian: create /etc/network path if missing
>   manage DHCP for containers with custom entrypoint
>
>  src/PVE/API2/LXC.pm         | 53 ++++++++++++++++++++--
>  src/PVE/LXC.pm              | 88 ++++++++++++++++++++++++++++++++++---
>  src/PVE/LXC/Config.pm       | 19 +++++++-
>  src/PVE/LXC/Setup/Debian.pm |  1 +
>  4 files changed, 152 insertions(+), 9 deletions(-)
>
>
> Summary over all repositories:
>   17 files changed, 595 insertions(+), 11 deletions(-)



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel