* [pve-devel] [PATCH proxmox v4 01/15] io: introduce RangeReader for bounded reads
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH proxmox v4 02/15] add proxmox-oci crate Filip Schauer
` (13 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Introduce a reader that exposes a sub-range of an underlying reader.
This will be used for reading individual files out of a tar archive when
parsing an OCI image.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v3:
* add a commit message
* add rustdoc comments
* add unit tests
Introduced in v3
proxmox-io/src/lib.rs | 3 +
proxmox-io/src/range_reader.rs | 175 +++++++++++++++++++++++++++++++++
2 files changed, 178 insertions(+)
create mode 100644 proxmox-io/src/range_reader.rs
diff --git a/proxmox-io/src/lib.rs b/proxmox-io/src/lib.rs
index 1be005ff..a05b9232 100644
--- a/proxmox-io/src/lib.rs
+++ b/proxmox-io/src/lib.rs
@@ -6,6 +6,9 @@
#![deny(unsafe_op_in_unsafe_fn)]
#![cfg_attr(docsrs, feature(doc_cfg, doc_auto_cfg))]
+mod range_reader;
+pub use range_reader::RangeReader;
+
mod read;
pub use read::ReadExt;
diff --git a/proxmox-io/src/range_reader.rs b/proxmox-io/src/range_reader.rs
new file mode 100644
index 00000000..3f4c54fe
--- /dev/null
+++ b/proxmox-io/src/range_reader.rs
@@ -0,0 +1,175 @@
+use std::io::{Read, Seek, SeekFrom};
+use std::ops::Range;
+
+/// A reader that only exposes a sub-range of an underlying `Read + Seek`.
+///
+/// # Examples
+///
+/// ```
+/// # use proxmox_io::RangeReader;
+/// # use std::io::{Cursor, Read, Seek, SeekFrom};
+/// # fn func() -> Result<(), std::io::Error> {
+/// let reader = Cursor::new("Lorem ipsum dolor sit amet");
+///
+/// let mut range_reader = RangeReader::new(reader, 6..17);
+///
+/// // Read all bytes in the range
+/// let mut buf = Vec::new();
+/// range_reader.read_to_end(&mut buf)?;
+/// assert_eq!(buf, "ipsum dolor".as_bytes());
+///
+/// // Seek back to start of the range and read one byte
+/// range_reader.seek(SeekFrom::Start(0))?;
+/// let mut b = [0u8; 1];
+/// range_reader.read_exact(&mut b)?;
+/// assert_eq!(b, "i".as_bytes());
+///
+/// # Ok(())
+/// # }
+/// # func().unwrap();
+/// ```
+pub struct RangeReader<R: Read + Seek> {
+ /// Underlying reader
+ reader: R,
+
+ /// Range inside `R`
+ range: Range<u64>,
+
+ /// Relative position inside `range`
+ position: u64,
+
+ /// True once the initial seek has been performed
+ has_seeked: bool,
+}
+
+impl<R: Read + Seek> RangeReader<R> {
+ pub fn new(reader: R, range: Range<u64>) -> Self {
+ Self {
+ reader,
+ range,
+ position: 0,
+ has_seeked: false,
+ }
+ }
+
+ pub fn into_inner(self) -> R {
+ self.reader
+ }
+
+ pub fn size(&self) -> usize {
+ (self.range.end - self.range.start) as usize
+ }
+
+ pub fn remaining(&self) -> usize {
+ self.size() - self.position as usize
+ }
+}
+
+impl<R: Read + Seek> Read for RangeReader<R> {
+ fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
+ let max_read = buf.len().min(self.remaining());
+ let limited_buf = &mut buf[..max_read];
+
+ if !self.has_seeked {
+ self.reader
+ .seek(SeekFrom::Start(self.range.start + self.position))?;
+ self.has_seeked = true;
+ }
+
+ let bytes_read = self.reader.read(limited_buf)?;
+ self.position += bytes_read.min(max_read) as u64;
+
+ Ok(bytes_read)
+ }
+}
+
+impl<R: Read + Seek> Seek for RangeReader<R> {
+ fn seek(&mut self, pos: SeekFrom) -> std::io::Result<u64> {
+ self.position = match pos {
+ SeekFrom::Start(position) => position.min(self.size() as u64),
+ SeekFrom::End(offset) => {
+ if offset > self.size() as i64 {
+ return Err(std::io::Error::new(
+ std::io::ErrorKind::InvalidInput,
+ "Tried to seek before the beginning of the file",
+ ));
+ }
+
+ (if offset <= 0 {
+ self.size()
+ } else {
+ self.size() - offset as usize
+ }) as u64
+ }
+ SeekFrom::Current(offset) => {
+ if let Some(position) = self.position.checked_add_signed(offset) {
+ position.min(self.size() as u64)
+ } else {
+ return Err(std::io::Error::new(
+ std::io::ErrorKind::InvalidInput,
+ "Tried to seek before the beginning of the file",
+ ));
+ }
+ }
+ };
+
+ self.reader
+ .seek(SeekFrom::Start(self.range.start + self.position))?;
+ self.has_seeked = true;
+
+ Ok(self.position)
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::RangeReader;
+ use std::io::{Cursor, Read, Seek, SeekFrom};
+
+ #[test]
+ fn test_read_full_range() {
+ let reader = Cursor::new("Hello world!");
+ let mut range_reader = RangeReader::new(reader, 6..11);
+
+ let mut buf = Vec::new();
+ let len = range_reader.read_to_end(&mut buf).unwrap();
+
+ assert_eq!(len, 5);
+ assert_eq!(buf, "world".as_bytes());
+ }
+
+ #[test]
+ fn test_read_partial() {
+ let reader = Cursor::new("Hello world!");
+ let mut range_reader = RangeReader::new(reader, 0..5);
+
+ let mut buf = [0u8; 4];
+ range_reader.read_exact(&mut buf).unwrap();
+
+ assert_eq!(buf, "Hell".as_bytes());
+ }
+
+ #[test]
+ fn test_seek_and_read() {
+ let reader = Cursor::new("Lorem ipsum dolor sit amet");
+ let mut range_reader = RangeReader::new(reader, 6..21);
+
+ assert_eq!(range_reader.seek(SeekFrom::Start(6)).unwrap(), 6);
+ let mut buf = [0u8; 5];
+ range_reader.read_exact(&mut buf).unwrap();
+
+ assert_eq!(buf, "dolor".as_bytes());
+ }
+
+ #[test]
+ fn test_seek_out_of_range() {
+ let reader = Cursor::new("Lorem ipsum dolor sit amet");
+ let mut range_reader = RangeReader::new(reader, 6..21);
+
+ let err = range_reader.seek(SeekFrom::Current(-3)).unwrap_err();
+ assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput);
+
+ let err = range_reader.seek(SeekFrom::End(20)).unwrap_err();
+ assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput);
+ }
+}
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH proxmox v4 02/15] add proxmox-oci crate
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH proxmox v4 01/15] io: introduce RangeReader for bounded reads Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH proxmox v4 03/15] proxmox-oci: add tests for whiteout handling Filip Schauer
` (12 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
This crate can parse an OCI image tarball and extract its rootfs. Layers
are applied in sequence, but an overlay filesystem is currently not
used.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v3:
* correctly handle whiteout edge cases
* parse_and_extract_image: add argument for CPU architecture
Changed since v2:
* remove reachable unwraps & refactor code
* increase hasher buffer size from 4096 to 32768 (matching internal
sha2::Digest buffering)
* preserve permissions and xattrs during rootfs extraction
* handle whiteouts & opaque whiteouts
Cargo.toml | 1 +
proxmox-oci/Cargo.toml | 22 +++
proxmox-oci/debian/changelog | 5 +
proxmox-oci/debian/control | 45 +++++
proxmox-oci/debian/debcargo.toml | 7 +
proxmox-oci/src/lib.rs | 324 +++++++++++++++++++++++++++++++
proxmox-oci/src/oci_tar_image.rs | 144 ++++++++++++++
7 files changed, 548 insertions(+)
create mode 100644 proxmox-oci/Cargo.toml
create mode 100644 proxmox-oci/debian/changelog
create mode 100644 proxmox-oci/debian/control
create mode 100644 proxmox-oci/debian/debcargo.toml
create mode 100644 proxmox-oci/src/lib.rs
create mode 100644 proxmox-oci/src/oci_tar_image.rs
diff --git a/Cargo.toml b/Cargo.toml
index ce249371..ff12d7b7 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -27,6 +27,7 @@ members = [
"proxmox-network-api",
"proxmox-network-types",
"proxmox-notify",
+ "proxmox-oci",
"proxmox-openid",
"proxmox-product-config",
"proxmox-resource-scheduling",
diff --git a/proxmox-oci/Cargo.toml b/proxmox-oci/Cargo.toml
new file mode 100644
index 00000000..4daff6ab
--- /dev/null
+++ b/proxmox-oci/Cargo.toml
@@ -0,0 +1,22 @@
+[package]
+name = "proxmox-oci"
+description = "OCI image parsing and extraction"
+version = "0.1.0"
+
+authors.workspace = true
+edition.workspace = true
+exclude.workspace = true
+homepage.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version.workspace = true
+
+[dependencies]
+flate2.workspace = true
+oci-spec = "0.8.1"
+sha2 = "0.10"
+tar.workspace = true
+thiserror = "1"
+zstd.workspace = true
+
+proxmox-io.workspace = true
diff --git a/proxmox-oci/debian/changelog b/proxmox-oci/debian/changelog
new file mode 100644
index 00000000..754d06c1
--- /dev/null
+++ b/proxmox-oci/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-oci (0.1.0-1) bookworm; urgency=medium
+
+ * Initial release.
+
+ -- Proxmox Support Team <support@proxmox.com> Mon, 28 Apr 2025 12:34:56 +0200
diff --git a/proxmox-oci/debian/control b/proxmox-oci/debian/control
new file mode 100644
index 00000000..f33331c5
--- /dev/null
+++ b/proxmox-oci/debian/control
@@ -0,0 +1,45 @@
+Source: rust-proxmox-oci
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.82) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-flate2-1+default-dev <!nocheck>,
+ librust-oci-spec-0.8+default-dev (>= 0.8.1-~~) <!nocheck>,
+ librust-proxmox-io-1+default-dev (>= 1.2.0-~~) <!nocheck>,
+ librust-sha2-0.10+default-dev <!nocheck>,
+ librust-tar-0.4+default-dev <!nocheck>,
+ librust-thiserror-1+default-dev <!nocheck>,
+ librust-zstd-0.13+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.0
+Vcs-Git: git://git.proxmox.com/git/proxmox.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: proxmox-oci
+Rules-Requires-Root: no
+
+Package: librust-proxmox-oci-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-flate2-1+default-dev,
+ librust-oci-spec-0.8+default-dev (>= 0.8.1-~~),
+ librust-proxmox-io-1+default-dev (>= 1.2.0-~~),
+ librust-sha2-0.10+default-dev,
+ librust-tar-0.4+default-dev,
+ librust-thiserror-1+default-dev,
+ librust-zstd-0.13+default-dev
+Provides:
+ librust-proxmox-oci+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0-dev (= ${binary:Version}),
+ librust-proxmox-oci-0+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1.0-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1.0+default-dev (= ${binary:Version})
+Description: OCI image parsing and extraction - Rust source code
+ Source code for Debianized Rust crate "proxmox-oci"
diff --git a/proxmox-oci/debian/debcargo.toml b/proxmox-oci/debian/debcargo.toml
new file mode 100644
index 00000000..b7864cdb
--- /dev/null
+++ b/proxmox-oci/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox.git"
diff --git a/proxmox-oci/src/lib.rs b/proxmox-oci/src/lib.rs
new file mode 100644
index 00000000..cf0e4271
--- /dev/null
+++ b/proxmox-oci/src/lib.rs
@@ -0,0 +1,324 @@
+use std::collections::HashMap;
+use std::fs::{read_dir, remove_dir_all, remove_file, File};
+use std::io::{Read, Seek, SeekFrom};
+use std::path::{Path, PathBuf};
+use std::str::FromStr;
+
+use flate2::read::GzDecoder;
+pub use oci_spec::image::{Arch, Config};
+use oci_spec::image::{ImageConfiguration, ImageManifest, MediaType};
+use oci_spec::OciSpecError;
+use sha2::digest::generic_array::GenericArray;
+use sha2::{Digest, Sha256};
+use tar::{Archive, EntryType};
+use thiserror::Error;
+
+mod oci_tar_image;
+use oci_tar_image::{OciTarImage, OciTarImageBlob};
+
+const WHITEOUT_PREFIX: &str = ".wh.";
+const OPAQUE_WHITEOUT_NAME: &str = ".wh..wh..opq";
+
+fn compute_digest<R: Read, H: Digest>(
+ mut reader: R,
+ mut hasher: H,
+) -> std::io::Result<GenericArray<u8, H::OutputSize>> {
+ let mut buf = proxmox_io::boxed::zeroed(32768);
+
+ loop {
+ let bytes_read = reader.read(&mut buf)?;
+ if bytes_read == 0 {
+ break Ok(hasher.finalize());
+ }
+
+ hasher.update(&buf[..bytes_read]);
+ }
+}
+
+fn compute_sha256<R: Read>(reader: R) -> std::io::Result<oci_spec::image::Sha256Digest> {
+ let digest = compute_digest(reader, Sha256::new())?;
+ Ok(oci_spec::image::Sha256Digest::from_str(&format!("{digest:x}")).expect("valid digest"))
+}
+
+/// Build a mapping from uncompressed layer digests (as found in the image config's `rootfs.diff_ids`)
+/// to their corresponding compressed-layer digests (i.e. the filenames under `blobs/<algorithm>/<digest>`)
+fn build_layer_map<R: Read + Seek>(
+ mut oci_tar_image: OciTarImage<R>,
+ image_manifest: &ImageManifest,
+) -> Result<
+ (
+ OciTarImage<R>,
+ HashMap<oci_spec::image::Digest, oci_spec::image::Descriptor>,
+ ),
+ ExtractError,
+> {
+ let mut layer_mapping = HashMap::new();
+
+ for layer in image_manifest.layers() {
+ let digest = match layer.media_type() {
+ MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => layer.digest().clone(),
+ MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
+ let mut compressed_blob = oci_tar_image
+ .open_blob(layer.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer.digest().clone()))?;
+ let decoder = GzDecoder::new(&mut compressed_blob);
+ let hash = compute_sha256(decoder)?.into();
+ oci_tar_image = compressed_blob.into_oci_tar_image();
+ hash
+ }
+ MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
+ let mut compressed_blob = oci_tar_image
+ .open_blob(layer.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer.digest().clone()))?;
+ let decoder = zstd::Decoder::new(&mut compressed_blob)?;
+ let hash = compute_sha256(decoder)?.into();
+ oci_tar_image = compressed_blob.into_oci_tar_image();
+ hash
+ }
+ // Skip any other non-ImageLayer related media types.
+ // Match explicitly to avoid missing new image layer types when oci-spec updates.
+ MediaType::Descriptor
+ | MediaType::LayoutHeader
+ | MediaType::ImageManifest
+ | MediaType::ImageIndex
+ | MediaType::ImageConfig
+ | MediaType::ArtifactManifest
+ | MediaType::EmptyJSON
+ | MediaType::Other(_) => continue,
+ };
+
+ layer_mapping.insert(digest, layer.clone());
+ }
+
+ Ok((oci_tar_image, layer_mapping))
+}
+
+#[derive(Debug, Error)]
+pub enum ProxmoxOciError {
+ #[error("Error while parsing OCI image: {0}")]
+ ParseError(#[from] ParseError),
+ #[error("Error while extracting OCI image: {0}")]
+ ExtractError(#[from] ExtractError),
+}
+
+/// Extract the rootfs of an OCI image tar and return the image config.
+///
+/// # Arguments
+///
+/// * `oci_tar_path` - Path to the OCI image tar archive
+/// * `rootfs_path` - Destination path where the rootfs will be extracted to
+/// * `arch` - Optional CPU architecture used to pick the first matching manifest from a multi-arch
+/// image index. If `None`, the first manifest will be used.
+pub fn parse_and_extract_image<P: AsRef<Path>>(
+ oci_tar_path: P,
+ rootfs_path: P,
+ arch: Option<&Arch>,
+) -> Result<Option<Config>, ProxmoxOciError> {
+ let (oci_tar_image, image_manifest, image_config) = parse_image(oci_tar_path, arch)?;
+
+ extract_image_rootfs(oci_tar_image, &image_manifest, &image_config, rootfs_path)?;
+
+ Ok(image_config.config().clone())
+}
+
+#[derive(Debug, Error)]
+pub enum ParseError {
+ #[error("OCI spec error: {0}")]
+ OciSpec(#[from] OciSpecError),
+ #[error("Wrong media type")]
+ WrongMediaType,
+ #[error("IO error: {0}")]
+ Io(#[from] std::io::Error),
+ #[error("Unsupported CPU architecture")]
+ UnsupportedArchitecture,
+ #[error("Missing image config")]
+ MissingImageConfig,
+}
+
+fn parse_image<P: AsRef<Path>>(
+ oci_tar_path: P,
+ arch: Option<&Arch>,
+) -> Result<(OciTarImage<File>, ImageManifest, ImageConfiguration), ParseError> {
+ let oci_tar_file = File::open(oci_tar_path)?;
+ let mut oci_tar_image = OciTarImage::new(oci_tar_file)?;
+
+ let image_manifest = oci_tar_image
+ .image_manifest(arch)
+ .ok_or(ParseError::UnsupportedArchitecture)??;
+
+ let image_config_descriptor = image_manifest.config();
+
+ if image_config_descriptor.media_type() != &MediaType::ImageConfig {
+ return Err(ParseError::WrongMediaType);
+ }
+
+ let mut image_config_file = oci_tar_image
+ .open_blob(image_config_descriptor.digest())
+ .ok_or(ParseError::MissingImageConfig)?;
+ let image_config = ImageConfiguration::from_reader(&mut image_config_file)?;
+
+ Ok((
+ image_config_file.into_oci_tar_image(),
+ image_manifest,
+ image_config,
+ ))
+}
+
+#[derive(Debug, Error)]
+pub enum ExtractError {
+ #[error("Incorrectly formatted digest: \"{0}\"")]
+ InvalidDigest(String),
+ #[error("Unknown layer digest {0} found in rootfs.diff_ids")]
+ UnknownLayerDigest(oci_spec::image::Digest),
+ #[error("Layer file {0} mentioned in image manifest is missing")]
+ MissingLayerFile(oci_spec::image::Digest),
+ #[error("IO error: {0}")]
+ Io(#[from] std::io::Error),
+ #[error("Layer has wrong media type: {0}")]
+ WrongMediaType(String),
+}
+
+fn extract_image_rootfs<R: Read + Seek, P: AsRef<Path>>(
+ oci_tar_image: OciTarImage<R>,
+ image_manifest: &ImageManifest,
+ image_config: &ImageConfiguration,
+ target_path: P,
+) -> Result<(), ExtractError> {
+ let (mut oci_tar_image, layer_map) = build_layer_map(oci_tar_image, image_manifest)?;
+
+ for layer in image_config.rootfs().diff_ids() {
+ let layer_digest = oci_spec::image::Digest::from_str(layer)
+ .map_err(|_| ExtractError::InvalidDigest(layer.to_string()))?;
+ let layer_descriptor = layer_map
+ .get(&layer_digest)
+ .ok_or(ExtractError::UnknownLayerDigest(layer_digest.clone()))?;
+ let mut layer_file = oci_tar_image
+ .open_blob(layer_descriptor.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer_digest))?;
+
+ type DecodeFn<T> = Box<dyn for<'a> Fn(&'a mut T) -> std::io::Result<Box<dyn Read + 'a>>>;
+ let decode_fn: DecodeFn<OciTarImageBlob<R>> = match layer_descriptor.media_type() {
+ MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => {
+ Box::new(|file| Ok(Box::new(file)))
+ }
+ MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
+ Box::new(|file| Ok(Box::new(GzDecoder::new(file))))
+ }
+ MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
+ Box::new(|file| Ok(Box::new(zstd::Decoder::new(file)?)))
+ }
+ // Error on any other non-ImageLayer related media types.
+ // Match explicitly to avoid missing new image layer types when oci-spec updates.
+ media_type @ (MediaType::Descriptor
+ | MediaType::LayoutHeader
+ | MediaType::ImageManifest
+ | MediaType::ImageIndex
+ | MediaType::ImageConfig
+ | MediaType::ArtifactManifest
+ | MediaType::EmptyJSON
+ | MediaType::Other(_)) => {
+ return Err(ExtractError::WrongMediaType(media_type.to_string()))
+ }
+ };
+
+ apply_whiteouts(&mut decode_fn(&mut layer_file)?, &target_path)?;
+ layer_file.seek(SeekFrom::Start(0))?;
+ extract_archive(&mut decode_fn(&mut layer_file)?, &target_path)?;
+
+ oci_tar_image = layer_file.into_oci_tar_image();
+ }
+
+ Ok(())
+}
+
+/// Apply whiteouts on previous layers
+fn apply_whiteouts<R: Read, P: AsRef<Path>>(reader: &mut R, target_path: P) -> std::io::Result<()> {
+ let mut archive = Archive::new(reader);
+
+ for entry in archive.entries()? {
+ let file = entry?;
+ if file.header().entry_type() != EntryType::Regular {
+ continue;
+ }
+
+ let filepath = file.path()?;
+ if let Some(filename) = filepath.file_name() {
+ if filename == OPAQUE_WHITEOUT_NAME {
+ if let Some(parent) = filepath.parent() {
+ let whiteout_abs_path = target_path.as_ref().join(parent);
+ if whiteout_abs_path.exists() {
+ for direntry in read_dir(whiteout_abs_path)? {
+ remove_path(direntry?.path())?;
+ }
+ }
+ }
+ } else if let Some(filename) = filename.to_str() {
+ // TODO: Simplify this once OsStr::strip_prefix is implemented
+ if let Some(filename_stripped) = filename.strip_prefix(WHITEOUT_PREFIX) {
+ let whiteout_path = match filename_stripped {
+ "." => match filepath.parent() {
+ Some(p) if p.parent().is_some() => p,
+ _ => continue, // Prevent whiteout of root directory
+ },
+ ".." => continue, // Prevent whiteout of grandparent directory
+ fname => &filepath.with_file_name(fname),
+ };
+ let whiteout_abs_path = target_path.as_ref().join(whiteout_path);
+ if whiteout_abs_path.exists() {
+ remove_path(whiteout_abs_path)?;
+ }
+ }
+ }
+ }
+ }
+
+ Ok(())
+}
+
+fn extract_archive<R: Read, P: AsRef<Path>>(reader: &mut R, target_path: P) -> std::io::Result<()> {
+ let mut archive = Archive::new(reader);
+ archive.set_preserve_ownerships(true);
+ archive.set_preserve_permissions(true);
+ archive.set_unpack_xattrs(true);
+
+ // Delay directory entries until the end (they will be created if needed by descendants),
+ // to ensure that directory permissions do not interfere with descendant extraction.
+ let mut directories = Vec::new();
+ for entry in archive.entries()? {
+ let mut file = entry?;
+ if file.header().entry_type() == EntryType::Directory {
+ directories.push(file);
+ continue;
+ } else if file.header().entry_type() == EntryType::Regular {
+ // Skip whiteout files
+ if let Some(filename) = file.path()?.file_name() {
+ if filename == OPAQUE_WHITEOUT_NAME {
+ continue;
+ } else if let Some(filename) = filename.to_str() {
+ if filename.starts_with(WHITEOUT_PREFIX) {
+ continue;
+ }
+ }
+ }
+ }
+
+ file.unpack_in(&target_path)?;
+ }
+
+ // Apply the directories in reverse topological order,
+ // to avoid failure on restrictive parent directory permissions.
+ directories.sort_by(|a, b| b.path_bytes().cmp(&a.path_bytes()));
+ for mut dir in directories {
+ dir.unpack_in(&target_path)?;
+ }
+
+ Ok(())
+}
+
+fn remove_path(path: PathBuf) -> std::io::Result<()> {
+ if path.metadata()?.is_dir() {
+ remove_dir_all(path)
+ } else {
+ remove_file(path)
+ }
+}
diff --git a/proxmox-oci/src/oci_tar_image.rs b/proxmox-oci/src/oci_tar_image.rs
new file mode 100644
index 00000000..23e1bfe0
--- /dev/null
+++ b/proxmox-oci/src/oci_tar_image.rs
@@ -0,0 +1,144 @@
+use std::collections::HashMap;
+use std::io::{Read, Seek, SeekFrom};
+use std::ops::Range;
+use std::path::{Path, PathBuf};
+
+use oci_spec::image::{Arch, Digest, ImageIndex, ImageManifest, MediaType};
+use oci_spec::OciSpecError;
+use tar::Archive;
+
+use proxmox_io::RangeReader;
+
+#[derive(Clone)]
+struct TarEntry {
+ range: Range<u64>,
+}
+
+impl TarEntry {
+ fn new(range: Range<u64>) -> Self {
+ Self { range }
+ }
+}
+
+pub struct OciTarImage<R: Read + Seek> {
+ reader: R,
+ entries: HashMap<PathBuf, TarEntry>,
+ image_index: ImageIndex,
+}
+
+impl<R: Read + Seek> OciTarImage<R> {
+ pub fn new(reader: R) -> oci_spec::Result<Self> {
+ let mut archive = Archive::new(reader);
+ let entries = archive.entries_with_seek()?;
+ let mut entries_index = HashMap::new();
+ let mut image_index = None;
+
+ for entry in entries {
+ let mut entry = entry?;
+ let offset = entry.raw_file_position();
+ let size = entry.size();
+ let path = entry.path()?.into_owned();
+
+ if path.as_path() == Path::new("index.json") {
+ image_index = Some(ImageIndex::from_reader(&mut entry)?);
+ }
+
+ let tar_entry = TarEntry::new(offset..(offset + size));
+ entries_index.insert(path, tar_entry);
+ }
+
+ if let Some(image_index) = image_index {
+ Ok(Self {
+ reader: archive.into_inner(),
+ entries: entries_index,
+ image_index,
+ })
+ } else {
+ Err(OciSpecError::Other("Missing index.json file".into()))
+ }
+ }
+
+ pub fn image_index(&self) -> &ImageIndex {
+ &self.image_index
+ }
+
+ fn get_blob_entry(&self, digest: &Digest) -> Option<TarEntry> {
+ let path = get_blob_path(digest);
+ self.entries.get(&path).cloned()
+ }
+
+ pub fn open_blob(self, digest: &Digest) -> Option<OciTarImageBlob<R>> {
+ if let Some(entry) = self.get_blob_entry(digest) {
+ Some(OciTarImageBlob::new(self, entry.range))
+ } else {
+ None
+ }
+ }
+
+ pub fn image_manifest(
+ &mut self,
+ architecture: Option<&Arch>,
+ ) -> Option<oci_spec::Result<ImageManifest>> {
+ let digest = match self.image_index.manifests().iter().find(|d| {
+ d.media_type() == &MediaType::ImageManifest
+ && architecture
+ .is_none_or(|a| d.platform().as_ref().is_none_or(|p| p.architecture() == a))
+ }) {
+ Some(descriptor) => descriptor.digest(),
+ None => return None,
+ };
+
+ if let Some(entry) = self.get_blob_entry(digest) {
+ let mut range_reader = RangeReader::new(&mut self.reader, entry.range);
+ Some(ImageManifest::from_reader(&mut range_reader))
+ } else {
+ Some(Err(OciSpecError::Other(format!(
+ "Image manifest with digest {digest} mentioned in image index is missing"
+ ))))
+ }
+ }
+}
+
+fn get_blob_path(digest: &Digest) -> PathBuf {
+ let algorithm = digest.algorithm();
+ let digest = digest.digest();
+ format!("blobs/{algorithm}/{digest}").into()
+}
+
+pub struct OciTarImageBlob<R: Read + Seek> {
+ range_reader: RangeReader<R>,
+ entries: HashMap<PathBuf, TarEntry>,
+ image_index: ImageIndex,
+}
+
+impl<R: Read + Seek> OciTarImageBlob<R> {
+ fn new(archive: OciTarImage<R>, range: Range<u64>) -> Self {
+ let range_reader = RangeReader::new(archive.reader, range);
+
+ Self {
+ range_reader,
+ entries: archive.entries,
+ image_index: archive.image_index,
+ }
+ }
+
+ pub fn into_oci_tar_image(self) -> OciTarImage<R> {
+ OciTarImage {
+ reader: self.range_reader.into_inner(),
+ entries: self.entries,
+ image_index: self.image_index,
+ }
+ }
+}
+
+impl<R: Read + Seek> Read for OciTarImageBlob<R> {
+ fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
+ self.range_reader.read(buf)
+ }
+}
+
+impl<R: Read + Seek> Seek for OciTarImageBlob<R> {
+ fn seek(&mut self, pos: SeekFrom) -> std::io::Result<u64> {
+ self.range_reader.seek(pos)
+ }
+}
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH proxmox v4 03/15] proxmox-oci: add tests for whiteout handling
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH proxmox v4 01/15] io: introduce RangeReader for bounded reads Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH proxmox v4 02/15] add proxmox-oci crate Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH proxmox-perl-rs v4 04/15] add Perl mapping for OCI container image parser/extractor Filip Schauer
` (11 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Test extracting OCI images with whiteout special cases.
Test cases inspired by:
https://github.com/containerd/containerd/blob/4312e076a8a3/pkg/archive/tar_test.go
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v4
proxmox-oci/Cargo.toml | 3 +
| 92 ++++++++++++++++++
.../oci_test_whiteout_current_directory.tar | Bin 0 -> 8704 bytes
.../oci_test_whiteout_dead_symlink_parent.tar | Bin 0 -> 8704 bytes
.../oci_test_whiteout_root_breakout.tar | Bin 0 -> 7168 bytes
...oci_test_whiteout_root_parent_breakout.tar | Bin 0 -> 7168 bytes
.../oci_test_whiteout_symlink.tar | Bin 0 -> 8704 bytes
7 files changed, 95 insertions(+)
create mode 100644 proxmox-oci/tests/extract_whiteouts.rs
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar
diff --git a/proxmox-oci/Cargo.toml b/proxmox-oci/Cargo.toml
index 4daff6ab..b162bf4f 100644
--- a/proxmox-oci/Cargo.toml
+++ b/proxmox-oci/Cargo.toml
@@ -20,3 +20,6 @@ thiserror = "1"
zstd.workspace = true
proxmox-io.workspace = true
+
+[dev-dependencies]
+proxmox-sys.workspace = true
--git a/proxmox-oci/tests/extract_whiteouts.rs b/proxmox-oci/tests/extract_whiteouts.rs
new file mode 100644
index 00000000..71ec4dea
--- /dev/null
+++ b/proxmox-oci/tests/extract_whiteouts.rs
@@ -0,0 +1,92 @@
+use std::fs::remove_dir_all;
+
+use proxmox_oci::{parse_and_extract_image, Arch};
+use proxmox_sys::fs::make_tmp_dir;
+
+#[test]
+fn test_whiteout_root_breakout() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_root_breakout.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ // Check that the whiteout did not remove the root directory
+ assert!(extract_dir.exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_root_parent_breakout() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ // Check that the whiteout did not remove the root directory
+ assert!(extract_dir.exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_current_directory() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_current_directory.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(!extract_dir.join("etc").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_symlink() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_symlink.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(extract_dir.join("etc/passwd").exists());
+ assert!(!extract_dir.join("localetc").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_dead_symlink_parent() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(extract_dir.join("etc/passwd").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar
new file mode 100644
index 0000000000000000000000000000000000000000..56b34cfd774b74b8410839bb48333a38409b70d9
GIT binary patch
literal 8704
zcmeHLO^6&t6mB*q#3g!342vKc1`LtSPO9p4Rdp|tlb4`}#RLxti(9`lz0A)tGaIrA
zYd}yC&DoGcSn(o)H&34Q5IiafdhsfF^&%b&@m0@cC$r;j;?B&%V%MR6URS+(^<I77
zd)2z$(wz&@R2#v@35miT+s^xwvfeW#OtPrNr)pJS`Ff|T+Gx_aa|O$>Z7*(IeFrpN
z=kf2<6ysuA+aUPmzr~*dDdK-1J76gOh7zk%sbfF{aw3>ad7PwPLP`@Qq@!SgE5ekv
z95|;$0+&h%pK!~q-o@e&|M&@Lgm-L@p+XqY?dM$1C<q>v#>V6>_6&*t=lDPK@$utd
z-219Fb3_3M3OoUoKuTCDXyUnZi2}*EW&&G^`@j=rg-nEuDL9kr*gC^}0ua5qaOvk?
z=70F{C$semiDuyBN%F#TzsGkQwbhl?dc9WHCr-(jaPF4K%GQx{3zbXfXBIzr`TiS!
z{(5fi=;te^PPC)w{`@zuM#qLiRJ=^j14H=Nf+4Dj2PzE}OgP|H#lW>pxC8?>rkW^W
zK&3(%%?T5!L1-KB+6mY<{>6X``uLX=3j9wm>I2OI|7X7p_}8GBN?FRZrHNoj?x`lR
zaUMj>ElG*dNcSKGc;cAQQgDt}#vzf;Tc6{<x<wKCoy(Vd(0_R5#f$GD`3nKnCF1&^
ze|u?V?z7TMbB~UG^x(m7uYG;zHgd)5f3!Y4{phSe_U&JC?#}IdN3MSVu*V9Yy#L+v
z(Z#>-ej3dV>@J=r<AEXkQ|B3mR(PeX3aNk+ZG^{N5Ku2uP$psCSR+-;4fj$zNW9Tl
zA(&XH{o<cfxg-BGDX{)4^8aLFyqA_+v!zm5wM{MSI@4WmyYeEwH>`-u=ffw{cB<=a
zkTR?Z7f7<e_!UYQNxDdQ1q7xH#^U&DuX`=)bX)BW?0#)IAK&VPq4lh}exui;-D-7x
z5Qdp?g6_r|&RMRj4cG4ET~KH<@88WVcDt7MLQ(ICvN$eF^=B9;9JlN#=LRn0b#r#}
z@DbDy{*{P5!E#Rv=egz~scWSrW_Xrb=DeXPagI~Py}@)(EMbnOoZ8rtSaG(?MhnIM
z9`!#Rt^XO5w7~x^?wqJN$A8DvTtmHs0L)rVgeJ?O+i*5hS8{kFC9JL0Go!lL;llN%
zt+Y&5$r`HaDh<`lyz6u;*XjIf2gmn#qt*0TbpZ2cRA%yFL#lE*sNw{HNct==ZlZe3
zGo9>aAUpsb@S7XnCn6-U9N0aULK~iO0r_yWzk+|2=BqcewE^MSY{x3bJ#G(F_t!x=
z<VNQ!DfJn4I<}8{R2a2gik|IwT|SIVI#F02n6j*C-Hl4X?ewO63jY(}JO2MrN@ZN|
z|MVg<(7YG__lZ3eYx^U^e@zW})i=V0<CY#Hd?G~P^iY)*d@>04$P?!A1<YBk_3p^0
zi3y&De*$9o{6{ICH46MsEC72bx>e4rxVQ2xuw4O`X@yisd2^3}#izmpg$D``6dpL#
G9{2}Zftk|)
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar
new file mode 100644
index 0000000000000000000000000000000000000000..b1b5094689402cce0433174f79c4595c10dae392
GIT binary patch
literal 8704
zcmeHLU5Hdw93K=H8)k*T@*$~h?Bl_9JfHX6yDu7)U}~C5LLuw<+`YOpv(DTVM_baO
zAYADo>Y-?%5A&&njE$_fC=7)xq8@w*q`)2`33^Df?SJo#wzKQBqq8eW_dMLW_dn<3
zcmC)6e&>JwYN)KLt1|Oi5EPD3CjNyo{dcOZ{*Dn5B-6v^6;-42k!mC>0IAclV&qV+
zPOj7bZc=-dtbcV-67J2{Hi-Rled~`YO4t9U?0}~AR}9ukVro2PJ>_}MNyM#2fwDeB
z#Bt(dC#AHOAdh=SaxN&>0aWz_30w=r@%pnV)QG>KT%igk6kmC-+X=xQtbfOxY=NiA
z{6FXafoxAtzN!apQ9fxa)v#R3_w|s6Vr$WwP;RRYTd?Hi@bFNmWds#g?Jb!?S%-zN
zC<knzC`+NU)u^x+uiRCI_DN%1F1c_3TE+vk-&ws`Q&-T~gaa@@45KboeLm6>Nx`fV
z#%e#XlpA5K@QAXKD<iQMJfMNoL}&>WxUkwGB1yoBQY^t3PdwZo&yG?KC`S|EL$U^b
z0OQjM@H+hVCy+D;`~^5wRu1eBhZ{p(YZjg&fKvya(@X$DBb*b+X(N@4>Eswuz&hbH
zwH#qfg+)GPK2yeUYuvwGH|BvENkAtv>1nurez03(F)$yP>LeG+cvx{PsHKi6t&ugv
zC*X2VBdwiKm=i%2p^SQ#IAwyF*r}*vhv$y$Xts%@>BoF|ph^EbrBw@%v8ext{v+5p
z)AgTwBGvyE?wnI|qW`j@2gAtfXrxjX`=asc8kyG>D<f-S?InotUgY-@z769(6!alp
zpq@u4A&fnh5IR2?R-<yI7R%?3L{7`Gl;(%R(#XDgjY_#3x!Bk^7^38W$v0#-Zj(r*
zbN!)#YIPL`Q$zV7Bq0nGQwClTXhS6gCmacewWL@&%(!q6#xQF*NE0`V351!mL4YK4
zN=(t6B<AhYR2$FZ92A!Y4B}1(T4g|kaUeA%Txp+!;ChrGUm=0H&$-r?_)J^h1YSNt
zyaS%2x1wq_@xRjpsm0EmjsFns&EUVlDgRrzb570o9A3P*cHx7;1q+p|+Bj;-{Q4cv
z9z?lCXO|xp=$nc0V}l>Rt?oEp9^SI_!4=us`janQ{N?4XTMr_$WX~J>#+Utea^lXh
z=a-$SWmg=3=KQJ4dsg2bOy2v`XCHm|=ecu#UB=Jmf4@7o>+kdLp4k55YnL9luW)G7
zAFH4E`IE`h6NfKNzPjbHFSFwp-&&u^&FRF_&f4-o6aPC6>ROCAJO5|g{}^K_|690o
zPR%j@FTDPK%zrrPcb7+^{s`_`{oCV!0KOiv(rm32;Q-&gSypT*N+ja+SDPL`v#xjD
z_dl%s`s~7Qb1c)fW~WmN-aC|ev+G3n#_kPAj&vW{ICkj#i)+_*xm>m@7iErqasNw?
zt~s{(^H)xd-?iwSolD1_dw5Ca)NP+G&veb{oYKzP_dr-Oc3&aBx3}NsdTY+cf6Axv
zA7d<1{$H<~Z>W9}|F_8hG!i>Eo(#EWYN!$5Ps`|ZntGfP*Vl785Ys1-`6xWod_v2;
z0eQS$xO;)px2)3fAG7fv(P{BN#&Fk6&;O1c-$n-q@<|qVL$U>~W`Og!fC?x-+C~Ry
RmDB^N2T~8D9=Ihv@DEuzwG;pV
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar
new file mode 100644
index 0000000000000000000000000000000000000000..36e25851148064fdc39d0fee90bdd5762e187989
GIT binary patch
literal 7168
zcmeHL%ZeLE6m35+X2`5;jCT=D7J<pM>)BPMW!8aU7_vwhl6sDMkko_JV>}+TF=Uf?
zX^1fd@&oya{6Gx;iF`tqxm8j|mNN2;qmc=5*P_*Zse9`_>eRU<)343=LAKF>U~q)8
z_zPqC_f6UK9U~%0*5MoVYG(X++~^UwwC-Bba~NmAt!wW+>6=vlaZeLoZfYCEyu4fe
zF-2|t-^&hIsK0ewIa~(kC^gD)jhPqHV?|sUj0s-DgHVd8@|H{M5n&2Di<~x$yK;rc
zvHt80XvA+Ae?tWc%Fo`DoRE^ztg|*ISNO9a{=o5ny4C6AW83os{goe$t9qDsJLIz{
z9e7viy^|vkl3X7h^((6znDF4y&=s|<ifW(_d@<0&D)@0zJi<GNW6)2^wjPG+05szQ
zmp3+5TQCL6t~!7Pq8rnLs#_ApFcZo$;%Mn4a{+5CEi+0A%T>VEawlC$L<kD@rqwba
zZIF;9Gu|X0W8l@37#-mZyqqM*`}zd(0K#vQ<5Tx9A3#zE{|jK1or5RU(QK*nqOfI<
zAq1p?S2zH+N`(PU97K#sf+?VmDW^Tv)=<wF^IS>`N+2{r&P~taz)U8mC-B_gnrtm>
z^z%O!@xP`+=P!}4{Wkm)!j|zbIBM~~BAr(%{`kk+x4(b+Yw!9E&kz<$Ifton0TFHq
z6h=FQgAiJ1!%JIg*npCIg(TB)<5DAm2pEwDXRmjDfA!k_vWv3o^v)f$ee)0YV$eJM
z?y%qQ_030j#myh?<4^CSSJ$53`{|Q=*M9zd`^)@vc=y@UhiL1T%Cf!R{@l)PU1@l&
zu?sS=fd6%m)fH<+vVR-?sVJB64^2jk{}t)HQgPycJ+i&3@wPc0P04=XN;c#1k=KoP
zk$VZ83A;${626CV7pX4d1uaWL;h11wPp9{)aZ`^@B7N>BI;|s>=KX4T{CGNJRM$<2
z!dYX`qy{6O(Qed9R2jmzm4j(h7LHV%t|(n5EM?Af?X^^f2%)&P*a72vhdB<$y5KR)
z2o4KgBBr>Y5(zl0IEB+IPYdkBb8<O?ywYs{_l#9T=N}ibZ`+)27vi7eMf_uo1#9uY
zxr)p!C;YGI|7VT;zi)=my&B8_zSPmLd-eQBFU*8UP=AMBc%GFH=MC<0yKwf_$Ia$q
zz6k$_F7ZFcz%?!YH&>Cl<<mUf;=f2+;OrlO92clS`D8w?eQPt&W}wYLn}PpV2L1xB
CF8omd
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar
new file mode 100644
index 0000000000000000000000000000000000000000..12421d00acc42a589f2ccaf50cd085ceb34c4ad4
GIT binary patch
literal 7168
zcmeHKOK%iM5MCK4LKd9h5OKlU6Ucbz_hS#7azWyd1L6R(a8EzHW9{x(J7a`FK61)A
zQbeQ(DYr=ZEs5kSKOny#XR3P_n>DPxWV{?;TH2kNn(FH6>iWK_$p>X#-R&%OAQ&8>
zPQ1c6ef>~2xyOh|(pl$A^=fK-zpBj`Txz;j^c=?V=9=33Li6)<{#D-)E|!%BF)p8X
z{+Ocn{J%&B%$&bRURY+OVT36{xz@(B;JGANGR+ZIG&fdR61byMGR3^l9aYo@NFQqE
zcpT54eE^O44C6gi5S8NiKFJA*m-WV&oa4`o_#4On%eD3Ot{K~YQF~kOkA1fX&qF6z
z_fC9e$KKT5MJYjqZz8ox_(P0)NcRxWs1Sq_!r0@9dcUaZa(n>lPqxzhvWim&#c2Qe
zL}Of*b%?^aGAvLZ?7^DdfjRKwDye|b6(Kw=9H>1>S|FNGR8K%^Qj9E;!C<Ym@_{3Z
zEz=5HDRU}_7M5efl)#EeYLK)H(pFgoa@|e#w&4G8?Qm@-6TZoZ8T^NUbFNS>1f^Cp
zs&nhDM<`&;92F2s=Q!t-IW3G*7;7WAl))KE16E>A1S8nb!oQ^aWBk(=|8v55q2h%9
z%J%)xJOTg~<p`ij5_E^&73N`rCs4xP-k`9i28FxNMlLIDkrhL;<Fla|6~R|^_6%?A
zRxp3U8|5ezJ0~z-(3nlrW=thI*ntHikxW^jX`y=0Gga{-5+30!5<+r(N<=_li76&5
z1wJgefixZb&!DT^*m+UxO_%!AE;!{zX>CJp19l2wL0JeWXsn#nNWf19flzKVWr%Z)
zCD)vq96KJMN}rgX?gKNOoIZsAt;OhV2LDYRnv<F5;h&Jx_=iZy7XNd?d7<K)KVQD=
zU;ou#xzx0>c8)gw{^#h(zPXJ$EA-kmbanM#_Uo{}yS+OY^auHs>vHu^x&0H`esk&8
zW_JJ1O7GQ;-+%n??X4fa`(yk1m2s!@%WtdqI^SRD{#s*aN1zxv|2&JleYRecx98!X
zay(oAF^2Dr7XM9RzUbVu>i@K`$7cQi6~j-q2C3_J#i@$kwDtU`7BVS6w-zoMv#&`H
z`nXN#S<3x-dAm9f|A?OY{>K=;HCy~I??g^4zwD+i{y|EC<1c_N&QONBhbMaNT^oTm
N0&N7^2z*@;cn3S&@<;#x
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar
new file mode 100644
index 0000000000000000000000000000000000000000..7e6835cab889566cdf1d8dc1a42b69642b9d0463
GIT binary patch
literal 8704
zcmeHL-D_M$6u(Kdl$bo!hEVjuuoMlZ+0OUeSrC0Fg`j95ML`M8d}eRDA8~h6l9oV0
z{D3?Og28|WKVF0iz6t&VN?QyFeG)_p;;RUKlGd5K+nSBtW|K_{#XApo?>&3Y%$YOu
z`^}l7N>x=q869W?zEJ?8@ax<5->tIdI|PWMXpA1HSFO$$>kV1+F7-NA@Z7h1lk2s=
zhh(qP^{*EtqU@lyL5R!l)*oV!t^Y&W0d4EAq*e@AV2ISxafCP!MiL+ew}v2zfmFc8
zmRT875JP7PHc&IHsWMWU4jzZ~CtIKqzJ0q36^Jmr``(lz#4wM>dd6f2f7-<VbNpW#
z92?8kb<vjP>$X-;s+C+}44nwlvNef3yS!k%BqtXZN{N;YuW<50#pJ6x$tPtwXY*xQ
zNt~@W@)zL5eBIlp#=2T@$(*+g19ab6)2yv4Z)}n|KR|G!NmK>R9T6I92_W<vRT3`!
z282ooY$SukQ7a9B8WE$d<{Sz|B_x(;MF@2~^)XC5xfo`LG4)=qq>h*5vX6rozLh%O
zbiexvq~_lLygyYY<}M}+t)cEWi`WSvIOA3d<6#DcWJDN?Bw@tH0>uhqOtdrrN+hHM
zkj26xU~vqv=IGz9Tl0W8PMuC?GS&+H!T64ViMg{%OR1cSjmE}uj$v$L=vRUPg#^<a
zBO@3i$V)L$R1hGvl@5Xb`i{aH_*L-FnZcF8Hjri?2Ni)f{_ho59h?lL{s8_56!Pu-
zPchE;ze70pRh;s_tm$IXu)48WYm$B0Fp?rPr)yR=tO?wU0HTv1o<#I4gasf9faWn{
z2qQ$un@#m%Qg2ji%Yi;;QaY^$D$SLW%HmQ}qgJgpToASnhDLh8<XW^FY*MOp?p$KL
zT9bvnsKxwTJIDluK%$sAE@BN7r^-_$@~$8zrFKFwEQkY?P>rde3L~MNCW^%I7T!s7
z-h%a7ELMQ|Kc!?wX-~aA*P%ef2KfVpSVlQyRzM4lLR@j@6CbT(M|`4$bE(HO{<-of
zvQ``Zdp(dk?983;k09EHf5;)s@ZTYv`zn5B_3+`%>)#an2b8Sa=;g_&8*>{FME%nP
z1E73)O3U^2^))cQhBo^?8vXeAXy27D-+S)E7x<}{o(H#|cye|1>U8nxJI^lN{B?-l
z9v&XPe(l)E(CM3>{eJnz+b<3t;y?WO+N;<4M*H9S`fpAM<K_(b{M+wO|8-~l*yCT!
z9v_Hi|NQE=XlP$2m37v21lsW5Yf{%?)Sd7TIo*bTglLBU4$<6Salrp~pIiy}_b2`N
z>SE(u!{4>ey%7=u|24f__J{ZRaGi;Mp7?3=YJl_9<doYu;z<G@Jqm^&`-7~Ni}SPd
zrBbn^hDJQhA#-QI?8d<OWPa*If8pJyK0S5omlW_LL$xUS;MV&`qC@*Tk*v2pMIfmd
zyOa<2_C1+*WwaCiu?Kt5X|>2->LDlD`QKG=dsIsE{|@z^R$*uF#gO}!hFS^!wu(-#
zrN=FKeY2*+SXNKyVsH35-lV^Kd8Y3|d5<aq*AKe?1I)JL-`_Pe{O>W`yE40!ORKoA
hq+8%#1vm%uAP;gYyXq=a&LWUSAd5g2fk!9;{{UmyntT8N
literal 0
HcmV?d00001
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH proxmox-perl-rs v4 04/15] add Perl mapping for OCI container image parser/extractor
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (2 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH proxmox v4 03/15] proxmox-oci: add tests for whiteout handling Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH lxc v4 05/15] lxc: conf: split `lxc.environment` into `runtime` and `hooks` Filip Schauer
` (10 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This patch depends on the proxmox-oci crate added in patch 02/15.
Changed since v3:
* improve rustdoc comments
* parse_and_extract_image: add argument for CPU architecture
Changed since v2:
* rebase onto newest master (6132d4d36cbd)
* forward all errors to Perl
* remove oci-spec dependency
Changed since v1:
* rebase on latest master (3d9806cb3c7f)
* add new dependencies to debian/control
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 1 +
pve-rs/debian/control | 1 +
pve-rs/src/bindings/mod.rs | 3 +++
pve-rs/src/bindings/oci.rs | 29 +++++++++++++++++++++++++++++
5 files changed, 35 insertions(+)
create mode 100644 pve-rs/src/bindings/oci.rs
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index 963031a..905c5c9 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -39,6 +39,7 @@ proxmox-http-error = "1"
proxmox-log = "1"
proxmox-network-types = "0.1"
proxmox-notify = { version = "1", features = ["pve-context"] }
+proxmox-oci = "0.1.0"
proxmox-openid = "1.0.2"
proxmox-resource-scheduling = "1"
proxmox-section-config = "3"
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index 13a5418..aa7181e 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -27,6 +27,7 @@ PERLMOD_GENPACKAGE := /usr/lib/perlmod/genpackage.pl \
PERLMOD_PACKAGES := \
PVE::RS::Firewall::SDN \
+ PVE::RS::OCI \
PVE::RS::OpenId \
PVE::RS::ResourceScheduling::Static \
PVE::RS::SDN::Fabrics \
diff --git a/pve-rs/debian/control b/pve-rs/debian/control
index d0b1af6..091406b 100644
--- a/pve-rs/debian/control
+++ b/pve-rs/debian/control
@@ -25,6 +25,7 @@ Build-Depends: cargo:native <!nocheck>,
librust-proxmox-log-1+default-dev,
librust-proxmox-notify-1+default-dev (>= 0.5.4),
librust-proxmox-notify-1+pve-context-dev,
+ librust-proxmox-oci-0.1+default-dev,
librust-proxmox-openid-1+default-dev (>= 1.0.2-~~),
librust-proxmox-resource-scheduling-1+default-dev,
librust-proxmox-shared-cache-1+default-dev,
diff --git a/pve-rs/src/bindings/mod.rs b/pve-rs/src/bindings/mod.rs
index 7730de3..c21b328 100644
--- a/pve-rs/src/bindings/mod.rs
+++ b/pve-rs/src/bindings/mod.rs
@@ -1,5 +1,8 @@
//! This contains all the perl bindings.
+mod oci;
+pub use oci::pve_rs_oci;
+
mod resource_scheduling_static;
pub use resource_scheduling_static::pve_rs_resource_scheduling_static;
diff --git a/pve-rs/src/bindings/oci.rs b/pve-rs/src/bindings/oci.rs
new file mode 100644
index 0000000..7f5acaf
--- /dev/null
+++ b/pve-rs/src/bindings/oci.rs
@@ -0,0 +1,29 @@
+#[perlmod::package(name = "PVE::RS::OCI", lib = "pve_rs")]
+pub mod pve_rs_oci {
+ //! The `PVE::RS::OCI` package.
+ //!
+ //! Provides bindings for the [`proxmox_oci`] crate.
+
+ use anyhow::Error;
+ use proxmox_oci::Config;
+
+ /// Extract the rootfs of an OCI image tar and return the image config.
+ ///
+ /// # Arguments
+ ///
+ /// * `oci_tar_path` - Path to the OCI image tar archive
+ /// * `rootfs_path` - Destination path where the rootfs will be extracted to
+ /// * `arch` - Optional CPU architecture used to pick the first matching manifest from a multi-arch
+ /// image index. If `None`, the first manifest will be used.
+ #[export]
+ pub fn parse_and_extract_image(
+ oci_tar_path: &str,
+ rootfs_path: &str,
+ arch: Option<&str>,
+ ) -> Result<Config, Error> {
+ let arch = arch.map(Into::into);
+ proxmox_oci::parse_and_extract_image(oci_tar_path, rootfs_path, arch.as_ref())
+ .map(|config| config.unwrap_or_default())
+ .map_err(Into::into)
+ }
+}
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH lxc v4 05/15] lxc: conf: split `lxc.environment` into `runtime` and `hooks`
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (3 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH proxmox-perl-rs v4 04/15] add Perl mapping for OCI container image parser/extractor Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 06/15] config: add `lxc.environment.runtime`/`hooks` Filip Schauer
` (9 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Introduce `lxc.environment.runtime` to set environment variables only
for the container init process and `lxc.environment.hooks` to set
environement variables only for hooks. Leave the original
`lxc.environment` unchanged. It still applies to everything.
This will be needed by containers created from OCI images with custom
environment variables that could interfere with hooks.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v4
...c.environment-into-runtime-and-hooks.patch | 324 ++++++++++++++++++
debian/patches/series | 1 +
2 files changed, 325 insertions(+)
create mode 100644 debian/patches/pve/0003-PVE-conf-split-lxc.environment-into-runtime-and-hooks.patch
diff --git a/debian/patches/pve/0003-PVE-conf-split-lxc.environment-into-runtime-and-hooks.patch b/debian/patches/pve/0003-PVE-conf-split-lxc.environment-into-runtime-and-hooks.patch
new file mode 100644
index 0000000..93e81f3
--- /dev/null
+++ b/debian/patches/pve/0003-PVE-conf-split-lxc.environment-into-runtime-and-hooks.patch
@@ -0,0 +1,324 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Filip Schauer <f.schauer@proxmox.com>
+Date: Mon, 8 Sep 2025 11:11:31 +0200
+Subject: [PATCH] PVE: conf: split `lxc.environment` into `runtime` and `hooks`
+
+Introduce `lxc.environment.runtime` to set environment variables only
+for the container init process and `lxc.environment.hooks` to set
+environement variables only for hooks. Leave the original
+`lxc.environment` unchanged. It still applies to everything.
+
+Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
+---
+ src/lxc/attach.c | 6 +++-
+ src/lxc/conf.c | 14 +++++----
+ src/lxc/conf.h | 15 +++++++---
+ src/lxc/confile.c | 72 +++++++++++++++++++++++++++++++++++++++++------
+ src/lxc/start.c | 12 ++++++--
+ 5 files changed, 98 insertions(+), 21 deletions(-)
+
+diff --git a/src/lxc/attach.c b/src/lxc/attach.c
+index 8f2f7a37c..f22f83f0d 100644
+--- a/src/lxc/attach.c
++++ b/src/lxc/attach.c
+@@ -879,7 +879,11 @@ static int lxc_attach_set_environment(struct attach_context *ctx,
+
+ /* Set container environment variables.*/
+ if (ctx->container->lxc_conf) {
+- ret = lxc_set_environment(ctx->container->lxc_conf);
++ ret = lxc_set_environment(&ctx->container->lxc_conf->environment);
++ if (ret < 0)
++ return -1;
++
++ ret = lxc_set_environment(&ctx->container->lxc_conf->environment_runtime);
+ if (ret < 0)
+ return -1;
+ }
+diff --git a/src/lxc/conf.c b/src/lxc/conf.c
+index 1899b2806..7533e2830 100644
+--- a/src/lxc/conf.c
++++ b/src/lxc/conf.c
+@@ -3209,6 +3209,8 @@ struct lxc_conf *lxc_conf_init(void)
+ new->root_nsuid_map = NULL;
+ new->root_nsgid_map = NULL;
+ INIT_LIST_HEAD(&new->environment);
++ INIT_LIST_HEAD(&new->environment_runtime);
++ INIT_LIST_HEAD(&new->environment_hooks);
+ INIT_LIST_HEAD(&new->limits);
+ INIT_LIST_HEAD(&new->sysctls);
+ INIT_LIST_HEAD(&new->procs);
+@@ -4239,18 +4241,18 @@ int lxc_clear_groups(struct lxc_conf *c)
+ return 0;
+ }
+
+-int lxc_clear_environment(struct lxc_conf *c)
++int lxc_clear_environment(struct list_head *environment)
+ {
+ struct environment_entry *env, *nenv;
+
+- list_for_each_entry_safe(env, nenv, &c->environment, head) {
++ list_for_each_entry_safe(env, nenv, environment, head) {
+ list_del(&env->head);
+ free(env->key);
+ free(env->val);
+ free(env);
+ }
+
+- INIT_LIST_HEAD(&c->environment);
++ INIT_LIST_HEAD(environment);
+ return 0;
+ }
+
+@@ -4359,7 +4361,7 @@ void lxc_conf_free(struct lxc_conf *conf)
+ lxc_clear_mount_entries(conf);
+ lxc_clear_idmaps(conf);
+ lxc_clear_groups(conf);
+- lxc_clear_environment(conf);
++ lxc_clear_environment(&conf->environment);
+ lxc_clear_limits(conf, "lxc.prlimit");
+ lxc_clear_sysctls(conf, "lxc.sysctl");
+ lxc_clear_procs(conf, "lxc.proc");
+@@ -5210,11 +5212,11 @@ void suggest_default_idmap(void)
+ ERROR("lxc.idmap = g 0 %u %u", gid, grange);
+ }
+
+-int lxc_set_environment(const struct lxc_conf *conf)
++int lxc_set_environment(const struct list_head *environment)
+ {
+ struct environment_entry *env;
+
+- list_for_each_entry(env, &conf->environment, head) {
++ list_for_each_entry(env, environment, head) {
+ int ret;
+
+ ret = setenv(env->key, env->val, 1);
+diff --git a/src/lxc/conf.h b/src/lxc/conf.h
+index 31cd39e3f..d16564f10 100644
+--- a/src/lxc/conf.h
++++ b/src/lxc/conf.h
+@@ -506,10 +506,17 @@ struct lxc_conf {
+ unsigned int monitor_unshare;
+ unsigned int monitor_signal_pdeath;
+
+- /* list of environment variables we'll add to the container when
+- * started */
++ /* list of environment variables to provide to both the container's init
++ * process and hooks */
+ struct list_head environment;
+
++ /* list of environment variables to provide to the container's init
++ * process */
++ struct list_head environment_runtime;
++
++ /* list of environment variables to provide to container hooks */
++ struct list_head environment_hooks;
++
+ /* text representation of the config file */
+ char *unexpanded_config;
+ size_t unexpanded_len;
+@@ -599,7 +606,7 @@ __hidden extern int lxc_clear_automounts(struct lxc_conf *c);
+ __hidden extern int lxc_clear_hooks(struct lxc_conf *c, const char *key);
+ __hidden extern int lxc_clear_idmaps(struct lxc_conf *c);
+ __hidden extern int lxc_clear_groups(struct lxc_conf *c);
+-__hidden extern int lxc_clear_environment(struct lxc_conf *c);
++__hidden extern int lxc_clear_environment(struct list_head *environment);
+ __hidden extern int lxc_clear_limits(struct lxc_conf *c, const char *key);
+ __hidden extern int lxc_delete_autodev(struct lxc_handler *handler);
+ __hidden extern int lxc_clear_autodev_tmpfs_size(struct lxc_conf *c);
+@@ -710,7 +717,7 @@ static inline int lxc_personality(personality_t persona)
+ return personality(persona);
+ }
+
+-__hidden extern int lxc_set_environment(const struct lxc_conf *conf);
++__hidden extern int lxc_set_environment(const struct list_head *environment);
+ __hidden extern int parse_cap(const char *cap_name, __u32 *cap);
+
+ #endif /* __LXC_CONF_H */
+diff --git a/src/lxc/confile.c b/src/lxc/confile.c
+index 5045741bb..875418792 100644
+--- a/src/lxc/confile.c
++++ b/src/lxc/confile.c
+@@ -80,6 +80,8 @@ lxc_config_define(console_rotate);
+ lxc_config_define(console_size);
+ lxc_config_define(unsupported_key);
+ lxc_config_define(environment);
++lxc_config_define(environment_runtime);
++lxc_config_define(environment_hooks);
+ lxc_config_define(ephemeral);
+ lxc_config_define(execute_cmd);
+ lxc_config_define(group);
+@@ -211,6 +213,8 @@ static struct lxc_config_t config_jump_table[] = {
+ { "lxc.console.rotate", true, set_config_console_rotate, get_config_console_rotate, clr_config_console_rotate, },
+ { "lxc.console.size", true, set_config_console_size, get_config_console_size, clr_config_console_size, },
+ { "lxc.sched.core", true, set_config_sched_core, get_config_sched_core, clr_config_sched_core, },
++ { "lxc.environment.runtime", true, set_config_environment_runtime, get_config_environment_runtime, clr_config_environment_runtime },
++ { "lxc.environment.hooks", true, set_config_environment_hooks, get_config_environment_hooks, clr_config_environment_hooks },
+ { "lxc.environment", true, set_config_environment, get_config_environment, clr_config_environment, },
+ { "lxc.ephemeral", true, set_config_ephemeral, get_config_ephemeral, clr_config_ephemeral, },
+ { "lxc.execute.cmd", true, set_config_execute_cmd, get_config_execute_cmd, clr_config_execute_cmd, },
+@@ -1574,15 +1578,15 @@ static int set_config_group(const char *key, const char *value,
+ return 0;
+ }
+
+-static int set_config_environment(const char *key, const char *value,
+- struct lxc_conf *lxc_conf, void *data)
++static int set_config_environment_impl(const char *value,
++ struct list_head *environment)
+ {
+ __do_free char *dup = NULL, *val = NULL;
+ __do_free struct environment_entry *new_env = NULL;
+ char *env_val;
+
+ if (lxc_config_value_empty(value))
+- return lxc_clear_environment(lxc_conf);
++ return lxc_clear_environment(environment);
+
+ new_env = zalloc(sizeof(struct environment_entry));
+ if (!new_env)
+@@ -1609,12 +1613,30 @@ static int set_config_environment(const char *key, const char *value,
+ new_env->key = move_ptr(dup);
+ new_env->val = move_ptr(val);
+
+- list_add_tail(&new_env->head, &lxc_conf->environment);
++ list_add_tail(&new_env->head, environment);
+ move_ptr(new_env);
+
+ return 0;
+ }
+
++static int set_config_environment(const char *key, const char *value,
++ struct lxc_conf *lxc_conf, void *data)
++{
++ return set_config_environment_impl(value, &lxc_conf->environment);
++}
++
++static int set_config_environment_runtime(const char *key, const char* value,
++ struct lxc_conf *lxc_conf, void *data)
++{
++ return set_config_environment_impl(value, &lxc_conf->environment_runtime);
++}
++
++static int set_config_environment_hooks(const char *key, const char* value,
++ struct lxc_conf *lxc_conf, void *data)
++{
++ return set_config_environment_impl(value, &lxc_conf->environment_hooks);
++}
++
+ static int set_config_tty_max(const char *key, const char *value,
+ struct lxc_conf *lxc_conf, void *data)
+ {
+@@ -4473,8 +4495,8 @@ static int get_config_group(const char *key, char *retv, int inlen,
+ return fulllen;
+ }
+
+-static int get_config_environment(const char *key, char *retv, int inlen,
+- struct lxc_conf *c, void *data)
++static int get_config_environment_impl(char *retv, int inlen,
++ struct list_head *environment)
+ {
+ int len, fulllen = 0;
+ struct environment_entry *env;
+@@ -4484,13 +4506,32 @@ static int get_config_environment(const char *key, char *retv, int inlen,
+ else
+ memset(retv, 0, inlen);
+
+- list_for_each_entry(env, &c->environment, head) {
++ list_for_each_entry(env, environment, head) {
+ strprint(retv, inlen, "%s=%s\n", env->key, env->val);
+ }
+
+ return fulllen;
+ }
+
++static int get_config_environment(const char *key, char *retv, int inlen,
++ struct lxc_conf *c, void *data)
++{
++ return get_config_environment_impl(retv, inlen, &c->environment);
++}
++
++static int get_config_environment_runtime(const char *key, char *retv,
++ int inlen, struct lxc_conf *c,
++ void *data)
++{
++ return get_config_environment_impl(retv, inlen, &c->environment_runtime);
++}
++
++static int get_config_environment_hooks(const char *key, char *retv, int inlen,
++ struct lxc_conf *c, void *data)
++{
++ return get_config_environment_impl(retv, inlen, &c->environment_hooks);
++}
++
+ static int get_config_execute_cmd(const char *key, char *retv, int inlen,
+ struct lxc_conf *c, void *data)
+ {
+@@ -5211,7 +5252,19 @@ static inline int clr_config_group(const char *key, struct lxc_conf *c,
+ static inline int clr_config_environment(const char *key, struct lxc_conf *c,
+ void *data)
+ {
+- return lxc_clear_environment(c);
++ return lxc_clear_environment(&c->environment);
++}
++
++static inline int clr_config_environment_runtime(const char *key,
++ struct lxc_conf *c, void *data)
++{
++ return lxc_clear_environment(&c->environment_runtime);
++}
++
++static inline int clr_config_environment_hooks(const char *key,
++ struct lxc_conf *c, void *data)
++{
++ return lxc_clear_environment(&c->environment_hooks);
+ }
+
+ static inline int clr_config_execute_cmd(const char *key, struct lxc_conf *c,
+@@ -6607,6 +6660,9 @@ int lxc_list_subkeys(struct lxc_conf *conf, const char *key, char *retv,
+ } else if (strequal(key, "lxc.console")) {
+ strprint(retv, inlen, "logfile\n");
+ strprint(retv, inlen, "path\n");
++ } else if (strequal(key, "lxc.environment")) {
++ strprint(retv, inlen, "runtime\n");
++ strprint(retv, inlen, "hooks\n");
+ } else if (strequal(key, "lxc.seccomp")) {
+ strprint(retv, inlen, "profile\n");
+ } else if (strequal(key, "lxc.signal")) {
+diff --git a/src/lxc/start.c b/src/lxc/start.c
+index 3160459b4..fc6ca147a 100644
+--- a/src/lxc/start.c
++++ b/src/lxc/start.c
+@@ -1260,10 +1260,14 @@ static int do_start(void *data)
+ * to allow them to be used by the various hooks, such as the start
+ * hook below.
+ */
+- ret = lxc_set_environment(handler->conf);
++ ret = lxc_set_environment(&handler->conf->environment);
+ if (ret < 0)
+ goto out_warn_father;
+
++ ret = lxc_set_environment(&handler->conf->environment_hooks);
++ if (ret < 0)
++ goto out_warn_father;
++
+ if (!lxc_sync_wait_parent(handler, START_SYNC_POST_CONFIGURE))
+ goto out_warn_father;
+
+@@ -1361,10 +1365,14 @@ static int do_start(void *data)
+ if (ret < 0)
+ SYSERROR("Failed to clear environment.");
+
+- ret = lxc_set_environment(handler->conf);
++ ret = lxc_set_environment(&handler->conf->environment);
+ if (ret < 0)
+ goto out_warn_father;
+
++ ret = lxc_set_environment(&handler->conf->environment_runtime);
++ if (ret < 0)
++ goto out_warn_father;
++
+ ret = putenv("container=lxc");
+ if (ret < 0) {
+ SYSERROR("Failed to set environment variable: container=lxc");
+--
+2.47.2
+
diff --git a/debian/patches/series b/debian/patches/series
index 5f3f0b6..b1353eb 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -2,3 +2,4 @@ apparmor/0001-apparmor-allow-lxc-start-to-create-user-namespaces.patch
apparmor/0002-apparmor-use-abi-directive-in-apparmor-profiles.patch
pve/0001-PVE-Config-deny-rw-mounting-of-sys-and-proc.patch
pve/0002-PVE-Config-attach-always-use-getent.patch
+pve/0003-PVE-conf-split-lxc.environment-into-runtime-and-hooks.patch
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH container v4 06/15] config: add `lxc.environment.runtime`/`hooks`
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (4 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH lxc v4 05/15] lxc: conf: split `lxc.environment` into `runtime` and `hooks` Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 07/15] add support for OCI images as container templates Filip Schauer
` (8 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Add `lxc.environment.runtime` and `lxc.environment.hooks` config keys.
These allow setting environment variables separately for the container
init process and for LXC hooks.
This will be needed by containers created from OCI images with custom
environment variables which could interfere with hook execution.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v4
src/PVE/LXC/Config.pm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 5d3749e..1038fa7 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -736,6 +736,8 @@ my $valid_lxc_conf_keys = {
'lxc.start.order' => 1,
'lxc.group' => 1,
'lxc.environment' => 1,
+ 'lxc.environment.runtime' => 1,
+ 'lxc.environment.hooks' => 1,
# All these are namespaced via CLONE_NEWIPC (see namespaces(7)).
'lxc.sysctl.fs.mqueue' => 1,
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH container v4 07/15] add support for OCI images as container templates
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (5 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 06/15] config: add `lxc.environment.runtime`/`hooks` Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 08/15] config: add entrypoint parameter Filip Schauer
` (7 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
This aims to add basic support for the Open Container Initiative image
format according to the specification. [0]
[0] https://github.com/opencontainers/image-spec/blob/main/spec.md
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This patch depends on changes made to proxmox-perl-rs in patch 04/15.
Meaning that proxmox-perl-rs needs to be bumped and a dependency & build
dependency to libpve-rs-perl needs to be added to debian/control.
Changed since v3:
* correctly handle case where $archive is '-'
* replace unnecessary regex comparisons with `eq`
* pass environment variables to container via `lxc.container.runtime`
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* check whether archive is an OCI image before trying to parse it as one
Changed since v1:
* fix entrypoint command missing Cmd
* set lxc.signal.halt according to StopSignal (Fixes container shutdown)
src/PVE/API2/LXC.pm | 94 ++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 84 insertions(+), 10 deletions(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 1f89c87..bc87bc3 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -19,9 +19,11 @@ use PVE::Storage;
use PVE::RESTHandler;
use PVE::RPCEnvironment;
use PVE::ReplicationConfig;
+use PVE::RS::OCI;
use PVE::LXC;
use PVE::LXC::Create;
use PVE::LXC::Migrate;
+use PVE::LXC::Namespaces;
use PVE::GuestHelpers;
use PVE::VZDump::Plugin;
use PVE::API2::LXC::Config;
@@ -529,19 +531,91 @@ __PACKAGE__->register_method({
eval {
my $rootdir = PVE::LXC::mount_all($vmid, $storage_cfg, $conf, 1);
+ my $archivepath = '-';
+ $archivepath = PVE::Storage::abs_filesystem_path($storage_cfg, $archive)
+ if ($archive ne '-');
$bwlimit = PVE::Storage::get_bandwidth_limit(
'restore', [keys %used_storages], $bwlimit,
);
- print "restoring '$archive' now..\n"
- if $restore && $archive ne '-';
- PVE::LXC::Create::restore_archive(
- $storage_cfg,
- $archive,
- $rootdir,
- $conf,
- $ignore_unpack_errors,
- $bwlimit,
- );
+ my $is_oci = 0;
+
+ if ($restore && $archive ne '-') {
+ print "restoring '$archive' now..\n";
+ } elsif ($archivepath =~ /\.tar$/) {
+ # Check whether archive is an OCI image
+ my ($has_oci_layout, $has_index_json, $has_blobs) = (0, 0, 0);
+ PVE::Tools::run_command(
+ ['tar', '-tf', $archivepath],
+ outfunc => sub {
+ my $line = shift;
+ $has_oci_layout = 1 if $line eq 'oci-layout';
+ $has_index_json = 1 if $line eq 'index.json';
+ $has_blobs = 1 if $line =~ /^blobs\//m;
+ },
+ );
+
+ $is_oci = 1 if $has_oci_layout && $has_index_json && $has_blobs;
+ }
+
+ if ($is_oci) {
+ # Extract the OCI image
+ my ($id_map, undef, undef) = PVE::LXC::parse_id_maps($conf);
+ my $oci_config = PVE::LXC::Namespaces::run_in_userns(
+ sub {
+ PVE::RS::OCI::parse_and_extract_image($archivepath, $rootdir);
+ },
+ $id_map,
+ );
+
+ # Set the entrypoint and arguments if specified by the OCI image
+ my @init_cmd = ();
+ push(@init_cmd, @{ $oci_config->{Entrypoint} })
+ if $oci_config->{Entrypoint};
+ push(@init_cmd, @{ $oci_config->{Cmd} }) if $oci_config->{Cmd};
+ if (@init_cmd) {
+ my $init_cmd_str = shift(@init_cmd);
+ if (@init_cmd) {
+ $init_cmd_str .= ' ';
+ $init_cmd_str .= join(
+ ' ',
+ map {
+ my $s = $_;
+ $s =~ s/"/\\"/g;
+ qq{"$_"}
+ } @init_cmd,
+ );
+ }
+ if ($init_cmd_str ne '/sbin/init') {
+ push @{ $conf->{lxc} }, ['lxc.init.cmd', $init_cmd_str];
+
+ # An entrypoint other than /sbin/init breaks the tty console mode.
+ # This is fixed by setting cmode: console
+ $conf->{cmode} = 'console';
+ }
+ }
+
+ push @{ $conf->{lxc} }, ['lxc.init.cwd', $oci_config->{WorkingDir}]
+ if ($oci_config->{WorkingDir});
+
+ if (my $envs = $oci_config->{Env}) {
+ for my $env (@{$envs}) {
+ push @{ $conf->{lxc} }, ['lxc.environment.runtime', $env];
+ }
+ }
+
+ my $stop_signal = $oci_config->{StopSignal} // "SIGTERM";
+ push @{ $conf->{lxc} }, ['lxc.signal.halt', $stop_signal];
+ } else {
+ # Not an OCI image, so restore it as an LXC image instead
+ PVE::LXC::Create::restore_archive(
+ $storage_cfg,
+ $archive,
+ $rootdir,
+ $conf,
+ $ignore_unpack_errors,
+ $bwlimit,
+ );
+ }
if ($restore) {
print "merging backed-up and given configuration..\n";
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH container v4 08/15] config: add entrypoint parameter
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (6 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 07/15] add support for OCI images as container templates Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 09/15] configure static IP in LXC config for custom entrypoint Filip Schauer
` (6 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
src/PVE/API2/LXC.pm | 2 +-
src/PVE/LXC.pm | 2 ++
src/PVE/LXC/Config.pm | 12 ++++++++++++
3 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index bc87bc3..55f5b7a 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -586,7 +586,7 @@ __PACKAGE__->register_method({
);
}
if ($init_cmd_str ne '/sbin/init') {
- push @{ $conf->{lxc} }, ['lxc.init.cmd', $init_cmd_str];
+ $conf->{entrypoint} = $init_cmd_str;
# An entrypoint other than /sbin/init breaks the tty console mode.
# This is fixed by setting cmode: console
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index a445a85..5eaa57c 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -862,6 +862,8 @@ sub update_lxc_config {
$raw .= "lxc.rootfs.path = $dir/rootfs\n";
+ $raw .= "lxc.init.cmd = $conf->{entrypoint}\n" if defined($conf->{entrypoint});
+
foreach my $k (sort keys %$conf) {
next if $k !~ m/^net(\d+)$/;
my $ind = $1;
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 1038fa7..56cb01c 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -638,6 +638,12 @@ my $confdesc = {
enum => ['shell', 'console', 'tty'],
default => 'tty',
},
+ entrypoint => {
+ optional => 1,
+ type => 'string',
+ description => "Absolute path from container rootfs to the binary to use as init.",
+ default => '/sbin/init',
+ },
protection => {
optional => 1,
type => 'boolean',
@@ -1861,6 +1867,12 @@ sub get_cmode {
return $conf->{cmode} // $confdesc->{cmode}->{default};
}
+sub get_entrypoint {
+ my ($class, $conf) = @_;
+
+ return $conf->{entrypoint} // $confdesc->{entrypoint}->{default};
+}
+
sub valid_volume_keys {
my ($class, $reverse) = @_;
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH container v4 09/15] configure static IP in LXC config for custom entrypoint
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (7 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 08/15] config: add entrypoint parameter Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 10/15] setup: debian: create /etc/network path if missing Filip Schauer
` (5 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
When a container uses the default `/sbin/init` entrypoint, network
interface configuration is usually managed by processes within the
container. However, containers with a different entrypoint might not
have any internal network management process. Consequently, IP addresses
might not be assigned.
This change ensures that a static IP address is explicitly set in the
LXC config for the container.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* add an "ipmanagehost" property to pct.conf to control whether network
interface IP configuration should be handled by the host.
src/PVE/API2/LXC.pm | 4 ++++
src/PVE/LXC.pm | 15 +++++++++++++++
src/PVE/LXC/Config.pm | 14 ++++++++++++++
3 files changed, 33 insertions(+)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 55f5b7a..0c0755b 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -591,6 +591,10 @@ __PACKAGE__->register_method({
# An entrypoint other than /sbin/init breaks the tty console mode.
# This is fixed by setting cmode: console
$conf->{cmode} = 'console';
+
+ # Manage the IP configuration for the container. A container with a
+ # custom entrypoint likely lacks internal network management.
+ $conf->{ipmanagehost} = 1;
}
}
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 5eaa57c..6fdef79 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -886,6 +886,21 @@ sub update_lxc_config {
if ($lxc_major >= 4) {
$raw .= "lxc.net.$ind.script.up = /usr/share/lxc/lxcnetaddbr\n";
}
+
+ if ((!defined($d->{link_down}) || $d->{link_down} != 1) && $conf->{ipmanagehost}) {
+ if (defined($d->{ip})) {
+ die "$k: DHCP is not supported with a custom entrypoint\n" if $d->{ip} eq 'dhcp';
+ $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n" if $d->{ip} ne 'manual';
+ }
+ $raw .= "lxc.net.$ind.ipv4.gateway = $d->{gw}\n" if defined($d->{gw});
+ if (defined($d->{ip6})) {
+ die "$k: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
+ if $d->{ip6} =~ /^(auto|dhcp)$/;
+ $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} ne 'manual';
+ }
+ $raw .= "lxc.net.$ind.ipv6.gateway = $d->{gw6}\n" if defined($d->{gw6});
+ $raw .= "lxc.net.$ind.flags = up\n";
+ }
}
my $had_cpuset = 0;
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 56cb01c..afa2fcf 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -594,6 +594,12 @@ my $confdesc = {
. " This is saved as comment inside the configuration file.",
maxLength => 1024 * 8,
},
+ ipmanagehost => {
+ type => 'boolean',
+ description =>
+ "Whether this interface's IP configuration should be managed by the host.",
+ optional => 1,
+ },
searchdomain => {
optional => 1,
type => 'string',
@@ -1288,6 +1294,14 @@ sub update_pct_config {
die "$opt: MTU size '$mtu' is bigger than bridge MTU '$bridge_mtu'\n"
if ($mtu > $bridge_mtu);
}
+
+ if ((!defined($res->{link_down}) || $res->{link_down} != 1) && $conf->{ipmanagehost}) {
+ die "$opt: DHCP is not supported with a custom entrypoint\n"
+ if defined($res->{ip}) && $res->{ip} eq 'dhcp';
+
+ die "$opt: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
+ if defined($res->{ip6}) && $res->{ip6} =~ /^(auto|dhcp)$/;
+ }
} elsif ($opt =~ m/^dev(\d+)$/) {
my $device = $class->parse_device($value);
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH container v4 10/15] setup: debian: create /etc/network path if missing
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (8 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 09/15] configure static IP in LXC config for custom entrypoint Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 11/15] setup: recursively mkdir /etc/systemd/{network, system-preset} Filip Schauer
` (4 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
This prevents an error during Debian container setup when the
/etc/network directory is missing. This fixes container creation from
Debian based OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/LXC/Setup/Debian.pm | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/PVE/LXC/Setup/Debian.pm b/src/PVE/LXC/Setup/Debian.pm
index 3924e8c..c37ca86 100644
--- a/src/PVE/LXC/Setup/Debian.pm
+++ b/src/PVE/LXC/Setup/Debian.pm
@@ -431,6 +431,7 @@ sub setup_network {
"auto lo\niface lo inet loopback\n" . "iface lo inet6 loopback\n\n" . $interfaces;
}
+ $self->ct_make_path('/etc/network');
$self->ct_file_set_contents($filename, $interfaces);
}
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH container v4 11/15] setup: recursively mkdir /etc/systemd/{network, system-preset}
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (9 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 10/15] setup: debian: create /etc/network path if missing Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 12/15] implement host-managed DHCP for containers with `ipmanagehost` Filip Schauer
` (3 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Ensure that both /etc/systemd/network and /etc/systemd/system-preset
exist before writing files into them. This fixes container creation from
the docker.io/fedora & docker.io/ubuntu OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
Introduced in v2
src/PVE/LXC/Setup/Base.pm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/PVE/LXC/Setup/Base.pm b/src/PVE/LXC/Setup/Base.pm
index 806265b..1c35731 100644
--- a/src/PVE/LXC/Setup/Base.pm
+++ b/src/PVE/LXC/Setup/Base.pm
@@ -313,6 +313,7 @@ DATA
$data .= "IPv6AcceptRA = $accept_ra\n";
$data .= $routes if $routes;
+ $self->ct_make_path('/etc/systemd/network');
$self->ct_file_set_contents($filename, $data);
}
}
@@ -352,7 +353,7 @@ sub setup_systemd_preset {
}
}
- $self->ct_mkdir('/etc/systemd/system-preset', 0755);
+ $self->ct_make_path('/etc/systemd/system-preset');
$self->ct_file_set_contents(
'/etc/systemd/system-preset/00-pve.preset', $preset_data,
);
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH container v4 12/15] implement host-managed DHCP for containers with `ipmanagehost`
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (10 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 11/15] setup: recursively mkdir /etc/systemd/{network, system-preset} Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH storage v4 13/15] allow .tar container templates Filip Schauer
` (2 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Use `dhclient` on the host to manage DHCP for container network
interfaces when `ipmanagehost` is enabled in the container config.
This is useful for containers that lack an internal network management
stack, like many containers based on OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changes since v3:
* keep lxc pidfd around to ensure the existence of $rootdir until
dhclients are started
* update commit message to reflect changes from v3
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* add an "ipmanagehost" property to pct.conf to control whether network
interface IP configuration should be handled by the host.
* manage_dhclient: add a FIXME comment regarding the AppArmor profile:
"use a profile that confines writes to /var/lib/lxc/$vmid and rootfs"
* kill_dhclients: untaint pid from pidfile
* fix manage_dhclient called with 'stop' instead of 'start' for IPv6
when container is started
src/PVE/LXC.pm | 103 +++++++++++++++++++++++++++++++++++++-----
src/PVE/LXC/Config.pm | 17 ++++---
2 files changed, 101 insertions(+), 19 deletions(-)
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 6fdef79..69bbd24 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -888,15 +888,12 @@ sub update_lxc_config {
}
if ((!defined($d->{link_down}) || $d->{link_down} != 1) && $conf->{ipmanagehost}) {
- if (defined($d->{ip})) {
- die "$k: DHCP is not supported with a custom entrypoint\n" if $d->{ip} eq 'dhcp';
- $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n" if $d->{ip} ne 'manual';
- }
+ $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n"
+ if defined($d->{ip}) && $d->{ip} !~ /^(dhcp|manual)$/;
$raw .= "lxc.net.$ind.ipv4.gateway = $d->{gw}\n" if defined($d->{gw});
if (defined($d->{ip6})) {
- die "$k: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
- if $d->{ip6} =~ /^(auto|dhcp)$/;
- $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} ne 'manual';
+ die "$k: SLAAC is not supported with ipmanagehost\n" if $d->{ip6} eq 'auto';
+ $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} !~ /^(dhcp|manual)$/;
}
$raw .= "lxc.net.$ind.ipv6.gateway = $d->{gw6}\n" if defined($d->{gw6});
$raw .= "lxc.net.$ind.flags = up\n";
@@ -1079,6 +1076,8 @@ sub vm_stop_cleanup {
PVE::Storage::deactivate_volumes($storage_cfg, $vollist);
};
warn $@ if $@; # avoid errors - just warn
+
+ kill_dhclients($vmid, '*') if $conf->{ipmanagehost};
}
sub net_tap_plug : prototype($$) {
@@ -1310,6 +1309,52 @@ sub get_interfaces {
return $res;
}
+sub manage_dhclient {
+ my ($action, $vmid, $ipversion, $eth, $rootdir) = @_;
+
+ File::Path::make_path("/var/lib/lxc/$vmid/hook") if $action eq 'start';
+ my $pidfile = "/var/lib/lxc/$vmid/hook/dhclient$ipversion-$eth.pid";
+ my $leasefile = "/var/lib/lxc/$vmid/hook/dhclient$ipversion-$eth.leases";
+ my $scriptfile = '/usr/share/lxc/hooks/dhclient-script';
+ PVE::Tools::run_command([
+ 'lxc-attach',
+ '-n',
+ $vmid,
+ '-s',
+ 'NETWORK|UTSNAME',
+ '--',
+ 'aa-exec',
+ '-p',
+ 'unconfined', # FIXME: use a profile that confines writes to /var/lib/lxc/$vmid and rootfs
+ '/sbin/dhclient',
+ $action eq 'start' ? '-1' : '-r',
+ "-$ipversion",
+ '-pf',
+ $pidfile,
+ '-lf',
+ $leasefile,
+ '-e',
+ "ROOTFS=$rootdir",
+ '-sf',
+ $scriptfile,
+ $eth,
+ ]);
+}
+
+sub kill_dhclients {
+ my ($vmid, $eth) = @_;
+
+ foreach my $pidfile (glob("/var/lib/lxc/$vmid/hook/dhclient*-$eth.pid")) {
+ my $pid = eval { file_get_contents($pidfile) };
+ if (!$@) {
+ chomp $pid;
+ next if $pid !~ /^(\d+)$/;
+ kill 9, $1;
+ unlink($pidfile);
+ }
+ }
+}
+
sub update_ipconfig {
my ($vmid, $conf, $opt, $eth, $newnet, $rootdir) = @_;
@@ -1346,11 +1391,21 @@ sub update_ipconfig {
# step 1: add new IP, if this fails we cancel
my $is_real_ip = ($newip && $newip !~ /^(?:auto|dhcp|manual)$/);
- if ($change_ip && $is_real_ip) {
- eval { &$ipcmd($family_opt, 'addr', 'add', $newip, 'dev', $eth); };
- if (my $err = $@) {
- warn $err;
- return;
+ if ($change_ip) {
+ if ($conf->{ipmanagehost}) {
+ if ($newip && $newip eq 'dhcp') {
+ manage_dhclient('start', $vmid, $ipversion, $eth, $rootdir);
+ } elsif ($oldip && $oldip eq 'dhcp') {
+ manage_dhclient('stop', $vmid, $ipversion, $eth, $rootdir);
+ }
+ }
+
+ if ($is_real_ip) {
+ eval { &$ipcmd($family_opt, 'addr', 'add', $newip, 'dev', $eth); };
+ if (my $err = $@) {
+ warn $err;
+ return;
+ }
}
}
@@ -3001,6 +3056,30 @@ sub vm_start {
}
PVE::GuestHelpers::exec_hookscript($conf, $vmid, 'post-start');
+ if ($conf->{ipmanagehost}) {
+ my @dhcpv4_interfaces = ();
+ my @dhcpv6_interfaces = ();
+ foreach my $k (sort keys %$conf) {
+ next if $k !~ m/^net(\d+)$/;
+ my $d = PVE::LXC::Config->parse_lxc_network($conf->{$k});
+ push @dhcpv4_interfaces, $d->{name} if $d->{ip} && $d->{ip} eq 'dhcp';
+ push @dhcpv6_interfaces, $d->{name} if $d->{ip6} && $d->{ip6} eq 'dhcp';
+ }
+
+ my ($pid, $pidfd) = PVE::LXC::open_lxc_pid($vmid);
+ my $rootdir = "/proc/$pid/root";
+
+ foreach my $eth (@dhcpv4_interfaces) {
+ eval { manage_dhclient('start', $vmid, 4, $eth, $rootdir) };
+ PVE::RESTEnvironment::log_warn("DHCP failed - $@") if $@;
+ }
+
+ foreach my $eth (@dhcpv6_interfaces) {
+ eval { manage_dhclient('start', $vmid, 6, $eth, $rootdir) };
+ PVE::RESTEnvironment::log_warn("DHCP failed - $@") if $@;
+ }
+ }
+
return;
}
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index afa2fcf..3797ca5 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -1295,12 +1295,13 @@ sub update_pct_config {
if ($mtu > $bridge_mtu);
}
- if ((!defined($res->{link_down}) || $res->{link_down} != 1) && $conf->{ipmanagehost}) {
- die "$opt: DHCP is not supported with a custom entrypoint\n"
- if defined($res->{ip}) && $res->{ip} eq 'dhcp';
-
- die "$opt: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
- if defined($res->{ip6}) && $res->{ip6} =~ /^(auto|dhcp)$/;
+ if (
+ (!defined($res->{link_down}) || $res->{link_down} != 1)
+ && $conf->{ipmanagehost}
+ && defined($res->{ip6})
+ && $res->{ip6} eq 'auto'
+ ) {
+ die "$opt: SLAAC is not supported with ipmanagehost\n";
}
} elsif ($opt =~ m/^dev(\d+)$/) {
my $device = $class->parse_device($value);
@@ -1585,9 +1586,11 @@ sub vmconfig_hotplug_pending {
$cgroup->change_cpu_shares(undef);
} elsif ($opt =~ m/^net(\d)$/) {
my $netid = $1;
+ my $net = parse_lxc_network($conf->{$opt});
+ PVE::LXC::kill_dhclients($vmid, $net->{name}) if $conf->{ipmanagehost};
+
PVE::Network::veth_delete("veth${vmid}i$netid");
if ($have_sdn) {
- my $net = PVE::LXC::Config->parse_lxc_network($conf->{$opt});
print "delete ips from $opt\n";
eval {
PVE::Network::SDN::Vnets::del_ips_from_mac(
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH storage v4 13/15] allow .tar container templates
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (11 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH container v4 12/15] implement host-managed DHCP for containers with `ipmanagehost` Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH manager v4 14/15] ui: storage upload: accept *.tar files as vztmpl Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH docs v4 15/15] ct: add OCI image docs Filip Schauer
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
This is needed for OCI container images bundled as tar files, as
generated by `docker save`. OCI images do not need additional
compression, since the content is usually compressed already.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* Modify VZTMPL_EXT_RE_1 regex to put "tar" into capture group when
matching on a .tar file.
src/PVE/Storage.pm | 2 +-
src/PVE/Storage/Plugin.pm | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 1dde2b7..0bab945 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -115,7 +115,7 @@ PVE::Storage::Plugin->init();
our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
-our $VZTMPL_EXT_RE_1 = qr/\.tar\.(gz|xz|zst|bz2)/i;
+our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 2291d72..ef05a61 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1618,7 +1618,7 @@ my $get_subdir_files = sub {
} elsif ($tt eq 'vztmpl') {
next if $fn !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
- $info = { volid => "$sid:vztmpl/$1", format => "t$2" };
+ $info = { volid => "$sid:vztmpl/$1", format => $2 eq 'tar' ? $2 : "t$2" };
} elsif ($tt eq 'backup') {
next if $fn !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH manager v4 14/15] ui: storage upload: accept *.tar files as vztmpl
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (12 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH storage v4 13/15] allow .tar container templates Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
2025-09-08 15:02 ` [pve-devel] [PATCH docs v4 15/15] ct: add OCI image docs Filip Schauer
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This depends on the change made to pve-storage in patch 13/15.
It might make sense to bump pve-storage and with it bump the dependency
to libpve-storage-perl in debian/control.
Changed since v2:
* rebase onto newest master (84b22751f211) and re-format
Introduced in v2
www/manager6/window/UploadToStorage.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
index cc53596d..df6a9178 100644
--- a/www/manager6/window/UploadToStorage.js
+++ b/www/manager6/window/UploadToStorage.js
@@ -11,7 +11,7 @@ Ext.define('PVE.window.UploadToStorage', {
acceptedExtensions: {
import: ['.ova', '.qcow2', '.raw', '.vmdk'],
iso: ['.img', '.iso'],
- vztmpl: ['.tar.gz', '.tar.xz', '.tar.zst'],
+ vztmpl: ['.tar', '.tar.gz', '.tar.xz', '.tar.zst'],
},
// accepted for file selection, will be renamed to real extension
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread* [pve-devel] [PATCH docs v4 15/15] ct: add OCI image docs
2025-09-08 15:02 [pve-devel] [PATCH container/docs/lxc/manager/proxmox{, -perl-rs}/storage v4 00/15] support OCI images as container templates Filip Schauer
` (13 preceding siblings ...)
2025-09-08 15:02 ` [pve-devel] [PATCH manager v4 14/15] ui: storage upload: accept *.tar files as vztmpl Filip Schauer
@ 2025-09-08 15:02 ` Filip Schauer
14 siblings, 0 replies; 16+ messages in thread
From: Filip Schauer @ 2025-09-08 15:02 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v3
pct.adoc | 72 +++++++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 64 insertions(+), 8 deletions(-)
diff --git a/pct.adoc b/pct.adoc
index d6146eb..08dcf33 100644
--- a/pct.adoc
+++ b/pct.adoc
@@ -54,15 +54,22 @@ the cluster setup, and they can use the same network and storage resources as
virtual machines. You can also use the {pve} firewall, or manage containers
using the HA framework.
-Our primary goal is to offer an environment that provides the benefits of using a
-VM, but without the additional overhead. This means that Proxmox Containers can
-be categorized as ``System Containers'', rather than ``Application Containers''.
+Our primary goal has traditionally been to offer an environment that provides
+the benefits of using a VM, but without the additional overhead. This means that
+Proxmox Containers have been primarily categorized as ``System Containers''.
-NOTE: If you want to run application containers, for example, 'Docker' images, it
-is recommended that you run them inside a Proxmox QEMU VM. This will give you
-all the advantages of application containerization, while also providing the
-benefits that VMs offer, such as strong isolation from the host and the ability
-to live-migrate, which otherwise isn't possible with containers.
+With the introduction of OCI (**O**pen **C**ontainer **I**nitiative) image support,
+Proxmox VE now also integrates ``Application Containers''. When creating a
+container from an OCI image, the image is automatically converted to the
+LXC stack that Proxmox VE uses.
+
+This approach allows users to benefit from a wide ecosystem of pre-packaged
+applications while retaining the robust management features of Proxmox VE.
+
+While running lightweight ``Application Containers'' directly offers significant
+advantages over a full VM, for use cases demanding maximum isolation and
+the ability to live-migrate, nesting containers inside a Proxmox QEMU VM remains
+a recommended practice.
Technology Overview
@@ -256,6 +263,12 @@ Container Images
Container images, sometimes also referred to as ``templates'' or
``appliances'', are `tar` archives which contain everything to run a container.
+Proxmox VE can utilize two main types of images: *System Container Templates*
+for creating full virtual environments, and *Application Container Images* based
+on the OCI standard for running specific applications.
+
+System Container Templates
+~~~~~~~~~~~~~~~~~~~~~~~~~~
{pve} itself provides a variety of basic templates for the
xref:pct_supported_distributions[most common Linux distributions]. They can be
@@ -336,6 +349,49 @@ delete that image later with:
# pveam remove local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz
----
+Open Container Initiative (OCI) Images (Experimental)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Proxmox VE can also use OCI images to create containers. This makes it easy to
+run pre-packaged applications. A container created from an OCI image still uses
+the existing LXC framework.
+
+Obtaining OCI Images
+~~~~~~~~~~~~~~~~~~~~
+
+An OCI image can be pulled from a registry and saved as a `tar` archive. You can
+use tools like `docker`, `podman`, or `skopeo` for this. For example, to obtain
+the Apache HTTP Server (httpd) image from Docker Hub:
+
+Using Docker:
+
+----
+# docker pull httpd
+# docker save httpd > httpd.tar
+----
+
+Using Podman:
+
+When using Podman, the archive format must be explicitly specified as
+`oci-archive`.
+
+----
+# podman pull httpd
+# podman save --format=oci-archive httpd > httpd.tar
+----
+
+Using skopeo:
+
+----
+# skopeo copy docker://httpd:latest oci-archive:httpd.tar:latest
+----
+
+The resulting `httpd.tar` archive is an OCI image. It can be uploaded to a
+Proxmox VE storage that supports container templates via the *Upload* button in
+the storage content view of the web interface.
+
+Once the template is on a storage, you can create the container with
+`pct create` or use the wizard in the web interface.
[[pct_settings]]
Container Settings
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 16+ messages in thread