* [pve-devel] [PATCH proxmox v5 01/17] io: introduce RangeReader for bounded reads
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 02/17] add proxmox-oci crate Filip Schauer
` (15 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Introduce a reader that exposes a sub-range of an underlying reader.
This will be used for reading individual files out of a tar archive when
parsing an OCI image.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v3:
* add a commit message
* add rustdoc comments
* add unit tests
Introduced in v3
proxmox-io/src/lib.rs | 3 +
proxmox-io/src/range_reader.rs | 175 +++++++++++++++++++++++++++++++++
2 files changed, 178 insertions(+)
create mode 100644 proxmox-io/src/range_reader.rs
diff --git a/proxmox-io/src/lib.rs b/proxmox-io/src/lib.rs
index 1be005ff..a05b9232 100644
--- a/proxmox-io/src/lib.rs
+++ b/proxmox-io/src/lib.rs
@@ -6,6 +6,9 @@
#![deny(unsafe_op_in_unsafe_fn)]
#![cfg_attr(docsrs, feature(doc_cfg, doc_auto_cfg))]
+mod range_reader;
+pub use range_reader::RangeReader;
+
mod read;
pub use read::ReadExt;
diff --git a/proxmox-io/src/range_reader.rs b/proxmox-io/src/range_reader.rs
new file mode 100644
index 00000000..3f4c54fe
--- /dev/null
+++ b/proxmox-io/src/range_reader.rs
@@ -0,0 +1,175 @@
+use std::io::{Read, Seek, SeekFrom};
+use std::ops::Range;
+
+/// A reader that only exposes a sub-range of an underlying `Read + Seek`.
+///
+/// # Examples
+///
+/// ```
+/// # use proxmox_io::RangeReader;
+/// # use std::io::{Cursor, Read, Seek, SeekFrom};
+/// # fn func() -> Result<(), std::io::Error> {
+/// let reader = Cursor::new("Lorem ipsum dolor sit amet");
+///
+/// let mut range_reader = RangeReader::new(reader, 6..17);
+///
+/// // Read all bytes in the range
+/// let mut buf = Vec::new();
+/// range_reader.read_to_end(&mut buf)?;
+/// assert_eq!(buf, "ipsum dolor".as_bytes());
+///
+/// // Seek back to start of the range and read one byte
+/// range_reader.seek(SeekFrom::Start(0))?;
+/// let mut b = [0u8; 1];
+/// range_reader.read_exact(&mut b)?;
+/// assert_eq!(b, "i".as_bytes());
+///
+/// # Ok(())
+/// # }
+/// # func().unwrap();
+/// ```
+pub struct RangeReader<R: Read + Seek> {
+ /// Underlying reader
+ reader: R,
+
+ /// Range inside `R`
+ range: Range<u64>,
+
+ /// Relative position inside `range`
+ position: u64,
+
+ /// True once the initial seek has been performed
+ has_seeked: bool,
+}
+
+impl<R: Read + Seek> RangeReader<R> {
+ pub fn new(reader: R, range: Range<u64>) -> Self {
+ Self {
+ reader,
+ range,
+ position: 0,
+ has_seeked: false,
+ }
+ }
+
+ pub fn into_inner(self) -> R {
+ self.reader
+ }
+
+ pub fn size(&self) -> usize {
+ (self.range.end - self.range.start) as usize
+ }
+
+ pub fn remaining(&self) -> usize {
+ self.size() - self.position as usize
+ }
+}
+
+impl<R: Read + Seek> Read for RangeReader<R> {
+ fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
+ let max_read = buf.len().min(self.remaining());
+ let limited_buf = &mut buf[..max_read];
+
+ if !self.has_seeked {
+ self.reader
+ .seek(SeekFrom::Start(self.range.start + self.position))?;
+ self.has_seeked = true;
+ }
+
+ let bytes_read = self.reader.read(limited_buf)?;
+ self.position += bytes_read.min(max_read) as u64;
+
+ Ok(bytes_read)
+ }
+}
+
+impl<R: Read + Seek> Seek for RangeReader<R> {
+ fn seek(&mut self, pos: SeekFrom) -> std::io::Result<u64> {
+ self.position = match pos {
+ SeekFrom::Start(position) => position.min(self.size() as u64),
+ SeekFrom::End(offset) => {
+ if offset > self.size() as i64 {
+ return Err(std::io::Error::new(
+ std::io::ErrorKind::InvalidInput,
+ "Tried to seek before the beginning of the file",
+ ));
+ }
+
+ (if offset <= 0 {
+ self.size()
+ } else {
+ self.size() - offset as usize
+ }) as u64
+ }
+ SeekFrom::Current(offset) => {
+ if let Some(position) = self.position.checked_add_signed(offset) {
+ position.min(self.size() as u64)
+ } else {
+ return Err(std::io::Error::new(
+ std::io::ErrorKind::InvalidInput,
+ "Tried to seek before the beginning of the file",
+ ));
+ }
+ }
+ };
+
+ self.reader
+ .seek(SeekFrom::Start(self.range.start + self.position))?;
+ self.has_seeked = true;
+
+ Ok(self.position)
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::RangeReader;
+ use std::io::{Cursor, Read, Seek, SeekFrom};
+
+ #[test]
+ fn test_read_full_range() {
+ let reader = Cursor::new("Hello world!");
+ let mut range_reader = RangeReader::new(reader, 6..11);
+
+ let mut buf = Vec::new();
+ let len = range_reader.read_to_end(&mut buf).unwrap();
+
+ assert_eq!(len, 5);
+ assert_eq!(buf, "world".as_bytes());
+ }
+
+ #[test]
+ fn test_read_partial() {
+ let reader = Cursor::new("Hello world!");
+ let mut range_reader = RangeReader::new(reader, 0..5);
+
+ let mut buf = [0u8; 4];
+ range_reader.read_exact(&mut buf).unwrap();
+
+ assert_eq!(buf, "Hell".as_bytes());
+ }
+
+ #[test]
+ fn test_seek_and_read() {
+ let reader = Cursor::new("Lorem ipsum dolor sit amet");
+ let mut range_reader = RangeReader::new(reader, 6..21);
+
+ assert_eq!(range_reader.seek(SeekFrom::Start(6)).unwrap(), 6);
+ let mut buf = [0u8; 5];
+ range_reader.read_exact(&mut buf).unwrap();
+
+ assert_eq!(buf, "dolor".as_bytes());
+ }
+
+ #[test]
+ fn test_seek_out_of_range() {
+ let reader = Cursor::new("Lorem ipsum dolor sit amet");
+ let mut range_reader = RangeReader::new(reader, 6..21);
+
+ let err = range_reader.seek(SeekFrom::Current(-3)).unwrap_err();
+ assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput);
+
+ let err = range_reader.seek(SeekFrom::End(20)).unwrap_err();
+ assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput);
+ }
+}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH proxmox v5 02/17] add proxmox-oci crate
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 01/17] io: introduce RangeReader for bounded reads Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 03/17] proxmox-oci: add tests for whiteout handling Filip Schauer
` (14 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This crate can parse an OCI image tarball and extract its rootfs. Layers
are applied in sequence, but an overlay filesystem is currently not
used.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v4:
* update oci-spec dependency to 0.8.3
Changed since v3:
* correctly handle whiteout edge cases
* parse_and_extract_image: add argument for CPU architecture
Changed since v2:
* remove reachable unwraps & refactor code
* increase hasher buffer size from 4096 to 32768 (matching internal
sha2::Digest buffering)
* preserve permissions and xattrs during rootfs extraction
* handle whiteouts & opaque whiteouts
Cargo.toml | 1 +
proxmox-oci/Cargo.toml | 22 +++
proxmox-oci/debian/changelog | 5 +
proxmox-oci/debian/control | 45 +++++
proxmox-oci/debian/debcargo.toml | 7 +
proxmox-oci/src/lib.rs | 324 +++++++++++++++++++++++++++++++
proxmox-oci/src/oci_tar_image.rs | 144 ++++++++++++++
7 files changed, 548 insertions(+)
create mode 100644 proxmox-oci/Cargo.toml
create mode 100644 proxmox-oci/debian/changelog
create mode 100644 proxmox-oci/debian/control
create mode 100644 proxmox-oci/debian/debcargo.toml
create mode 100644 proxmox-oci/src/lib.rs
create mode 100644 proxmox-oci/src/oci_tar_image.rs
diff --git a/Cargo.toml b/Cargo.toml
index ce249371..ff12d7b7 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -27,6 +27,7 @@ members = [
"proxmox-network-api",
"proxmox-network-types",
"proxmox-notify",
+ "proxmox-oci",
"proxmox-openid",
"proxmox-product-config",
"proxmox-resource-scheduling",
diff --git a/proxmox-oci/Cargo.toml b/proxmox-oci/Cargo.toml
new file mode 100644
index 00000000..6a87c0bd
--- /dev/null
+++ b/proxmox-oci/Cargo.toml
@@ -0,0 +1,22 @@
+[package]
+name = "proxmox-oci"
+description = "OCI image parsing and extraction"
+version = "0.1.0"
+
+authors.workspace = true
+edition.workspace = true
+exclude.workspace = true
+homepage.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version.workspace = true
+
+[dependencies]
+flate2.workspace = true
+oci-spec = "0.8.3"
+sha2 = "0.10"
+tar.workspace = true
+thiserror = "1"
+zstd.workspace = true
+
+proxmox-io.workspace = true
diff --git a/proxmox-oci/debian/changelog b/proxmox-oci/debian/changelog
new file mode 100644
index 00000000..754d06c1
--- /dev/null
+++ b/proxmox-oci/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-oci (0.1.0-1) bookworm; urgency=medium
+
+ * Initial release.
+
+ -- Proxmox Support Team <support@proxmox.com> Mon, 28 Apr 2025 12:34:56 +0200
diff --git a/proxmox-oci/debian/control b/proxmox-oci/debian/control
new file mode 100644
index 00000000..5f116817
--- /dev/null
+++ b/proxmox-oci/debian/control
@@ -0,0 +1,45 @@
+Source: rust-proxmox-oci
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.82) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-flate2-1+default-dev <!nocheck>,
+ librust-oci-spec-0.8+default-dev (>= 0.8.3-~~) <!nocheck>,
+ librust-proxmox-io-1+default-dev (>= 1.2.0-~~) <!nocheck>,
+ librust-sha2-0.10+default-dev <!nocheck>,
+ librust-tar-0.4+default-dev <!nocheck>,
+ librust-thiserror-1+default-dev <!nocheck>,
+ librust-zstd-0.13+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.0
+Vcs-Git: git://git.proxmox.com/git/proxmox.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: proxmox-oci
+Rules-Requires-Root: no
+
+Package: librust-proxmox-oci-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-flate2-1+default-dev,
+ librust-oci-spec-0.8+default-dev (>= 0.8.3-~~),
+ librust-proxmox-io-1+default-dev (>= 1.2.0-~~),
+ librust-sha2-0.10+default-dev,
+ librust-tar-0.4+default-dev,
+ librust-thiserror-1+default-dev,
+ librust-zstd-0.13+default-dev
+Provides:
+ librust-proxmox-oci+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0-dev (= ${binary:Version}),
+ librust-proxmox-oci-0+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1.0-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1.0+default-dev (= ${binary:Version})
+Description: OCI image parsing and extraction - Rust source code
+ Source code for Debianized Rust crate "proxmox-oci"
diff --git a/proxmox-oci/debian/debcargo.toml b/proxmox-oci/debian/debcargo.toml
new file mode 100644
index 00000000..b7864cdb
--- /dev/null
+++ b/proxmox-oci/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox.git"
diff --git a/proxmox-oci/src/lib.rs b/proxmox-oci/src/lib.rs
new file mode 100644
index 00000000..cf0e4271
--- /dev/null
+++ b/proxmox-oci/src/lib.rs
@@ -0,0 +1,324 @@
+use std::collections::HashMap;
+use std::fs::{read_dir, remove_dir_all, remove_file, File};
+use std::io::{Read, Seek, SeekFrom};
+use std::path::{Path, PathBuf};
+use std::str::FromStr;
+
+use flate2::read::GzDecoder;
+pub use oci_spec::image::{Arch, Config};
+use oci_spec::image::{ImageConfiguration, ImageManifest, MediaType};
+use oci_spec::OciSpecError;
+use sha2::digest::generic_array::GenericArray;
+use sha2::{Digest, Sha256};
+use tar::{Archive, EntryType};
+use thiserror::Error;
+
+mod oci_tar_image;
+use oci_tar_image::{OciTarImage, OciTarImageBlob};
+
+const WHITEOUT_PREFIX: &str = ".wh.";
+const OPAQUE_WHITEOUT_NAME: &str = ".wh..wh..opq";
+
+fn compute_digest<R: Read, H: Digest>(
+ mut reader: R,
+ mut hasher: H,
+) -> std::io::Result<GenericArray<u8, H::OutputSize>> {
+ let mut buf = proxmox_io::boxed::zeroed(32768);
+
+ loop {
+ let bytes_read = reader.read(&mut buf)?;
+ if bytes_read == 0 {
+ break Ok(hasher.finalize());
+ }
+
+ hasher.update(&buf[..bytes_read]);
+ }
+}
+
+fn compute_sha256<R: Read>(reader: R) -> std::io::Result<oci_spec::image::Sha256Digest> {
+ let digest = compute_digest(reader, Sha256::new())?;
+ Ok(oci_spec::image::Sha256Digest::from_str(&format!("{digest:x}")).expect("valid digest"))
+}
+
+/// Build a mapping from uncompressed layer digests (as found in the image config's `rootfs.diff_ids`)
+/// to their corresponding compressed-layer digests (i.e. the filenames under `blobs/<algorithm>/<digest>`)
+fn build_layer_map<R: Read + Seek>(
+ mut oci_tar_image: OciTarImage<R>,
+ image_manifest: &ImageManifest,
+) -> Result<
+ (
+ OciTarImage<R>,
+ HashMap<oci_spec::image::Digest, oci_spec::image::Descriptor>,
+ ),
+ ExtractError,
+> {
+ let mut layer_mapping = HashMap::new();
+
+ for layer in image_manifest.layers() {
+ let digest = match layer.media_type() {
+ MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => layer.digest().clone(),
+ MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
+ let mut compressed_blob = oci_tar_image
+ .open_blob(layer.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer.digest().clone()))?;
+ let decoder = GzDecoder::new(&mut compressed_blob);
+ let hash = compute_sha256(decoder)?.into();
+ oci_tar_image = compressed_blob.into_oci_tar_image();
+ hash
+ }
+ MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
+ let mut compressed_blob = oci_tar_image
+ .open_blob(layer.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer.digest().clone()))?;
+ let decoder = zstd::Decoder::new(&mut compressed_blob)?;
+ let hash = compute_sha256(decoder)?.into();
+ oci_tar_image = compressed_blob.into_oci_tar_image();
+ hash
+ }
+ // Skip any other non-ImageLayer related media types.
+ // Match explicitly to avoid missing new image layer types when oci-spec updates.
+ MediaType::Descriptor
+ | MediaType::LayoutHeader
+ | MediaType::ImageManifest
+ | MediaType::ImageIndex
+ | MediaType::ImageConfig
+ | MediaType::ArtifactManifest
+ | MediaType::EmptyJSON
+ | MediaType::Other(_) => continue,
+ };
+
+ layer_mapping.insert(digest, layer.clone());
+ }
+
+ Ok((oci_tar_image, layer_mapping))
+}
+
+#[derive(Debug, Error)]
+pub enum ProxmoxOciError {
+ #[error("Error while parsing OCI image: {0}")]
+ ParseError(#[from] ParseError),
+ #[error("Error while extracting OCI image: {0}")]
+ ExtractError(#[from] ExtractError),
+}
+
+/// Extract the rootfs of an OCI image tar and return the image config.
+///
+/// # Arguments
+///
+/// * `oci_tar_path` - Path to the OCI image tar archive
+/// * `rootfs_path` - Destination path where the rootfs will be extracted to
+/// * `arch` - Optional CPU architecture used to pick the first matching manifest from a multi-arch
+/// image index. If `None`, the first manifest will be used.
+pub fn parse_and_extract_image<P: AsRef<Path>>(
+ oci_tar_path: P,
+ rootfs_path: P,
+ arch: Option<&Arch>,
+) -> Result<Option<Config>, ProxmoxOciError> {
+ let (oci_tar_image, image_manifest, image_config) = parse_image(oci_tar_path, arch)?;
+
+ extract_image_rootfs(oci_tar_image, &image_manifest, &image_config, rootfs_path)?;
+
+ Ok(image_config.config().clone())
+}
+
+#[derive(Debug, Error)]
+pub enum ParseError {
+ #[error("OCI spec error: {0}")]
+ OciSpec(#[from] OciSpecError),
+ #[error("Wrong media type")]
+ WrongMediaType,
+ #[error("IO error: {0}")]
+ Io(#[from] std::io::Error),
+ #[error("Unsupported CPU architecture")]
+ UnsupportedArchitecture,
+ #[error("Missing image config")]
+ MissingImageConfig,
+}
+
+fn parse_image<P: AsRef<Path>>(
+ oci_tar_path: P,
+ arch: Option<&Arch>,
+) -> Result<(OciTarImage<File>, ImageManifest, ImageConfiguration), ParseError> {
+ let oci_tar_file = File::open(oci_tar_path)?;
+ let mut oci_tar_image = OciTarImage::new(oci_tar_file)?;
+
+ let image_manifest = oci_tar_image
+ .image_manifest(arch)
+ .ok_or(ParseError::UnsupportedArchitecture)??;
+
+ let image_config_descriptor = image_manifest.config();
+
+ if image_config_descriptor.media_type() != &MediaType::ImageConfig {
+ return Err(ParseError::WrongMediaType);
+ }
+
+ let mut image_config_file = oci_tar_image
+ .open_blob(image_config_descriptor.digest())
+ .ok_or(ParseError::MissingImageConfig)?;
+ let image_config = ImageConfiguration::from_reader(&mut image_config_file)?;
+
+ Ok((
+ image_config_file.into_oci_tar_image(),
+ image_manifest,
+ image_config,
+ ))
+}
+
+#[derive(Debug, Error)]
+pub enum ExtractError {
+ #[error("Incorrectly formatted digest: \"{0}\"")]
+ InvalidDigest(String),
+ #[error("Unknown layer digest {0} found in rootfs.diff_ids")]
+ UnknownLayerDigest(oci_spec::image::Digest),
+ #[error("Layer file {0} mentioned in image manifest is missing")]
+ MissingLayerFile(oci_spec::image::Digest),
+ #[error("IO error: {0}")]
+ Io(#[from] std::io::Error),
+ #[error("Layer has wrong media type: {0}")]
+ WrongMediaType(String),
+}
+
+fn extract_image_rootfs<R: Read + Seek, P: AsRef<Path>>(
+ oci_tar_image: OciTarImage<R>,
+ image_manifest: &ImageManifest,
+ image_config: &ImageConfiguration,
+ target_path: P,
+) -> Result<(), ExtractError> {
+ let (mut oci_tar_image, layer_map) = build_layer_map(oci_tar_image, image_manifest)?;
+
+ for layer in image_config.rootfs().diff_ids() {
+ let layer_digest = oci_spec::image::Digest::from_str(layer)
+ .map_err(|_| ExtractError::InvalidDigest(layer.to_string()))?;
+ let layer_descriptor = layer_map
+ .get(&layer_digest)
+ .ok_or(ExtractError::UnknownLayerDigest(layer_digest.clone()))?;
+ let mut layer_file = oci_tar_image
+ .open_blob(layer_descriptor.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer_digest))?;
+
+ type DecodeFn<T> = Box<dyn for<'a> Fn(&'a mut T) -> std::io::Result<Box<dyn Read + 'a>>>;
+ let decode_fn: DecodeFn<OciTarImageBlob<R>> = match layer_descriptor.media_type() {
+ MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => {
+ Box::new(|file| Ok(Box::new(file)))
+ }
+ MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
+ Box::new(|file| Ok(Box::new(GzDecoder::new(file))))
+ }
+ MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
+ Box::new(|file| Ok(Box::new(zstd::Decoder::new(file)?)))
+ }
+ // Error on any other non-ImageLayer related media types.
+ // Match explicitly to avoid missing new image layer types when oci-spec updates.
+ media_type @ (MediaType::Descriptor
+ | MediaType::LayoutHeader
+ | MediaType::ImageManifest
+ | MediaType::ImageIndex
+ | MediaType::ImageConfig
+ | MediaType::ArtifactManifest
+ | MediaType::EmptyJSON
+ | MediaType::Other(_)) => {
+ return Err(ExtractError::WrongMediaType(media_type.to_string()))
+ }
+ };
+
+ apply_whiteouts(&mut decode_fn(&mut layer_file)?, &target_path)?;
+ layer_file.seek(SeekFrom::Start(0))?;
+ extract_archive(&mut decode_fn(&mut layer_file)?, &target_path)?;
+
+ oci_tar_image = layer_file.into_oci_tar_image();
+ }
+
+ Ok(())
+}
+
+/// Apply whiteouts on previous layers
+fn apply_whiteouts<R: Read, P: AsRef<Path>>(reader: &mut R, target_path: P) -> std::io::Result<()> {
+ let mut archive = Archive::new(reader);
+
+ for entry in archive.entries()? {
+ let file = entry?;
+ if file.header().entry_type() != EntryType::Regular {
+ continue;
+ }
+
+ let filepath = file.path()?;
+ if let Some(filename) = filepath.file_name() {
+ if filename == OPAQUE_WHITEOUT_NAME {
+ if let Some(parent) = filepath.parent() {
+ let whiteout_abs_path = target_path.as_ref().join(parent);
+ if whiteout_abs_path.exists() {
+ for direntry in read_dir(whiteout_abs_path)? {
+ remove_path(direntry?.path())?;
+ }
+ }
+ }
+ } else if let Some(filename) = filename.to_str() {
+ // TODO: Simplify this once OsStr::strip_prefix is implemented
+ if let Some(filename_stripped) = filename.strip_prefix(WHITEOUT_PREFIX) {
+ let whiteout_path = match filename_stripped {
+ "." => match filepath.parent() {
+ Some(p) if p.parent().is_some() => p,
+ _ => continue, // Prevent whiteout of root directory
+ },
+ ".." => continue, // Prevent whiteout of grandparent directory
+ fname => &filepath.with_file_name(fname),
+ };
+ let whiteout_abs_path = target_path.as_ref().join(whiteout_path);
+ if whiteout_abs_path.exists() {
+ remove_path(whiteout_abs_path)?;
+ }
+ }
+ }
+ }
+ }
+
+ Ok(())
+}
+
+fn extract_archive<R: Read, P: AsRef<Path>>(reader: &mut R, target_path: P) -> std::io::Result<()> {
+ let mut archive = Archive::new(reader);
+ archive.set_preserve_ownerships(true);
+ archive.set_preserve_permissions(true);
+ archive.set_unpack_xattrs(true);
+
+ // Delay directory entries until the end (they will be created if needed by descendants),
+ // to ensure that directory permissions do not interfere with descendant extraction.
+ let mut directories = Vec::new();
+ for entry in archive.entries()? {
+ let mut file = entry?;
+ if file.header().entry_type() == EntryType::Directory {
+ directories.push(file);
+ continue;
+ } else if file.header().entry_type() == EntryType::Regular {
+ // Skip whiteout files
+ if let Some(filename) = file.path()?.file_name() {
+ if filename == OPAQUE_WHITEOUT_NAME {
+ continue;
+ } else if let Some(filename) = filename.to_str() {
+ if filename.starts_with(WHITEOUT_PREFIX) {
+ continue;
+ }
+ }
+ }
+ }
+
+ file.unpack_in(&target_path)?;
+ }
+
+ // Apply the directories in reverse topological order,
+ // to avoid failure on restrictive parent directory permissions.
+ directories.sort_by(|a, b| b.path_bytes().cmp(&a.path_bytes()));
+ for mut dir in directories {
+ dir.unpack_in(&target_path)?;
+ }
+
+ Ok(())
+}
+
+fn remove_path(path: PathBuf) -> std::io::Result<()> {
+ if path.metadata()?.is_dir() {
+ remove_dir_all(path)
+ } else {
+ remove_file(path)
+ }
+}
diff --git a/proxmox-oci/src/oci_tar_image.rs b/proxmox-oci/src/oci_tar_image.rs
new file mode 100644
index 00000000..23e1bfe0
--- /dev/null
+++ b/proxmox-oci/src/oci_tar_image.rs
@@ -0,0 +1,144 @@
+use std::collections::HashMap;
+use std::io::{Read, Seek, SeekFrom};
+use std::ops::Range;
+use std::path::{Path, PathBuf};
+
+use oci_spec::image::{Arch, Digest, ImageIndex, ImageManifest, MediaType};
+use oci_spec::OciSpecError;
+use tar::Archive;
+
+use proxmox_io::RangeReader;
+
+#[derive(Clone)]
+struct TarEntry {
+ range: Range<u64>,
+}
+
+impl TarEntry {
+ fn new(range: Range<u64>) -> Self {
+ Self { range }
+ }
+}
+
+pub struct OciTarImage<R: Read + Seek> {
+ reader: R,
+ entries: HashMap<PathBuf, TarEntry>,
+ image_index: ImageIndex,
+}
+
+impl<R: Read + Seek> OciTarImage<R> {
+ pub fn new(reader: R) -> oci_spec::Result<Self> {
+ let mut archive = Archive::new(reader);
+ let entries = archive.entries_with_seek()?;
+ let mut entries_index = HashMap::new();
+ let mut image_index = None;
+
+ for entry in entries {
+ let mut entry = entry?;
+ let offset = entry.raw_file_position();
+ let size = entry.size();
+ let path = entry.path()?.into_owned();
+
+ if path.as_path() == Path::new("index.json") {
+ image_index = Some(ImageIndex::from_reader(&mut entry)?);
+ }
+
+ let tar_entry = TarEntry::new(offset..(offset + size));
+ entries_index.insert(path, tar_entry);
+ }
+
+ if let Some(image_index) = image_index {
+ Ok(Self {
+ reader: archive.into_inner(),
+ entries: entries_index,
+ image_index,
+ })
+ } else {
+ Err(OciSpecError::Other("Missing index.json file".into()))
+ }
+ }
+
+ pub fn image_index(&self) -> &ImageIndex {
+ &self.image_index
+ }
+
+ fn get_blob_entry(&self, digest: &Digest) -> Option<TarEntry> {
+ let path = get_blob_path(digest);
+ self.entries.get(&path).cloned()
+ }
+
+ pub fn open_blob(self, digest: &Digest) -> Option<OciTarImageBlob<R>> {
+ if let Some(entry) = self.get_blob_entry(digest) {
+ Some(OciTarImageBlob::new(self, entry.range))
+ } else {
+ None
+ }
+ }
+
+ pub fn image_manifest(
+ &mut self,
+ architecture: Option<&Arch>,
+ ) -> Option<oci_spec::Result<ImageManifest>> {
+ let digest = match self.image_index.manifests().iter().find(|d| {
+ d.media_type() == &MediaType::ImageManifest
+ && architecture
+ .is_none_or(|a| d.platform().as_ref().is_none_or(|p| p.architecture() == a))
+ }) {
+ Some(descriptor) => descriptor.digest(),
+ None => return None,
+ };
+
+ if let Some(entry) = self.get_blob_entry(digest) {
+ let mut range_reader = RangeReader::new(&mut self.reader, entry.range);
+ Some(ImageManifest::from_reader(&mut range_reader))
+ } else {
+ Some(Err(OciSpecError::Other(format!(
+ "Image manifest with digest {digest} mentioned in image index is missing"
+ ))))
+ }
+ }
+}
+
+fn get_blob_path(digest: &Digest) -> PathBuf {
+ let algorithm = digest.algorithm();
+ let digest = digest.digest();
+ format!("blobs/{algorithm}/{digest}").into()
+}
+
+pub struct OciTarImageBlob<R: Read + Seek> {
+ range_reader: RangeReader<R>,
+ entries: HashMap<PathBuf, TarEntry>,
+ image_index: ImageIndex,
+}
+
+impl<R: Read + Seek> OciTarImageBlob<R> {
+ fn new(archive: OciTarImage<R>, range: Range<u64>) -> Self {
+ let range_reader = RangeReader::new(archive.reader, range);
+
+ Self {
+ range_reader,
+ entries: archive.entries,
+ image_index: archive.image_index,
+ }
+ }
+
+ pub fn into_oci_tar_image(self) -> OciTarImage<R> {
+ OciTarImage {
+ reader: self.range_reader.into_inner(),
+ entries: self.entries,
+ image_index: self.image_index,
+ }
+ }
+}
+
+impl<R: Read + Seek> Read for OciTarImageBlob<R> {
+ fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
+ self.range_reader.read(buf)
+ }
+}
+
+impl<R: Read + Seek> Seek for OciTarImageBlob<R> {
+ fn seek(&mut self, pos: SeekFrom) -> std::io::Result<u64> {
+ self.range_reader.seek(pos)
+ }
+}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH proxmox v5 03/17] proxmox-oci: add tests for whiteout handling
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 01/17] io: introduce RangeReader for bounded reads Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 02/17] add proxmox-oci crate Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox-perl-rs v5 04/17] add Perl mapping for OCI container image parser/extractor Filip Schauer
` (13 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Test extracting OCI images with whiteout special cases.
Test cases inspired by:
https://github.com/containerd/containerd/blob/4312e076a8a3/pkg/archive/tar_test.go
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v4
proxmox-oci/Cargo.toml | 3 +
| 92 ++++++++++++++++++
.../oci_test_whiteout_current_directory.tar | Bin 0 -> 8704 bytes
.../oci_test_whiteout_dead_symlink_parent.tar | Bin 0 -> 8704 bytes
.../oci_test_whiteout_root_breakout.tar | Bin 0 -> 7168 bytes
...oci_test_whiteout_root_parent_breakout.tar | Bin 0 -> 7168 bytes
.../oci_test_whiteout_symlink.tar | Bin 0 -> 8704 bytes
7 files changed, 95 insertions(+)
create mode 100644 proxmox-oci/tests/extract_whiteouts.rs
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar
diff --git a/proxmox-oci/Cargo.toml b/proxmox-oci/Cargo.toml
index 6a87c0bd..8d29da73 100644
--- a/proxmox-oci/Cargo.toml
+++ b/proxmox-oci/Cargo.toml
@@ -20,3 +20,6 @@ thiserror = "1"
zstd.workspace = true
proxmox-io.workspace = true
+
+[dev-dependencies]
+proxmox-sys.workspace = true
--git a/proxmox-oci/tests/extract_whiteouts.rs b/proxmox-oci/tests/extract_whiteouts.rs
new file mode 100644
index 00000000..71ec4dea
--- /dev/null
+++ b/proxmox-oci/tests/extract_whiteouts.rs
@@ -0,0 +1,92 @@
+use std::fs::remove_dir_all;
+
+use proxmox_oci::{parse_and_extract_image, Arch};
+use proxmox_sys::fs::make_tmp_dir;
+
+#[test]
+fn test_whiteout_root_breakout() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_root_breakout.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ // Check that the whiteout did not remove the root directory
+ assert!(extract_dir.exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_root_parent_breakout() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ // Check that the whiteout did not remove the root directory
+ assert!(extract_dir.exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_current_directory() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_current_directory.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(!extract_dir.join("etc").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_symlink() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_symlink.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(extract_dir.join("etc/passwd").exists());
+ assert!(!extract_dir.join("localetc").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_dead_symlink_parent() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(extract_dir.join("etc/passwd").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar
new file mode 100644
index 0000000000000000000000000000000000000000..56b34cfd774b74b8410839bb48333a38409b70d9
GIT binary patch
literal 8704
zcmeHLO^6&t6mB*q#3g!342vKc1`LtSPO9p4Rdp|tlb4`}#RLxti(9`lz0A)tGaIrA
zYd}yC&DoGcSn(o)H&34Q5IiafdhsfF^&%b&@m0@cC$r;j;?B&%V%MR6URS+(^<I77
zd)2z$(wz&@R2#v@35miT+s^xwvfeW#OtPrNr)pJS`Ff|T+Gx_aa|O$>Z7*(IeFrpN
z=kf2<6ysuA+aUPmzr~*dDdK-1J76gOh7zk%sbfF{aw3>ad7PwPLP`@Qq@!SgE5ekv
z95|;$0+&h%pK!~q-o@e&|M&@Lgm-L@p+XqY?dM$1C<q>v#>V6>_6&*t=lDPK@$utd
z-219Fb3_3M3OoUoKuTCDXyUnZi2}*EW&&G^`@j=rg-nEuDL9kr*gC^}0ua5qaOvk?
z=70F{C$semiDuyBN%F#TzsGkQwbhl?dc9WHCr-(jaPF4K%GQx{3zbXfXBIzr`TiS!
z{(5fi=;te^PPC)w{`@zuM#qLiRJ=^j14H=Nf+4Dj2PzE}OgP|H#lW>pxC8?>rkW^W
zK&3(%%?T5!L1-KB+6mY<{>6X``uLX=3j9wm>I2OI|7X7p_}8GBN?FRZrHNoj?x`lR
zaUMj>ElG*dNcSKGc;cAQQgDt}#vzf;Tc6{<x<wKCoy(Vd(0_R5#f$GD`3nKnCF1&^
ze|u?V?z7TMbB~UG^x(m7uYG;zHgd)5f3!Y4{phSe_U&JC?#}IdN3MSVu*V9Yy#L+v
z(Z#>-ej3dV>@J=r<AEXkQ|B3mR(PeX3aNk+ZG^{N5Ku2uP$psCSR+-;4fj$zNW9Tl
zA(&XH{o<cfxg-BGDX{)4^8aLFyqA_+v!zm5wM{MSI@4WmyYeEwH>`-u=ffw{cB<=a
zkTR?Z7f7<e_!UYQNxDdQ1q7xH#^U&DuX`=)bX)BW?0#)IAK&VPq4lh}exui;-D-7x
z5Qdp?g6_r|&RMRj4cG4ET~KH<@88WVcDt7MLQ(ICvN$eF^=B9;9JlN#=LRn0b#r#}
z@DbDy{*{P5!E#Rv=egz~scWSrW_Xrb=DeXPagI~Py}@)(EMbnOoZ8rtSaG(?MhnIM
z9`!#Rt^XO5w7~x^?wqJN$A8DvTtmHs0L)rVgeJ?O+i*5hS8{kFC9JL0Go!lL;llN%
zt+Y&5$r`HaDh<`lyz6u;*XjIf2gmn#qt*0TbpZ2cRA%yFL#lE*sNw{HNct==ZlZe3
zGo9>aAUpsb@S7XnCn6-U9N0aULK~iO0r_yWzk+|2=BqcewE^MSY{x3bJ#G(F_t!x=
z<VNQ!DfJn4I<}8{R2a2gik|IwT|SIVI#F02n6j*C-Hl4X?ewO63jY(}JO2MrN@ZN|
z|MVg<(7YG__lZ3eYx^U^e@zW})i=V0<CY#Hd?G~P^iY)*d@>04$P?!A1<YBk_3p^0
zi3y&De*$9o{6{ICH46MsEC72bx>e4rxVQ2xuw4O`X@yisd2^3}#izmpg$D``6dpL#
G9{2}Zftk|)
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar
new file mode 100644
index 0000000000000000000000000000000000000000..b1b5094689402cce0433174f79c4595c10dae392
GIT binary patch
literal 8704
zcmeHLU5Hdw93K=H8)k*T@*$~h?Bl_9JfHX6yDu7)U}~C5LLuw<+`YOpv(DTVM_baO
zAYADo>Y-?%5A&&njE$_fC=7)xq8@w*q`)2`33^Df?SJo#wzKQBqq8eW_dMLW_dn<3
zcmC)6e&>JwYN)KLt1|Oi5EPD3CjNyo{dcOZ{*Dn5B-6v^6;-42k!mC>0IAclV&qV+
zPOj7bZc=-dtbcV-67J2{Hi-Rled~`YO4t9U?0}~AR}9ukVro2PJ>_}MNyM#2fwDeB
z#Bt(dC#AHOAdh=SaxN&>0aWz_30w=r@%pnV)QG>KT%igk6kmC-+X=xQtbfOxY=NiA
z{6FXafoxAtzN!apQ9fxa)v#R3_w|s6Vr$WwP;RRYTd?Hi@bFNmWds#g?Jb!?S%-zN
zC<knzC`+NU)u^x+uiRCI_DN%1F1c_3TE+vk-&ws`Q&-T~gaa@@45KboeLm6>Nx`fV
z#%e#XlpA5K@QAXKD<iQMJfMNoL}&>WxUkwGB1yoBQY^t3PdwZo&yG?KC`S|EL$U^b
z0OQjM@H+hVCy+D;`~^5wRu1eBhZ{p(YZjg&fKvya(@X$DBb*b+X(N@4>Eswuz&hbH
zwH#qfg+)GPK2yeUYuvwGH|BvENkAtv>1nurez03(F)$yP>LeG+cvx{PsHKi6t&ugv
zC*X2VBdwiKm=i%2p^SQ#IAwyF*r}*vhv$y$Xts%@>BoF|ph^EbrBw@%v8ext{v+5p
z)AgTwBGvyE?wnI|qW`j@2gAtfXrxjX`=asc8kyG>D<f-S?InotUgY-@z769(6!alp
zpq@u4A&fnh5IR2?R-<yI7R%?3L{7`Gl;(%R(#XDgjY_#3x!Bk^7^38W$v0#-Zj(r*
zbN!)#YIPL`Q$zV7Bq0nGQwClTXhS6gCmacewWL@&%(!q6#xQF*NE0`V351!mL4YK4
zN=(t6B<AhYR2$FZ92A!Y4B}1(T4g|kaUeA%Txp+!;ChrGUm=0H&$-r?_)J^h1YSNt
zyaS%2x1wq_@xRjpsm0EmjsFns&EUVlDgRrzb570o9A3P*cHx7;1q+p|+Bj;-{Q4cv
z9z?lCXO|xp=$nc0V}l>Rt?oEp9^SI_!4=us`janQ{N?4XTMr_$WX~J>#+Utea^lXh
z=a-$SWmg=3=KQJ4dsg2bOy2v`XCHm|=ecu#UB=Jmf4@7o>+kdLp4k55YnL9luW)G7
zAFH4E`IE`h6NfKNzPjbHFSFwp-&&u^&FRF_&f4-o6aPC6>ROCAJO5|g{}^K_|690o
zPR%j@FTDPK%zrrPcb7+^{s`_`{oCV!0KOiv(rm32;Q-&gSypT*N+ja+SDPL`v#xjD
z_dl%s`s~7Qb1c)fW~WmN-aC|ev+G3n#_kPAj&vW{ICkj#i)+_*xm>m@7iErqasNw?
zt~s{(^H)xd-?iwSolD1_dw5Ca)NP+G&veb{oYKzP_dr-Oc3&aBx3}NsdTY+cf6Axv
zA7d<1{$H<~Z>W9}|F_8hG!i>Eo(#EWYN!$5Ps`|ZntGfP*Vl785Ys1-`6xWod_v2;
z0eQS$xO;)px2)3fAG7fv(P{BN#&Fk6&;O1c-$n-q@<|qVL$U>~W`Og!fC?x-+C~Ry
RmDB^N2T~8D9=Ihv@DEuzwG;pV
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar
new file mode 100644
index 0000000000000000000000000000000000000000..36e25851148064fdc39d0fee90bdd5762e187989
GIT binary patch
literal 7168
zcmeHL%ZeLE6m35+X2`5;jCT=D7J<pM>)BPMW!8aU7_vwhl6sDMkko_JV>}+TF=Uf?
zX^1fd@&oya{6Gx;iF`tqxm8j|mNN2;qmc=5*P_*Zse9`_>eRU<)343=LAKF>U~q)8
z_zPqC_f6UK9U~%0*5MoVYG(X++~^UwwC-Bba~NmAt!wW+>6=vlaZeLoZfYCEyu4fe
zF-2|t-^&hIsK0ewIa~(kC^gD)jhPqHV?|sUj0s-DgHVd8@|H{M5n&2Di<~x$yK;rc
zvHt80XvA+Ae?tWc%Fo`DoRE^ztg|*ISNO9a{=o5ny4C6AW83os{goe$t9qDsJLIz{
z9e7viy^|vkl3X7h^((6znDF4y&=s|<ifW(_d@<0&D)@0zJi<GNW6)2^wjPG+05szQ
zmp3+5TQCL6t~!7Pq8rnLs#_ApFcZo$;%Mn4a{+5CEi+0A%T>VEawlC$L<kD@rqwba
zZIF;9Gu|X0W8l@37#-mZyqqM*`}zd(0K#vQ<5Tx9A3#zE{|jK1or5RU(QK*nqOfI<
zAq1p?S2zH+N`(PU97K#sf+?VmDW^Tv)=<wF^IS>`N+2{r&P~taz)U8mC-B_gnrtm>
z^z%O!@xP`+=P!}4{Wkm)!j|zbIBM~~BAr(%{`kk+x4(b+Yw!9E&kz<$Ifton0TFHq
z6h=FQgAiJ1!%JIg*npCIg(TB)<5DAm2pEwDXRmjDfA!k_vWv3o^v)f$ee)0YV$eJM
z?y%qQ_030j#myh?<4^CSSJ$53`{|Q=*M9zd`^)@vc=y@UhiL1T%Cf!R{@l)PU1@l&
zu?sS=fd6%m)fH<+vVR-?sVJB64^2jk{}t)HQgPycJ+i&3@wPc0P04=XN;c#1k=KoP
zk$VZ83A;${626CV7pX4d1uaWL;h11wPp9{)aZ`^@B7N>BI;|s>=KX4T{CGNJRM$<2
z!dYX`qy{6O(Qed9R2jmzm4j(h7LHV%t|(n5EM?Af?X^^f2%)&P*a72vhdB<$y5KR)
z2o4KgBBr>Y5(zl0IEB+IPYdkBb8<O?ywYs{_l#9T=N}ibZ`+)27vi7eMf_uo1#9uY
zxr)p!C;YGI|7VT;zi)=my&B8_zSPmLd-eQBFU*8UP=AMBc%GFH=MC<0yKwf_$Ia$q
zz6k$_F7ZFcz%?!YH&>Cl<<mUf;=f2+;OrlO92clS`D8w?eQPt&W}wYLn}PpV2L1xB
CF8omd
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar
new file mode 100644
index 0000000000000000000000000000000000000000..12421d00acc42a589f2ccaf50cd085ceb34c4ad4
GIT binary patch
literal 7168
zcmeHKOK%iM5MCK4LKd9h5OKlU6Ucbz_hS#7azWyd1L6R(a8EzHW9{x(J7a`FK61)A
zQbeQ(DYr=ZEs5kSKOny#XR3P_n>DPxWV{?;TH2kNn(FH6>iWK_$p>X#-R&%OAQ&8>
zPQ1c6ef>~2xyOh|(pl$A^=fK-zpBj`Txz;j^c=?V=9=33Li6)<{#D-)E|!%BF)p8X
z{+Ocn{J%&B%$&bRURY+OVT36{xz@(B;JGANGR+ZIG&fdR61byMGR3^l9aYo@NFQqE
zcpT54eE^O44C6gi5S8NiKFJA*m-WV&oa4`o_#4On%eD3Ot{K~YQF~kOkA1fX&qF6z
z_fC9e$KKT5MJYjqZz8ox_(P0)NcRxWs1Sq_!r0@9dcUaZa(n>lPqxzhvWim&#c2Qe
zL}Of*b%?^aGAvLZ?7^DdfjRKwDye|b6(Kw=9H>1>S|FNGR8K%^Qj9E;!C<Ym@_{3Z
zEz=5HDRU}_7M5efl)#EeYLK)H(pFgoa@|e#w&4G8?Qm@-6TZoZ8T^NUbFNS>1f^Cp
zs&nhDM<`&;92F2s=Q!t-IW3G*7;7WAl))KE16E>A1S8nb!oQ^aWBk(=|8v55q2h%9
z%J%)xJOTg~<p`ij5_E^&73N`rCs4xP-k`9i28FxNMlLIDkrhL;<Fla|6~R|^_6%?A
zRxp3U8|5ezJ0~z-(3nlrW=thI*ntHikxW^jX`y=0Gga{-5+30!5<+r(N<=_li76&5
z1wJgefixZb&!DT^*m+UxO_%!AE;!{zX>CJp19l2wL0JeWXsn#nNWf19flzKVWr%Z)
zCD)vq96KJMN}rgX?gKNOoIZsAt;OhV2LDYRnv<F5;h&Jx_=iZy7XNd?d7<K)KVQD=
zU;ou#xzx0>c8)gw{^#h(zPXJ$EA-kmbanM#_Uo{}yS+OY^auHs>vHu^x&0H`esk&8
zW_JJ1O7GQ;-+%n??X4fa`(yk1m2s!@%WtdqI^SRD{#s*aN1zxv|2&JleYRecx98!X
zay(oAF^2Dr7XM9RzUbVu>i@K`$7cQi6~j-q2C3_J#i@$kwDtU`7BVS6w-zoMv#&`H
z`nXN#S<3x-dAm9f|A?OY{>K=;HCy~I??g^4zwD+i{y|EC<1c_N&QONBhbMaNT^oTm
N0&N7^2z*@;cn3S&@<;#x
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar
new file mode 100644
index 0000000000000000000000000000000000000000..7e6835cab889566cdf1d8dc1a42b69642b9d0463
GIT binary patch
literal 8704
zcmeHL-D_M$6u(Kdl$bo!hEVjuuoMlZ+0OUeSrC0Fg`j95ML`M8d}eRDA8~h6l9oV0
z{D3?Og28|WKVF0iz6t&VN?QyFeG)_p;;RUKlGd5K+nSBtW|K_{#XApo?>&3Y%$YOu
z`^}l7N>x=q869W?zEJ?8@ax<5->tIdI|PWMXpA1HSFO$$>kV1+F7-NA@Z7h1lk2s=
zhh(qP^{*EtqU@lyL5R!l)*oV!t^Y&W0d4EAq*e@AV2ISxafCP!MiL+ew}v2zfmFc8
zmRT875JP7PHc&IHsWMWU4jzZ~CtIKqzJ0q36^Jmr``(lz#4wM>dd6f2f7-<VbNpW#
z92?8kb<vjP>$X-;s+C+}44nwlvNef3yS!k%BqtXZN{N;YuW<50#pJ6x$tPtwXY*xQ
zNt~@W@)zL5eBIlp#=2T@$(*+g19ab6)2yv4Z)}n|KR|G!NmK>R9T6I92_W<vRT3`!
z282ooY$SukQ7a9B8WE$d<{Sz|B_x(;MF@2~^)XC5xfo`LG4)=qq>h*5vX6rozLh%O
zbiexvq~_lLygyYY<}M}+t)cEWi`WSvIOA3d<6#DcWJDN?Bw@tH0>uhqOtdrrN+hHM
zkj26xU~vqv=IGz9Tl0W8PMuC?GS&+H!T64ViMg{%OR1cSjmE}uj$v$L=vRUPg#^<a
zBO@3i$V)L$R1hGvl@5Xb`i{aH_*L-FnZcF8Hjri?2Ni)f{_ho59h?lL{s8_56!Pu-
zPchE;ze70pRh;s_tm$IXu)48WYm$B0Fp?rPr)yR=tO?wU0HTv1o<#I4gasf9faWn{
z2qQ$un@#m%Qg2ji%Yi;;QaY^$D$SLW%HmQ}qgJgpToASnhDLh8<XW^FY*MOp?p$KL
zT9bvnsKxwTJIDluK%$sAE@BN7r^-_$@~$8zrFKFwEQkY?P>rde3L~MNCW^%I7T!s7
z-h%a7ELMQ|Kc!?wX-~aA*P%ef2KfVpSVlQyRzM4lLR@j@6CbT(M|`4$bE(HO{<-of
zvQ``Zdp(dk?983;k09EHf5;)s@ZTYv`zn5B_3+`%>)#an2b8Sa=;g_&8*>{FME%nP
z1E73)O3U^2^))cQhBo^?8vXeAXy27D-+S)E7x<}{o(H#|cye|1>U8nxJI^lN{B?-l
z9v&XPe(l)E(CM3>{eJnz+b<3t;y?WO+N;<4M*H9S`fpAM<K_(b{M+wO|8-~l*yCT!
z9v_Hi|NQE=XlP$2m37v21lsW5Yf{%?)Sd7TIo*bTglLBU4$<6Salrp~pIiy}_b2`N
z>SE(u!{4>ey%7=u|24f__J{ZRaGi;Mp7?3=YJl_9<doYu;z<G@Jqm^&`-7~Ni}SPd
zrBbn^hDJQhA#-QI?8d<OWPa*If8pJyK0S5omlW_LL$xUS;MV&`qC@*Tk*v2pMIfmd
zyOa<2_C1+*WwaCiu?Kt5X|>2->LDlD`QKG=dsIsE{|@z^R$*uF#gO}!hFS^!wu(-#
zrN=FKeY2*+SXNKyVsH35-lV^Kd8Y3|d5<aq*AKe?1I)JL-`_Pe{O>W`yE40!ORKoA
hq+8%#1vm%uAP;gYyXq=a&LWUSAd5g2fk!9;{{UmyntT8N
literal 0
HcmV?d00001
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH proxmox-perl-rs v5 04/17] add Perl mapping for OCI container image parser/extractor
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (2 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 03/17] proxmox-oci: add tests for whiteout handling Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 05/17] config: add `lxc.environment.runtime`/`hooks` Filip Schauer
` (12 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This patch depends on the proxmox-oci crate added in patch 02/15.
Changed since v3:
* improve rustdoc comments
* parse_and_extract_image: add argument for CPU architecture
Changed since v2:
* rebase onto newest master (6132d4d36cbd)
* forward all errors to Perl
* remove oci-spec dependency
Changed since v1:
* rebase on latest master (3d9806cb3c7f)
* add new dependencies to debian/control
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 1 +
pve-rs/debian/control | 1 +
pve-rs/src/bindings/mod.rs | 3 +++
pve-rs/src/bindings/oci.rs | 29 +++++++++++++++++++++++++++++
5 files changed, 35 insertions(+)
create mode 100644 pve-rs/src/bindings/oci.rs
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index e40c55c..8d87399 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -39,6 +39,7 @@ proxmox-http-error = "1"
proxmox-log = "1"
proxmox-network-types = "0.1"
proxmox-notify = { version = "1", features = ["pve-context"] }
+proxmox-oci = "0.1.0"
proxmox-openid = "1.0.2"
proxmox-resource-scheduling = "1"
proxmox-section-config = "3"
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index 13a5418..aa7181e 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -27,6 +27,7 @@ PERLMOD_GENPACKAGE := /usr/lib/perlmod/genpackage.pl \
PERLMOD_PACKAGES := \
PVE::RS::Firewall::SDN \
+ PVE::RS::OCI \
PVE::RS::OpenId \
PVE::RS::ResourceScheduling::Static \
PVE::RS::SDN::Fabrics \
diff --git a/pve-rs/debian/control b/pve-rs/debian/control
index e5c2fc8..4b892a8 100644
--- a/pve-rs/debian/control
+++ b/pve-rs/debian/control
@@ -28,6 +28,7 @@ Build-Depends: cargo:native <!nocheck>,
librust-proxmox-network-types-0.1+default-dev,
librust-proxmox-notify-1+default-dev,
librust-proxmox-notify-1+pve-context-dev,
+ librust-proxmox-oci-0.1+default-dev,
librust-proxmox-openid-1+default-dev (>= 1.0.2-~~),
librust-proxmox-resource-scheduling-1+default-dev,
librust-proxmox-section-config-3+default-dev,
diff --git a/pve-rs/src/bindings/mod.rs b/pve-rs/src/bindings/mod.rs
index 7730de3..c21b328 100644
--- a/pve-rs/src/bindings/mod.rs
+++ b/pve-rs/src/bindings/mod.rs
@@ -1,5 +1,8 @@
//! This contains all the perl bindings.
+mod oci;
+pub use oci::pve_rs_oci;
+
mod resource_scheduling_static;
pub use resource_scheduling_static::pve_rs_resource_scheduling_static;
diff --git a/pve-rs/src/bindings/oci.rs b/pve-rs/src/bindings/oci.rs
new file mode 100644
index 0000000..7f5acaf
--- /dev/null
+++ b/pve-rs/src/bindings/oci.rs
@@ -0,0 +1,29 @@
+#[perlmod::package(name = "PVE::RS::OCI", lib = "pve_rs")]
+pub mod pve_rs_oci {
+ //! The `PVE::RS::OCI` package.
+ //!
+ //! Provides bindings for the [`proxmox_oci`] crate.
+
+ use anyhow::Error;
+ use proxmox_oci::Config;
+
+ /// Extract the rootfs of an OCI image tar and return the image config.
+ ///
+ /// # Arguments
+ ///
+ /// * `oci_tar_path` - Path to the OCI image tar archive
+ /// * `rootfs_path` - Destination path where the rootfs will be extracted to
+ /// * `arch` - Optional CPU architecture used to pick the first matching manifest from a multi-arch
+ /// image index. If `None`, the first manifest will be used.
+ #[export]
+ pub fn parse_and_extract_image(
+ oci_tar_path: &str,
+ rootfs_path: &str,
+ arch: Option<&str>,
+ ) -> Result<Config, Error> {
+ let arch = arch.map(Into::into);
+ proxmox_oci::parse_and_extract_image(oci_tar_path, rootfs_path, arch.as_ref())
+ .map(|config| config.unwrap_or_default())
+ .map_err(Into::into)
+ }
+}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH container v5 05/17] config: add `lxc.environment.runtime`/`hooks`
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (3 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox-perl-rs v5 04/17] add Perl mapping for OCI container image parser/extractor Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 06/17] add support for OCI images as container templates Filip Schauer
` (11 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add `lxc.environment.runtime` and `lxc.environment.hooks` config keys.
These allow setting environment variables separately for the container
init process and for LXC hooks.
This will be needed by containers created from OCI images with custom
environment variables which could interfere with hook execution.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This patch depends on the upstream lxc commit df8cf80e960e, which is
expected to be included in the next lxc release 6.0.6.
Introduced in v4
src/PVE/LXC/Config.pm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 5d3749e..1038fa7 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -736,6 +736,8 @@ my $valid_lxc_conf_keys = {
'lxc.start.order' => 1,
'lxc.group' => 1,
'lxc.environment' => 1,
+ 'lxc.environment.runtime' => 1,
+ 'lxc.environment.hooks' => 1,
# All these are namespaced via CLONE_NEWIPC (see namespaces(7)).
'lxc.sysctl.fs.mqueue' => 1,
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH container v5 06/17] add support for OCI images as container templates
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (4 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 05/17] config: add `lxc.environment.runtime`/`hooks` Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 07/17] config: add entrypoint parameter Filip Schauer
` (10 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This aims to add basic support for the Open Container Initiative image
format according to the specification. [0]
[0] https://github.com/opencontainers/image-spec/blob/main/spec.md
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This patch depends on changes made to proxmox-perl-rs in patch 04/17.
Meaning that proxmox-perl-rs needs to be bumped and a dependency & build
dependency to libpve-rs-perl needs to be added to debian/control.
Changed since v3:
* correctly handle case where $archive is '-'
* replace unnecessary regex comparisons with `eq`
* pass environment variables to container via `lxc.container.runtime`
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* check whether archive is an OCI image before trying to parse it as one
Changed since v1:
* fix entrypoint command missing Cmd
* set lxc.signal.halt according to StopSignal (Fixes container shutdown)
src/PVE/API2/LXC.pm | 94 ++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 84 insertions(+), 10 deletions(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index e53b388..7a44547 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -19,9 +19,11 @@ use PVE::Storage;
use PVE::RESTHandler;
use PVE::RPCEnvironment;
use PVE::ReplicationConfig;
+use PVE::RS::OCI;
use PVE::LXC;
use PVE::LXC::Create;
use PVE::LXC::Migrate;
+use PVE::LXC::Namespaces;
use PVE::GuestHelpers;
use PVE::VZDump::Plugin;
use PVE::API2::LXC::Config;
@@ -536,19 +538,91 @@ __PACKAGE__->register_method({
eval {
my $rootdir = PVE::LXC::mount_all($vmid, $storage_cfg, $conf, 1);
+ my $archivepath = '-';
+ $archivepath = PVE::Storage::abs_filesystem_path($storage_cfg, $archive)
+ if ($archive ne '-');
$bwlimit = PVE::Storage::get_bandwidth_limit(
'restore', [keys %used_storages], $bwlimit,
);
- print "restoring '$archive' now..\n"
- if $restore && $archive ne '-';
- PVE::LXC::Create::restore_archive(
- $storage_cfg,
- $archive,
- $rootdir,
- $conf,
- $ignore_unpack_errors,
- $bwlimit,
- );
+ my $is_oci = 0;
+
+ if ($restore && $archive ne '-') {
+ print "restoring '$archive' now..\n";
+ } elsif ($archivepath =~ /\.tar$/) {
+ # Check whether archive is an OCI image
+ my ($has_oci_layout, $has_index_json, $has_blobs) = (0, 0, 0);
+ PVE::Tools::run_command(
+ ['tar', '-tf', $archivepath],
+ outfunc => sub {
+ my $line = shift;
+ $has_oci_layout = 1 if $line eq 'oci-layout';
+ $has_index_json = 1 if $line eq 'index.json';
+ $has_blobs = 1 if $line =~ /^blobs\//m;
+ },
+ );
+
+ $is_oci = 1 if $has_oci_layout && $has_index_json && $has_blobs;
+ }
+
+ if ($is_oci) {
+ # Extract the OCI image
+ my ($id_map, undef, undef) = PVE::LXC::parse_id_maps($conf);
+ my $oci_config = PVE::LXC::Namespaces::run_in_userns(
+ sub {
+ PVE::RS::OCI::parse_and_extract_image($archivepath, $rootdir);
+ },
+ $id_map,
+ );
+
+ # Set the entrypoint and arguments if specified by the OCI image
+ my @init_cmd = ();
+ push(@init_cmd, @{ $oci_config->{Entrypoint} })
+ if $oci_config->{Entrypoint};
+ push(@init_cmd, @{ $oci_config->{Cmd} }) if $oci_config->{Cmd};
+ if (@init_cmd) {
+ my $init_cmd_str = shift(@init_cmd);
+ if (@init_cmd) {
+ $init_cmd_str .= ' ';
+ $init_cmd_str .= join(
+ ' ',
+ map {
+ my $s = $_;
+ $s =~ s/"/\\"/g;
+ qq{"$_"}
+ } @init_cmd,
+ );
+ }
+ if ($init_cmd_str ne '/sbin/init') {
+ push @{ $conf->{lxc} }, ['lxc.init.cmd', $init_cmd_str];
+
+ # An entrypoint other than /sbin/init breaks the tty console mode.
+ # This is fixed by setting cmode: console
+ $conf->{cmode} = 'console';
+ }
+ }
+
+ push @{ $conf->{lxc} }, ['lxc.init.cwd', $oci_config->{WorkingDir}]
+ if ($oci_config->{WorkingDir});
+
+ if (my $envs = $oci_config->{Env}) {
+ for my $env (@{$envs}) {
+ push @{ $conf->{lxc} }, ['lxc.environment.runtime', $env];
+ }
+ }
+
+ my $stop_signal = $oci_config->{StopSignal} // "SIGTERM";
+ push @{ $conf->{lxc} }, ['lxc.signal.halt', $stop_signal];
+ } else {
+ # Not an OCI image, so restore it as an LXC image instead
+ PVE::LXC::Create::restore_archive(
+ $storage_cfg,
+ $archive,
+ $rootdir,
+ $conf,
+ $ignore_unpack_errors,
+ $bwlimit,
+ );
+ }
if ($restore) {
print "merging backed-up and given configuration..\n";
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH container v5 07/17] config: add entrypoint parameter
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (5 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 06/17] add support for OCI images as container templates Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 08/17] configure static IP in LXC config for custom entrypoint Filip Schauer
` (9 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
src/PVE/API2/LXC.pm | 2 +-
src/PVE/LXC.pm | 2 ++
src/PVE/LXC/Config.pm | 12 ++++++++++++
3 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 7a44547..546f4ee 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -593,7 +593,7 @@ __PACKAGE__->register_method({
);
}
if ($init_cmd_str ne '/sbin/init') {
- push @{ $conf->{lxc} }, ['lxc.init.cmd', $init_cmd_str];
+ $conf->{entrypoint} = $init_cmd_str;
# An entrypoint other than /sbin/init breaks the tty console mode.
# This is fixed by setting cmode: console
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index a445a85..5eaa57c 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -862,6 +862,8 @@ sub update_lxc_config {
$raw .= "lxc.rootfs.path = $dir/rootfs\n";
+ $raw .= "lxc.init.cmd = $conf->{entrypoint}\n" if defined($conf->{entrypoint});
+
foreach my $k (sort keys %$conf) {
next if $k !~ m/^net(\d+)$/;
my $ind = $1;
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 1038fa7..56cb01c 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -638,6 +638,12 @@ my $confdesc = {
enum => ['shell', 'console', 'tty'],
default => 'tty',
},
+ entrypoint => {
+ optional => 1,
+ type => 'string',
+ description => "Absolute path from container rootfs to the binary to use as init.",
+ default => '/sbin/init',
+ },
protection => {
optional => 1,
type => 'boolean',
@@ -1861,6 +1867,12 @@ sub get_cmode {
return $conf->{cmode} // $confdesc->{cmode}->{default};
}
+sub get_entrypoint {
+ my ($class, $conf) = @_;
+
+ return $conf->{entrypoint} // $confdesc->{entrypoint}->{default};
+}
+
sub valid_volume_keys {
my ($class, $reverse) = @_;
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH container v5 08/17] configure static IP in LXC config for custom entrypoint
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (6 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 07/17] config: add entrypoint parameter Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 09/17] setup: debian: create /etc/network path if missing Filip Schauer
` (8 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
When a container uses the default `/sbin/init` entrypoint, network
interface configuration is usually managed by processes within the
container. However, containers with a different entrypoint might not
have any internal network management process. Consequently, IP addresses
might not be assigned.
This change ensures that a static IP address is explicitly set in the
LXC config for the container.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* add an "ipmanagehost" property to pct.conf to control whether network
interface IP configuration should be handled by the host.
src/PVE/API2/LXC.pm | 4 ++++
src/PVE/LXC.pm | 15 +++++++++++++++
src/PVE/LXC/Config.pm | 14 ++++++++++++++
3 files changed, 33 insertions(+)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 546f4ee..c8aa984 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -598,6 +598,10 @@ __PACKAGE__->register_method({
# An entrypoint other than /sbin/init breaks the tty console mode.
# This is fixed by setting cmode: console
$conf->{cmode} = 'console';
+
+ # Manage the IP configuration for the container. A container with a
+ # custom entrypoint likely lacks internal network management.
+ $conf->{ipmanagehost} = 1;
}
}
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 5eaa57c..6fdef79 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -886,6 +886,21 @@ sub update_lxc_config {
if ($lxc_major >= 4) {
$raw .= "lxc.net.$ind.script.up = /usr/share/lxc/lxcnetaddbr\n";
}
+
+ if ((!defined($d->{link_down}) || $d->{link_down} != 1) && $conf->{ipmanagehost}) {
+ if (defined($d->{ip})) {
+ die "$k: DHCP is not supported with a custom entrypoint\n" if $d->{ip} eq 'dhcp';
+ $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n" if $d->{ip} ne 'manual';
+ }
+ $raw .= "lxc.net.$ind.ipv4.gateway = $d->{gw}\n" if defined($d->{gw});
+ if (defined($d->{ip6})) {
+ die "$k: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
+ if $d->{ip6} =~ /^(auto|dhcp)$/;
+ $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} ne 'manual';
+ }
+ $raw .= "lxc.net.$ind.ipv6.gateway = $d->{gw6}\n" if defined($d->{gw6});
+ $raw .= "lxc.net.$ind.flags = up\n";
+ }
}
my $had_cpuset = 0;
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 56cb01c..afa2fcf 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -594,6 +594,12 @@ my $confdesc = {
. " This is saved as comment inside the configuration file.",
maxLength => 1024 * 8,
},
+ ipmanagehost => {
+ type => 'boolean',
+ description =>
+ "Whether this interface's IP configuration should be managed by the host.",
+ optional => 1,
+ },
searchdomain => {
optional => 1,
type => 'string',
@@ -1288,6 +1294,14 @@ sub update_pct_config {
die "$opt: MTU size '$mtu' is bigger than bridge MTU '$bridge_mtu'\n"
if ($mtu > $bridge_mtu);
}
+
+ if ((!defined($res->{link_down}) || $res->{link_down} != 1) && $conf->{ipmanagehost}) {
+ die "$opt: DHCP is not supported with a custom entrypoint\n"
+ if defined($res->{ip}) && $res->{ip} eq 'dhcp';
+
+ die "$opt: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
+ if defined($res->{ip6}) && $res->{ip6} =~ /^(auto|dhcp)$/;
+ }
} elsif ($opt =~ m/^dev(\d+)$/) {
my $device = $class->parse_device($value);
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH container v5 09/17] setup: debian: create /etc/network path if missing
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (7 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 08/17] configure static IP in LXC config for custom entrypoint Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 10/17] setup: recursively mkdir /etc/systemd/{network, system-preset} Filip Schauer
` (7 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This prevents an error during Debian container setup when the
/etc/network directory is missing. This fixes container creation from
Debian based OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/LXC/Setup/Debian.pm | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/PVE/LXC/Setup/Debian.pm b/src/PVE/LXC/Setup/Debian.pm
index 030d934..fa99ae4 100644
--- a/src/PVE/LXC/Setup/Debian.pm
+++ b/src/PVE/LXC/Setup/Debian.pm
@@ -487,6 +487,7 @@ sub setup_network {
"auto lo\niface lo inet loopback\n" . "iface lo inet6 loopback\n\n" . $interfaces;
}
+ $self->ct_make_path('/etc/network');
$self->ct_file_set_contents($filename, $interfaces);
}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH container v5 10/17] setup: recursively mkdir /etc/systemd/{network, system-preset}
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (8 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 09/17] setup: debian: create /etc/network path if missing Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 11/17] implement host-managed DHCP for containers with `ipmanagehost` Filip Schauer
` (6 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Ensure that both /etc/systemd/network and /etc/systemd/system-preset
exist before writing files into them. This fixes container creation from
the docker.io/fedora & docker.io/ubuntu OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
Introduced in v2
src/PVE/LXC/Setup/Base.pm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/PVE/LXC/Setup/Base.pm b/src/PVE/LXC/Setup/Base.pm
index a2c88ed..4442727 100644
--- a/src/PVE/LXC/Setup/Base.pm
+++ b/src/PVE/LXC/Setup/Base.pm
@@ -319,6 +319,7 @@ DATA
$data .= "IPv6AcceptRA = $accept_ra\n";
$data .= $routes if $routes;
+ $self->ct_make_path('/etc/systemd/network');
$self->ct_file_set_contents($filename, $data);
}
}
@@ -358,7 +359,7 @@ sub setup_systemd_preset {
}
}
- $self->ct_mkdir('/etc/systemd/system-preset', 0755);
+ $self->ct_make_path('/etc/systemd/system-preset');
$self->ct_file_set_contents(
'/etc/systemd/system-preset/00-pve.preset', $preset_data,
);
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH container v5 11/17] implement host-managed DHCP for containers with `ipmanagehost`
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (9 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 10/17] setup: recursively mkdir /etc/systemd/{network, system-preset} Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 12/17] allow .tar container templates Filip Schauer
` (5 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Use `dhclient` on the host to manage DHCP for container network
interfaces when `ipmanagehost` is enabled in the container config.
This is useful for containers that lack an internal network management
stack, like many containers based on OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changes since v3:
* keep lxc pidfd around to ensure the existence of $rootdir until
dhclients are started
* update commit message to reflect changes from v3
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* add an "ipmanagehost" property to pct.conf to control whether network
interface IP configuration should be handled by the host.
* manage_dhclient: add a FIXME comment regarding the AppArmor profile:
"use a profile that confines writes to /var/lib/lxc/$vmid and rootfs"
* kill_dhclients: untaint pid from pidfile
* fix manage_dhclient called with 'stop' instead of 'start' for IPv6
when container is started
src/PVE/LXC.pm | 103 +++++++++++++++++++++++++++++++++++++-----
src/PVE/LXC/Config.pm | 17 ++++---
2 files changed, 101 insertions(+), 19 deletions(-)
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 6fdef79..69bbd24 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -888,15 +888,12 @@ sub update_lxc_config {
}
if ((!defined($d->{link_down}) || $d->{link_down} != 1) && $conf->{ipmanagehost}) {
- if (defined($d->{ip})) {
- die "$k: DHCP is not supported with a custom entrypoint\n" if $d->{ip} eq 'dhcp';
- $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n" if $d->{ip} ne 'manual';
- }
+ $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n"
+ if defined($d->{ip}) && $d->{ip} !~ /^(dhcp|manual)$/;
$raw .= "lxc.net.$ind.ipv4.gateway = $d->{gw}\n" if defined($d->{gw});
if (defined($d->{ip6})) {
- die "$k: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
- if $d->{ip6} =~ /^(auto|dhcp)$/;
- $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} ne 'manual';
+ die "$k: SLAAC is not supported with ipmanagehost\n" if $d->{ip6} eq 'auto';
+ $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} !~ /^(dhcp|manual)$/;
}
$raw .= "lxc.net.$ind.ipv6.gateway = $d->{gw6}\n" if defined($d->{gw6});
$raw .= "lxc.net.$ind.flags = up\n";
@@ -1079,6 +1076,8 @@ sub vm_stop_cleanup {
PVE::Storage::deactivate_volumes($storage_cfg, $vollist);
};
warn $@ if $@; # avoid errors - just warn
+
+ kill_dhclients($vmid, '*') if $conf->{ipmanagehost};
}
sub net_tap_plug : prototype($$) {
@@ -1310,6 +1309,52 @@ sub get_interfaces {
return $res;
}
+sub manage_dhclient {
+ my ($action, $vmid, $ipversion, $eth, $rootdir) = @_;
+
+ File::Path::make_path("/var/lib/lxc/$vmid/hook") if $action eq 'start';
+ my $pidfile = "/var/lib/lxc/$vmid/hook/dhclient$ipversion-$eth.pid";
+ my $leasefile = "/var/lib/lxc/$vmid/hook/dhclient$ipversion-$eth.leases";
+ my $scriptfile = '/usr/share/lxc/hooks/dhclient-script';
+ PVE::Tools::run_command([
+ 'lxc-attach',
+ '-n',
+ $vmid,
+ '-s',
+ 'NETWORK|UTSNAME',
+ '--',
+ 'aa-exec',
+ '-p',
+ 'unconfined', # FIXME: use a profile that confines writes to /var/lib/lxc/$vmid and rootfs
+ '/sbin/dhclient',
+ $action eq 'start' ? '-1' : '-r',
+ "-$ipversion",
+ '-pf',
+ $pidfile,
+ '-lf',
+ $leasefile,
+ '-e',
+ "ROOTFS=$rootdir",
+ '-sf',
+ $scriptfile,
+ $eth,
+ ]);
+}
+
+sub kill_dhclients {
+ my ($vmid, $eth) = @_;
+
+ foreach my $pidfile (glob("/var/lib/lxc/$vmid/hook/dhclient*-$eth.pid")) {
+ my $pid = eval { file_get_contents($pidfile) };
+ if (!$@) {
+ chomp $pid;
+ next if $pid !~ /^(\d+)$/;
+ kill 9, $1;
+ unlink($pidfile);
+ }
+ }
+}
+
sub update_ipconfig {
my ($vmid, $conf, $opt, $eth, $newnet, $rootdir) = @_;
@@ -1346,11 +1391,21 @@ sub update_ipconfig {
# step 1: add new IP, if this fails we cancel
my $is_real_ip = ($newip && $newip !~ /^(?:auto|dhcp|manual)$/);
- if ($change_ip && $is_real_ip) {
- eval { &$ipcmd($family_opt, 'addr', 'add', $newip, 'dev', $eth); };
- if (my $err = $@) {
- warn $err;
- return;
+ if ($change_ip) {
+ if ($conf->{ipmanagehost}) {
+ if ($newip && $newip eq 'dhcp') {
+ manage_dhclient('start', $vmid, $ipversion, $eth, $rootdir);
+ } elsif ($oldip && $oldip eq 'dhcp') {
+ manage_dhclient('stop', $vmid, $ipversion, $eth, $rootdir);
+ }
+ }
+
+ if ($is_real_ip) {
+ eval { &$ipcmd($family_opt, 'addr', 'add', $newip, 'dev', $eth); };
+ if (my $err = $@) {
+ warn $err;
+ return;
+ }
}
}
@@ -3001,6 +3056,30 @@ sub vm_start {
}
PVE::GuestHelpers::exec_hookscript($conf, $vmid, 'post-start');
+ if ($conf->{ipmanagehost}) {
+ my @dhcpv4_interfaces = ();
+ my @dhcpv6_interfaces = ();
+ foreach my $k (sort keys %$conf) {
+ next if $k !~ m/^net(\d+)$/;
+ my $d = PVE::LXC::Config->parse_lxc_network($conf->{$k});
+ push @dhcpv4_interfaces, $d->{name} if $d->{ip} && $d->{ip} eq 'dhcp';
+ push @dhcpv6_interfaces, $d->{name} if $d->{ip6} && $d->{ip6} eq 'dhcp';
+ }
+
+ my ($pid, $pidfd) = PVE::LXC::open_lxc_pid($vmid);
+ my $rootdir = "/proc/$pid/root";
+
+ foreach my $eth (@dhcpv4_interfaces) {
+ eval { manage_dhclient('start', $vmid, 4, $eth, $rootdir) };
+ PVE::RESTEnvironment::log_warn("DHCP failed - $@") if $@;
+ }
+
+ foreach my $eth (@dhcpv6_interfaces) {
+ eval { manage_dhclient('start', $vmid, 6, $eth, $rootdir) };
+ PVE::RESTEnvironment::log_warn("DHCP failed - $@") if $@;
+ }
+ }
+
return;
}
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index afa2fcf..3797ca5 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -1295,12 +1295,13 @@ sub update_pct_config {
if ($mtu > $bridge_mtu);
}
- if ((!defined($res->{link_down}) || $res->{link_down} != 1) && $conf->{ipmanagehost}) {
- die "$opt: DHCP is not supported with a custom entrypoint\n"
- if defined($res->{ip}) && $res->{ip} eq 'dhcp';
-
- die "$opt: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
- if defined($res->{ip6}) && $res->{ip6} =~ /^(auto|dhcp)$/;
+ if (
+ (!defined($res->{link_down}) || $res->{link_down} != 1)
+ && $conf->{ipmanagehost}
+ && defined($res->{ip6})
+ && $res->{ip6} eq 'auto'
+ ) {
+ die "$opt: SLAAC is not supported with ipmanagehost\n";
}
} elsif ($opt =~ m/^dev(\d+)$/) {
my $device = $class->parse_device($value);
@@ -1585,9 +1586,11 @@ sub vmconfig_hotplug_pending {
$cgroup->change_cpu_shares(undef);
} elsif ($opt =~ m/^net(\d)$/) {
my $netid = $1;
+ my $net = parse_lxc_network($conf->{$opt});
+ PVE::LXC::kill_dhclients($vmid, $net->{name}) if $conf->{ipmanagehost};
+
PVE::Network::veth_delete("veth${vmid}i$netid");
if ($have_sdn) {
- my $net = PVE::LXC::Config->parse_lxc_network($conf->{$opt});
print "delete ips from $opt\n";
eval {
PVE::Network::SDN::Vnets::del_ips_from_mac(
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH storage v5 12/17] allow .tar container templates
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (10 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 11/17] implement host-managed DHCP for containers with `ipmanagehost` Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 13/17] api: add storage/{storage}/oci-registry-pull method Filip Schauer
` (4 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This is needed for OCI container images bundled as tar files, as
generated by `docker save`. OCI images do not need additional
compression, since the content is usually compressed already.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* Modify VZTMPL_EXT_RE_1 regex to put "tar" into capture group when
matching on a .tar file.
src/PVE/Storage.pm | 2 +-
src/PVE/Storage/Plugin.pm | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 1dde2b7..0bab945 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -115,7 +115,7 @@ PVE::Storage::Plugin->init();
our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
-our $VZTMPL_EXT_RE_1 = qr/\.tar\.(gz|xz|zst|bz2)/i;
+our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 8acd214..767087b 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1618,7 +1618,7 @@ my $get_subdir_files = sub {
} elsif ($tt eq 'vztmpl') {
next if $fn !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
- $info = { volid => "$sid:vztmpl/$1", format => "t$2" };
+ $info = { volid => "$sid:vztmpl/$1", format => $2 eq 'tar' ? $2 : "t$2" };
} elsif ($tt eq 'backup') {
next if $fn !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH storage v5 13/17] api: add storage/{storage}/oci-registry-pull method
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (11 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 12/17] allow .tar container templates Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 14/17] ui: storage upload: accept *.tar files as vztmpl Filip Schauer
` (3 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add a storage API method to pull an OCI image from a registry using
skopeo.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v5.
debian/control | 1 +
src/PVE/API2/Storage/Status.pm | 70 ++++++++++++++++++++++++++++++++++
2 files changed, 71 insertions(+)
diff --git a/debian/control b/debian/control
index 5341317..6bd55a2 100644
--- a/debian/control
+++ b/debian/control
@@ -54,6 +54,7 @@ Depends: bzip2,
${misc:Depends},
${perl:Depends},
Recommends: pve-esxi-import-tools (>= 0.6.0),
+ skopeo,
zfs-zed,
Description: Proxmox VE storage management library
This package contains the storage management library used by Proxmox VE.
diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 7bde4ec..12b7341 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -265,6 +265,7 @@ __PACKAGE__->register_method({
{ subdir => 'download-url' },
{ subdir => 'file-restore' },
{ subdir => 'import-metadata' },
+ { subdir => 'oci-registry-pull' },
{ subdir => 'prunebackups' },
{ subdir => 'rrd' },
{ subdir => 'rrddata' },
@@ -864,6 +865,75 @@ __PACKAGE__->register_method({
},
});
+__PACKAGE__->register_method({
+ name => 'oci_registry_pull',
+ path => '{storage}/oci-registry-pull',
+ method => 'POST',
+ description => "Pull an OCI image from a registry.",
+ proxyto => 'node',
+ permissions => {
+ check => [
+ 'and',
+ ['perm', '/storage/{storage}', ['Datastore.AllocateTemplate']],
+ ['perm', '/nodes/{node}', ['Sys.AccessNetwork']],
+ ],
+ },
+ protected => 1,
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ storage => get_standard_option('pve-storage-id'),
+ reference => {
+ description => "The reference to the OCI image to download.",
+ type => 'string',
+ pattern =>
+ '^(?:(?:[a-zA-Z\d]|[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d])'
+ . '(?:\.(?:[a-zA-Z\d]|[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d]))*(?::\d+)?/)?[a-z\d]+'
+ . '(?:/[a-z\d]+(?:(?:(?:[._]|__|[-]*)[a-z\d]+)+)?)*:\w[\w.-]{0,127}$',
+ },
+ },
+ },
+ returns => {
+ type => "string",
+ },
+ code => sub {
+ my ($param) = @_;
+
+ die "Install 'skopeo' to pull OCI images from registries.\n" if (!-f '/usr/bin/skopeo');
+
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $user = $rpcenv->get_user();
+
+ my $cfg = PVE::Storage::config();
+
+ my ($node, $storage) = $param->@{qw(node storage)};
+ my $scfg = PVE::Storage::storage_check_enabled($cfg, $storage, $node);
+
+ die "can't upload to storage type '$scfg->{type}', not a file based storage!\n"
+ if !defined($scfg->{path});
+
+ my $reference = $param->{reference};
+
+ die "storage '$storage' is not configured for content-type 'vztmpl'\n"
+ if !$scfg->{content}->{vztmpl};
+
+ my $filename = PVE::Storage::normalize_content_filename($reference);
+ my $path = PVE::Storage::get_vztmpl_dir($cfg, $storage);
+ PVE::Storage::activate_storage($cfg, $storage);
+
+ my $worker = sub {
+ PVE::Tools::run_command(
+ ["skopeo", "copy", "docker://$reference", "oci-archive:$path/$filename.tar"],
+ );
+ };
+
+ my $worker_id = PVE::Tools::encode_text($filename); # must not pass : or the like as w-ID
+
+ return $rpcenv->fork_worker('ociregistrypull', $worker_id, $user, $worker);
+ },
+});
+
__PACKAGE__->register_method({
name => 'get_import_metadata',
path => '{storage}/import-metadata',
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH manager v5 14/17] ui: storage upload: accept *.tar files as vztmpl
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (12 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 13/17] api: add storage/{storage}/oci-registry-pull method Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 15/17] api: add nodes/{node}/query-oci-repo-tags method Filip Schauer
` (2 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This is needed for OCI container images bundled as tar files.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This depends on the change made to pve-storage in patch 12/17.
It might make sense to bump pve-storage and with it bump the dependency
to libpve-storage-perl in debian/control.
Changed since v2:
* rebase onto newest master (84b22751f211) and re-format
Introduced in v2
www/manager6/window/UploadToStorage.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
index cc53596d..df6a9178 100644
--- a/www/manager6/window/UploadToStorage.js
+++ b/www/manager6/window/UploadToStorage.js
@@ -11,7 +11,7 @@ Ext.define('PVE.window.UploadToStorage', {
acceptedExtensions: {
import: ['.ova', '.qcow2', '.raw', '.vmdk'],
iso: ['.img', '.iso'],
- vztmpl: ['.tar.gz', '.tar.xz', '.tar.zst'],
+ vztmpl: ['.tar', '.tar.gz', '.tar.xz', '.tar.zst'],
},
// accepted for file selection, will be renamed to real extension
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH manager v5 15/17] api: add nodes/{node}/query-oci-repo-tags method
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (13 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 14/17] ui: storage upload: accept *.tar files as vztmpl Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 16/17] ui: template view: add OCI registry pull dialog Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH docs v5 17/17] ct: add OCI image docs Filip Schauer
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add an API method for querying the tags of a reference from an OCI
registry.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v5
PVE/API2/Nodes.pm | 47 +++++++++++++++++++++++++++++++++++++++++++++++
debian/control | 1 +
2 files changed, 48 insertions(+)
diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 4590b618..8f86afd9 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -279,6 +279,7 @@ __PACKAGE__->register_method({
{ name => 'netstat' },
{ name => 'network' },
{ name => 'qemu' },
+ { name => 'query-oci-repo-tags' },
{ name => 'query-url-metadata' },
{ name => 'replication' },
{ name => 'report' },
@@ -1761,6 +1762,52 @@ __PACKAGE__->register_method({
},
});
+__PACKAGE__->register_method({
+ name => 'query_oci_repo_tags',
+ path => 'query-oci-repo-tags',
+ method => 'GET',
+ description => "List all tags for an OCI repository reference.",
+ proxyto => 'node',
+ permissions => {
+ check => ['perm', '/nodes/{node}', ['Sys.AccessNetwork']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ reference => {
+ description => "The reference to the repository to query tags from.",
+ type => 'string',
+ pattern => '^(?:(?:[a-zA-Z\d]|[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d])'
+ . '(?:\.(?:[a-zA-Z\d]|[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d]))*(?::\d+)?/)?[a-z\d]+'
+ . '(?:(?:[._]|__|[-]*)[a-z\d]+)*(?:/[a-z\d]+(?:(?:[._]|__|[-]*)[a-z\d]+)*)*$',
+ },
+ },
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'string',
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+
+ die "Install 'skopeo' to list tags from OCI registries.\n" if (!-f '/usr/bin/skopeo');
+
+ my $reference = $param->{reference};
+ my $tags_json = "";
+ PVE::Tools::run_command(
+ ["skopeo", "list-tags", "docker://$reference"],
+ outfunc => sub {
+ $tags_json = $tags_json . shift;
+ },
+ );
+ my $tags = decode_json($tags_json);
+ return $tags->{Tags};
+ },
+});
+
__PACKAGE__->register_method({
name => 'query_url_metadata',
path => 'query-url-metadata',
diff --git a/debian/control b/debian/control
index 64b942ce..afd0386a 100644
--- a/debian/control
+++ b/debian/control
@@ -105,6 +105,7 @@ Depends: apt (>= 1.5~),
Recommends: proxmox-firewall,
proxmox-offline-mirror-helper,
pve-nvidia-vgpu-helper,
+ skopeo,
Conflicts: vlan, vzdump,
Replaces: vlan, vzdump,
Provides: vlan, vzdump,
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH manager v5 16/17] ui: template view: add OCI registry pull dialog
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (14 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 15/17] api: add nodes/{node}/query-oci-repo-tags method Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH docs v5 17/17] ct: add OCI image docs Filip Schauer
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add a button and a dialog for pulling images from an OCI registry.
Behind the scenes this uses the nodes/{node}/query-oci-repo-tags and
storage/{storage}/oci-registry-pull API methods.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v5
www/manager6/storage/TemplateView.js | 139 ++++++++++++++++++++++++++-
1 file changed, 138 insertions(+), 1 deletion(-)
diff --git a/www/manager6/storage/TemplateView.js b/www/manager6/storage/TemplateView.js
index 8a775bbd..de98d788 100644
--- a/www/manager6/storage/TemplateView.js
+++ b/www/manager6/storage/TemplateView.js
@@ -176,6 +176,130 @@ Ext.define('PVE.storage.TemplateDownload', {
},
});
+Ext.define('PVE.storage.OciRegistryPull', {
+ extend: 'Proxmox.window.Edit',
+ alias: 'widget.pveOciRegistryPull',
+ mixins: ['Proxmox.Mixin.CBind'],
+
+ method: 'POST',
+
+ showTaskViewer: true,
+
+ title: gettext('Pull from OCI Registry'),
+ submitText: gettext('Download'),
+ width: 450,
+
+ cbind: {
+ url: '/nodes/{nodename}/storage/{storage}/oci-registry-pull',
+ },
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ onReferenceChange: function (field, value) {
+ let me = this;
+ let view = me.getView();
+ let tagField = view.down('[name=tag]');
+ tagField.setComboItems([]);
+
+ let parts = value.split(':');
+ if (parts.length > 1) {
+ field.setValue(parts[0]);
+ tagField.setValue(parts[1]);
+ tagField.focus();
+ } else {
+ tagField.clearValue();
+ }
+ },
+
+ queryTags: function (field) {
+ let me = this;
+ let view = me.getView();
+ let refField = view.down('[name=reference]');
+ let reference = refField.value;
+ let tagField = view.down('[name=tag]');
+
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${view.nodename}/query-oci-repo-tags`,
+ method: 'GET',
+ params: {
+ reference,
+ },
+ waitMsgTarget: view,
+ failure: (res) => {
+ Ext.MessageBox.alert(gettext('Error'), res.htmlStatus);
+ },
+ success: function (res, opt) {
+ let tags = res.result.data;
+ tagField.clearValue();
+ tagField.setComboItems(tags.map((tag) => [tag, tag]));
+ },
+ });
+ },
+ },
+
+ items: [
+ {
+ xtype: 'inputpanel',
+ border: false,
+ onGetValues: function (values) {
+ return {
+ reference: values.reference + ':' + values.tag,
+ };
+ },
+ items: [
+ {
+ xtype: 'fieldcontainer',
+ layout: 'hbox',
+ fieldLabel: gettext('Reference'),
+ items: [
+ {
+ xtype: 'textfield',
+ name: 'reference',
+ allowBlank: false,
+ emptyText: 'registry.example.org/name',
+ flex: 1,
+ listeners: {
+ change: 'onReferenceChange',
+ },
+ },
+ {
+ xtype: 'button',
+ name: 'check',
+ text: gettext('Query Tags'),
+ margin: '0 0 0 5',
+ listeners: {
+ click: 'queryTags',
+ },
+ },
+ ],
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'tag',
+ allowBlank: false,
+ emptyText: 'latest',
+ fieldLabel: gettext('Tag'),
+ forceSelection: false,
+ editable: true,
+ typeAhead: true,
+ comboItems: [],
+ },
+ ],
+ },
+ ],
+
+ initComponent: function () {
+ var me = this;
+
+ if (!me.nodename) {
+ throw 'no node name specified';
+ }
+
+ me.callParent();
+ },
+});
+
Ext.define('PVE.storage.TemplateView', {
extend: 'PVE.storage.ContentView',
@@ -213,7 +337,20 @@ Ext.define('PVE.storage.TemplateView', {
},
});
- me.tbar = [templateButton];
+ var pullOciImageButton = Ext.create('Proxmox.button.Button', {
+ itemId: 'pull-oci-img-btn',
+ text: gettext('Pull from OCI Registry'),
+ handler: function () {
+ var win = Ext.create('PVE.storage.OciRegistryPull', {
+ nodename: nodename,
+ storage: storage,
+ taskDone: () => reload(),
+ });
+ win.show();
+ },
+ });
+
+ me.tbar = [templateButton, pullOciImageButton];
me.useUploadButton = true;
me.callParent();
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread* [pve-devel] [PATCH docs v5 17/17] ct: add OCI image docs
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (15 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 16/17] ui: template view: add OCI registry pull dialog Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v4:
* document the "Pull from OCI registry" feature
Introduced in v3
pct.adoc | 44 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 36 insertions(+), 8 deletions(-)
diff --git a/pct.adoc b/pct.adoc
index d6146eb..8212bc9 100644
--- a/pct.adoc
+++ b/pct.adoc
@@ -54,15 +54,22 @@ the cluster setup, and they can use the same network and storage resources as
virtual machines. You can also use the {pve} firewall, or manage containers
using the HA framework.
-Our primary goal is to offer an environment that provides the benefits of using a
-VM, but without the additional overhead. This means that Proxmox Containers can
-be categorized as ``System Containers'', rather than ``Application Containers''.
+Our primary goal has traditionally been to offer an environment that provides
+the benefits of using a VM, but without the additional overhead. This means that
+Proxmox Containers have been primarily categorized as ``System Containers''.
-NOTE: If you want to run application containers, for example, 'Docker' images, it
-is recommended that you run them inside a Proxmox QEMU VM. This will give you
-all the advantages of application containerization, while also providing the
-benefits that VMs offer, such as strong isolation from the host and the ability
-to live-migrate, which otherwise isn't possible with containers.
+With the introduction of OCI (**O**pen **C**ontainer **I**nitiative) image support,
+Proxmox VE now also integrates ``Application Containers''. When creating a
+container from an OCI image, the image is automatically converted to the
+LXC stack that Proxmox VE uses.
+
+This approach allows users to benefit from a wide ecosystem of pre-packaged
+applications while retaining the robust management features of Proxmox VE.
+
+While running lightweight ``Application Containers'' directly offers significant
+advantages over a full VM, for use cases demanding maximum isolation and
+the ability to live-migrate, nesting containers inside a Proxmox QEMU VM remains
+a recommended practice.
Technology Overview
@@ -256,6 +263,12 @@ Container Images
Container images, sometimes also referred to as ``templates'' or
``appliances'', are `tar` archives which contain everything to run a container.
+Proxmox VE can utilize two main types of images: *System Container Templates*
+for creating full virtual environments, and *Application Container Images* based
+on the OCI standard for running specific applications.
+
+System Container Templates
+~~~~~~~~~~~~~~~~~~~~~~~~~~
{pve} itself provides a variety of basic templates for the
xref:pct_supported_distributions[most common Linux distributions]. They can be
@@ -336,6 +349,21 @@ delete that image later with:
# pveam remove local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz
----
+Open Container Initiative (OCI) Images (technology preview)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Proxmox VE can also use OCI images to create containers. This makes it easy to
+run pre-packaged applications. A container created from an OCI image still uses
+the existing LXC framework.
+
+Obtaining OCI Images
+~~~~~~~~~~~~~~~~~~~~
+
+In the web interface an OCI image can be pulled from a registry using the *Pull
+from OCI registry* button on the container template view of a storage.
+
+Once the template is on a storage, you can create the container with
+`pct create` or use the wizard in the web interface.
[[pct_settings]]
Container Settings
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread