* [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates
@ 2025-10-08 17:10 Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 01/17] io: introduce RangeReader for bounded reads Filip Schauer
` (16 more replies)
0 siblings, 17 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add basic support for OCI (Open Container Initiative) images [0] as
container templates.
In the web UI on a storage supporting container templates, an OCI image
can be obtained from a registry using the "Pull from OCI Registry"
button. This requires `skopeo` to be installed.
Alternatively OCI images can be pulled using Docker:
```
$ docker pull httpd
$ docker save httpd > httpd.tar
```
Or using Podman:
```
$ podman pull httpd
$ podman save --format=oci-archive httpd > httpd.tar
```
The OCI image tar archive can then be used as a template during
container creation. It is automatically detected as an OCI image. The
resulting container still uses the existing LXC framework.
# Future plans
Currently all layers of the OCI image are squashed into one rootfs. In
the future a new content type could be introduced for individual
overlayfs layers. These read-only layers (+ one writable layer for each
container) could then be referred to in the container config's rootfs.
This could save storage space by deduplicating common layers between
different containers and it would also allow for containers to be
updated with new versions of OCI images by simply swapping out the
underlying layers.
# Dependencies:
To be able to build `proxmox-oci`, the `oci-spec` crate is required as a
dependency. A patch from Christoph [1] packages the `oci-spec` crate as
a deb package. Alternatively if the `oci-spec` crate is not yet
packaged, it can be downloaded from crates.io.
Here is a little script to download the `oci-spec` crate along with its
dependencies:
```sh
download_crate() {
CRATE_NAME=$1
CRATE_VERSION=$2
CRATE_SHA256=$3
wget https://crates.io/api/v1/crates/$CRATE_NAME/$CRATE_VERSION/download
COMPUTED_SHA256=$(sha256sum download | awk '{ print $1 }')
if [ "$COMPUTED_SHA256" != "$CRATE_SHA256" ]; then
echo "Checksum mismatch"; exit 1
fi
tar -xf download
rm download
mv $CRATE_NAME-$CRATE_VERSION /usr/share/cargo/registry/
echo "{\"package\":\"$CRATE_SHA256\",\"files\":{}}" > /usr/share/cargo/registry/$CRATE_NAME-$CRATE_VERSION/.cargo-checksum.json
}
download_crate strsim 0.11.1 7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f
download_crate ident_case 1.0.1 b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39
download_crate darling_macro 0.20.11 fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead
download_crate darling_core 0.20.11 0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e
download_crate darling 0.20.11 fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee
download_crate proc-macro-error-attr2 2.0.0 96de42df36bb9bba5542fe9f1a054b8cc87e172759a1868aa05c1f3acc89dfc5
download_crate derive_builder_core 0.20.2 2d5bcf7b024d6835cfb3d473887cd966994907effbe9227e8c8219824d06c4e8
download_crate thiserror-impl 2.0.0 22efd00f33f93fa62848a7cab956c3d38c8d43095efda1decfc2b3a5dc0b8972
download_crate rustversion 1.0.20 eded382c5f5f786b989652c49544c4877d9f015cc22e145a5ea8ea66c2921cd2
download_crate heck 0.5.0 2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea
download_crate proc-macro-error2 2.0.1 11ec05c52be0a07b08061f7dd003e7d7092e0472bc731b4af7bb1ef876109802
download_crate derive_builder_macro 0.20.2 ab63b0e2bf4d5928aff72e83a7dace85d7bba5fe12dcc3c5a572d78caffd3f3c
download_crate thiserror 2.0.0 15291287e9bff1bc6f9ff3409ed9af665bec7a5fc8ac079ea96be07bca0e2668
download_crate strum_macros 0.27.1 c77a8c5abcaf0f9ce05d62342b7d298c346515365c36b673df4ebe3ced01fde8
download_crate strum 0.27.1 f64def088c51c9510a8579e3c5d67c65349dcf755e5479ad3d010aa6454e2c32
download_crate getset 0.1.5 f3586f256131df87204eb733da72e3d3eb4f343c639f4b7be279ac7c48baeafe
download_crate derive_builder 0.20.2 507dfb09ea8b7fa618fcf76e953f4f5e192547945816d5358edffe39f6f94947
download_crate oci-spec 0.8.3 2eb4684653aeaba48dea019caa17b2773e1212e281d50b6fa759f36fe032239d
```
Since librust-oci-spec-dev is in the proxmox-oci/debian/control file, a
dummy package needs to be installed, so dpkg-checkbuilddeps does not
complain.
dummy_librust_oci_spec.equivs:
```
Package: librust-oci-spec-dev
Version: 0.8.3
Provides: librust-oci-spec-0.8+default-dev (= 0.8.3-1)
```
```
$ equivs-build dummy_librust_oci_spec.equivs
$ dpkg -i ./librust-oci-spec-dev_0.8.3_all.deb
```
# Build & install order:
OCI image support:
1. proxmox
2. proxmox-perl-rs
3. pve-container
.tar container template and registry pull support:
1. pve-storage
2. pve-manager
In no particular order:
* pve-docs
[0] https://github.com/opencontainers/image-spec/blob/main/spec.md
[1] https://lore.proxmox.com/pve-devel/20250606103719.533030-2-c.heiss@proxmox.com/
Changed since v4:
* proxmox: oci: update oci-spec dependency to 0.8.3
* add pull from OCI registry feature using `skopeo`
Changed since v3:
* proxmox: add rustdoc comments
* proxmox: add unit tests
* proxmox: correctly handle OCI image whiteout edge cases
* proxmox/proxmox-perl-rs: parse_and_extract_image: add argument for
CPU architecture
* proxmox-perl-rs: improve rustdoc comments
* lxc: conf: split `lxc.environment` into `runtime` and `hooks`
* pve-container: pass environment variables to container via
`lxc.container.runtime`
Changed since v2:
* lxcfs: lxc.mount.hook: override env variables from container config
* pve-container: rebase onto newest master (5a8b3f962f16) and re-format
with proxmox-perltidy
* pve-container: check whether archive is an OCI image before trying to
parse it as one
* pve-container: add an "ipmanagehost" property to pct.conf to indicate
that network interface IP configuration should be handled by the host.
* pve-container: manage_dhclient: add a FIXME comment regarding the
AppArmor profile: "use a profile that confines writes to
/var/lib/lxc/$vmid and rootfs"
* pve-container: kill_dhclients: untaint pid from pidfile
* pve-container: fix manage_dhclient called with 'stop' instead of
'start' for IPv6 when container is started
* pve-docs: add OCI image docs
* proxmox-perl-rs: rebase onto newest master (3809f1229602)
* proxmox-perl-rs: forward all errors to Perl
* proxmox-perl-rs: remove oci-spec dependency
* pve-manager: rebase onto newest master (84b22751f211) and re-format
* proxmox: io: introduce RangeReader for bounded reads
* proxmox: oci: remove reachable unwraps & refactor code
* proxmox: oci: increase hasher buffer size from 4096 to 32768 (matching
internal sha2::Digest buffering)
* proxmox: oci: preserve permissions and xattrs during rootfs extraction
* proxmox: oci: handle whiteouts & opaque whiteouts
* pve-storage: Modify VZTMPL_EXT_RE_1 regex to put "tar" into capture
group when matching on a .tar file.
Changed since v1:
* Fix entrypoint command missing Cmd
* Set lxc.signal.halt according to StopSignal (Fixes container shutdown)
* setup: Ensure that both /etc/systemd/network and
/etc/systemd/system-preset exist before writing files into them.
* ui: storage upload: accept *.tar files as vztmpl
* proxmox-perl-rs: rebase on latest master (3d9806cb3c7f)
* proxmox-perl-rs: add new dependencies to debian/control
* proxmox-oci: refactor errors and use `thiserror` to avoid boilerplate
proxmox:
Filip Schauer (3):
io: introduce RangeReader for bounded reads
add proxmox-oci crate
proxmox-oci: add tests for whiteout handling
Cargo.toml | 1 +
proxmox-io/src/lib.rs | 3 +
proxmox-io/src/range_reader.rs | 175 ++++++++++
proxmox-oci/Cargo.toml | 25 ++
proxmox-oci/debian/changelog | 5 +
proxmox-oci/debian/control | 45 +++
proxmox-oci/debian/debcargo.toml | 7 +
proxmox-oci/src/lib.rs | 324 ++++++++++++++++++
proxmox-oci/src/oci_tar_image.rs | 144 ++++++++
proxmox-oci/tests/extract_whiteouts.rs | 92 +++++
.../oci_test_whiteout_current_directory.tar | Bin 0 -> 8704 bytes
.../oci_test_whiteout_dead_symlink_parent.tar | Bin 0 -> 8704 bytes
.../oci_test_whiteout_root_breakout.tar | Bin 0 -> 7168 bytes
...oci_test_whiteout_root_parent_breakout.tar | Bin 0 -> 7168 bytes
.../oci_test_whiteout_symlink.tar | Bin 0 -> 8704 bytes
15 files changed, 821 insertions(+)
create mode 100644 proxmox-io/src/range_reader.rs
create mode 100644 proxmox-oci/Cargo.toml
create mode 100644 proxmox-oci/debian/changelog
create mode 100644 proxmox-oci/debian/control
create mode 100644 proxmox-oci/debian/debcargo.toml
create mode 100644 proxmox-oci/src/lib.rs
create mode 100644 proxmox-oci/src/oci_tar_image.rs
create mode 100644 proxmox-oci/tests/extract_whiteouts.rs
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar
proxmox-perl-rs:
Filip Schauer (1):
add Perl mapping for OCI container image parser/extractor
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 1 +
pve-rs/debian/control | 1 +
pve-rs/src/bindings/mod.rs | 3 +++
pve-rs/src/bindings/oci.rs | 29 +++++++++++++++++++++++++++++
5 files changed, 35 insertions(+)
create mode 100644 pve-rs/src/bindings/oci.rs
pve-container:
Filip Schauer (7):
config: add `lxc.environment.runtime`/`hooks`
add support for OCI images as container templates
config: add entrypoint parameter
configure static IP in LXC config for custom entrypoint
setup: debian: create /etc/network path if missing
setup: recursively mkdir /etc/systemd/{network,system-preset}
implement host-managed DHCP for containers with `ipmanagehost`
src/PVE/API2/LXC.pm | 98 +++++++++++++++++++++++++++++----
src/PVE/LXC.pm | 106 ++++++++++++++++++++++++++++++++++--
src/PVE/LXC/Config.pm | 33 ++++++++++-
src/PVE/LXC/Setup/Base.pm | 3 +-
src/PVE/LXC/Setup/Debian.pm | 1 +
5 files changed, 224 insertions(+), 17 deletions(-)
pve-storage:
Filip Schauer (2):
allow .tar container templates
api: add storage/{storage}/oci-registry-pull method
debian/control | 1 +
src/PVE/API2/Storage/Status.pm | 70 ++++++++++++++++++++++++++++++++++
src/PVE/Storage.pm | 2 +-
src/PVE/Storage/Plugin.pm | 2 +-
4 files changed, 73 insertions(+), 2 deletions(-)
pve-manager:
Filip Schauer (3):
ui: storage upload: accept *.tar files as vztmpl
api: add nodes/{node}/query-oci-repo-tags method
ui: template view: add OCI registry pull dialog
PVE/API2/Nodes.pm | 47 +++++++++
debian/control | 1 +
www/manager6/storage/TemplateView.js | 139 ++++++++++++++++++++++++-
www/manager6/window/UploadToStorage.js | 2 +-
4 files changed, 187 insertions(+), 2 deletions(-)
pve-docs:
Filip Schauer (1):
ct: add OCI image docs
pct.adoc | 44 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 36 insertions(+), 8 deletions(-)
Summary over all repositories:
34 files changed, 1376 insertions(+), 29 deletions(-)
--
Generated by git-murpp 0.6.0
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH proxmox v5 01/17] io: introduce RangeReader for bounded reads
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 02/17] add proxmox-oci crate Filip Schauer
` (15 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Introduce a reader that exposes a sub-range of an underlying reader.
This will be used for reading individual files out of a tar archive when
parsing an OCI image.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v3:
* add a commit message
* add rustdoc comments
* add unit tests
Introduced in v3
proxmox-io/src/lib.rs | 3 +
proxmox-io/src/range_reader.rs | 175 +++++++++++++++++++++++++++++++++
2 files changed, 178 insertions(+)
create mode 100644 proxmox-io/src/range_reader.rs
diff --git a/proxmox-io/src/lib.rs b/proxmox-io/src/lib.rs
index 1be005ff..a05b9232 100644
--- a/proxmox-io/src/lib.rs
+++ b/proxmox-io/src/lib.rs
@@ -6,6 +6,9 @@
#![deny(unsafe_op_in_unsafe_fn)]
#![cfg_attr(docsrs, feature(doc_cfg, doc_auto_cfg))]
+mod range_reader;
+pub use range_reader::RangeReader;
+
mod read;
pub use read::ReadExt;
diff --git a/proxmox-io/src/range_reader.rs b/proxmox-io/src/range_reader.rs
new file mode 100644
index 00000000..3f4c54fe
--- /dev/null
+++ b/proxmox-io/src/range_reader.rs
@@ -0,0 +1,175 @@
+use std::io::{Read, Seek, SeekFrom};
+use std::ops::Range;
+
+/// A reader that only exposes a sub-range of an underlying `Read + Seek`.
+///
+/// # Examples
+///
+/// ```
+/// # use proxmox_io::RangeReader;
+/// # use std::io::{Cursor, Read, Seek, SeekFrom};
+/// # fn func() -> Result<(), std::io::Error> {
+/// let reader = Cursor::new("Lorem ipsum dolor sit amet");
+///
+/// let mut range_reader = RangeReader::new(reader, 6..17);
+///
+/// // Read all bytes in the range
+/// let mut buf = Vec::new();
+/// range_reader.read_to_end(&mut buf)?;
+/// assert_eq!(buf, "ipsum dolor".as_bytes());
+///
+/// // Seek back to start of the range and read one byte
+/// range_reader.seek(SeekFrom::Start(0))?;
+/// let mut b = [0u8; 1];
+/// range_reader.read_exact(&mut b)?;
+/// assert_eq!(b, "i".as_bytes());
+///
+/// # Ok(())
+/// # }
+/// # func().unwrap();
+/// ```
+pub struct RangeReader<R: Read + Seek> {
+ /// Underlying reader
+ reader: R,
+
+ /// Range inside `R`
+ range: Range<u64>,
+
+ /// Relative position inside `range`
+ position: u64,
+
+ /// True once the initial seek has been performed
+ has_seeked: bool,
+}
+
+impl<R: Read + Seek> RangeReader<R> {
+ pub fn new(reader: R, range: Range<u64>) -> Self {
+ Self {
+ reader,
+ range,
+ position: 0,
+ has_seeked: false,
+ }
+ }
+
+ pub fn into_inner(self) -> R {
+ self.reader
+ }
+
+ pub fn size(&self) -> usize {
+ (self.range.end - self.range.start) as usize
+ }
+
+ pub fn remaining(&self) -> usize {
+ self.size() - self.position as usize
+ }
+}
+
+impl<R: Read + Seek> Read for RangeReader<R> {
+ fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
+ let max_read = buf.len().min(self.remaining());
+ let limited_buf = &mut buf[..max_read];
+
+ if !self.has_seeked {
+ self.reader
+ .seek(SeekFrom::Start(self.range.start + self.position))?;
+ self.has_seeked = true;
+ }
+
+ let bytes_read = self.reader.read(limited_buf)?;
+ self.position += bytes_read.min(max_read) as u64;
+
+ Ok(bytes_read)
+ }
+}
+
+impl<R: Read + Seek> Seek for RangeReader<R> {
+ fn seek(&mut self, pos: SeekFrom) -> std::io::Result<u64> {
+ self.position = match pos {
+ SeekFrom::Start(position) => position.min(self.size() as u64),
+ SeekFrom::End(offset) => {
+ if offset > self.size() as i64 {
+ return Err(std::io::Error::new(
+ std::io::ErrorKind::InvalidInput,
+ "Tried to seek before the beginning of the file",
+ ));
+ }
+
+ (if offset <= 0 {
+ self.size()
+ } else {
+ self.size() - offset as usize
+ }) as u64
+ }
+ SeekFrom::Current(offset) => {
+ if let Some(position) = self.position.checked_add_signed(offset) {
+ position.min(self.size() as u64)
+ } else {
+ return Err(std::io::Error::new(
+ std::io::ErrorKind::InvalidInput,
+ "Tried to seek before the beginning of the file",
+ ));
+ }
+ }
+ };
+
+ self.reader
+ .seek(SeekFrom::Start(self.range.start + self.position))?;
+ self.has_seeked = true;
+
+ Ok(self.position)
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::RangeReader;
+ use std::io::{Cursor, Read, Seek, SeekFrom};
+
+ #[test]
+ fn test_read_full_range() {
+ let reader = Cursor::new("Hello world!");
+ let mut range_reader = RangeReader::new(reader, 6..11);
+
+ let mut buf = Vec::new();
+ let len = range_reader.read_to_end(&mut buf).unwrap();
+
+ assert_eq!(len, 5);
+ assert_eq!(buf, "world".as_bytes());
+ }
+
+ #[test]
+ fn test_read_partial() {
+ let reader = Cursor::new("Hello world!");
+ let mut range_reader = RangeReader::new(reader, 0..5);
+
+ let mut buf = [0u8; 4];
+ range_reader.read_exact(&mut buf).unwrap();
+
+ assert_eq!(buf, "Hell".as_bytes());
+ }
+
+ #[test]
+ fn test_seek_and_read() {
+ let reader = Cursor::new("Lorem ipsum dolor sit amet");
+ let mut range_reader = RangeReader::new(reader, 6..21);
+
+ assert_eq!(range_reader.seek(SeekFrom::Start(6)).unwrap(), 6);
+ let mut buf = [0u8; 5];
+ range_reader.read_exact(&mut buf).unwrap();
+
+ assert_eq!(buf, "dolor".as_bytes());
+ }
+
+ #[test]
+ fn test_seek_out_of_range() {
+ let reader = Cursor::new("Lorem ipsum dolor sit amet");
+ let mut range_reader = RangeReader::new(reader, 6..21);
+
+ let err = range_reader.seek(SeekFrom::Current(-3)).unwrap_err();
+ assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput);
+
+ let err = range_reader.seek(SeekFrom::End(20)).unwrap_err();
+ assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput);
+ }
+}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH proxmox v5 02/17] add proxmox-oci crate
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 01/17] io: introduce RangeReader for bounded reads Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 03/17] proxmox-oci: add tests for whiteout handling Filip Schauer
` (14 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This crate can parse an OCI image tarball and extract its rootfs. Layers
are applied in sequence, but an overlay filesystem is currently not
used.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v4:
* update oci-spec dependency to 0.8.3
Changed since v3:
* correctly handle whiteout edge cases
* parse_and_extract_image: add argument for CPU architecture
Changed since v2:
* remove reachable unwraps & refactor code
* increase hasher buffer size from 4096 to 32768 (matching internal
sha2::Digest buffering)
* preserve permissions and xattrs during rootfs extraction
* handle whiteouts & opaque whiteouts
Cargo.toml | 1 +
proxmox-oci/Cargo.toml | 22 +++
proxmox-oci/debian/changelog | 5 +
proxmox-oci/debian/control | 45 +++++
proxmox-oci/debian/debcargo.toml | 7 +
proxmox-oci/src/lib.rs | 324 +++++++++++++++++++++++++++++++
proxmox-oci/src/oci_tar_image.rs | 144 ++++++++++++++
7 files changed, 548 insertions(+)
create mode 100644 proxmox-oci/Cargo.toml
create mode 100644 proxmox-oci/debian/changelog
create mode 100644 proxmox-oci/debian/control
create mode 100644 proxmox-oci/debian/debcargo.toml
create mode 100644 proxmox-oci/src/lib.rs
create mode 100644 proxmox-oci/src/oci_tar_image.rs
diff --git a/Cargo.toml b/Cargo.toml
index ce249371..ff12d7b7 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -27,6 +27,7 @@ members = [
"proxmox-network-api",
"proxmox-network-types",
"proxmox-notify",
+ "proxmox-oci",
"proxmox-openid",
"proxmox-product-config",
"proxmox-resource-scheduling",
diff --git a/proxmox-oci/Cargo.toml b/proxmox-oci/Cargo.toml
new file mode 100644
index 00000000..6a87c0bd
--- /dev/null
+++ b/proxmox-oci/Cargo.toml
@@ -0,0 +1,22 @@
+[package]
+name = "proxmox-oci"
+description = "OCI image parsing and extraction"
+version = "0.1.0"
+
+authors.workspace = true
+edition.workspace = true
+exclude.workspace = true
+homepage.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version.workspace = true
+
+[dependencies]
+flate2.workspace = true
+oci-spec = "0.8.3"
+sha2 = "0.10"
+tar.workspace = true
+thiserror = "1"
+zstd.workspace = true
+
+proxmox-io.workspace = true
diff --git a/proxmox-oci/debian/changelog b/proxmox-oci/debian/changelog
new file mode 100644
index 00000000..754d06c1
--- /dev/null
+++ b/proxmox-oci/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-oci (0.1.0-1) bookworm; urgency=medium
+
+ * Initial release.
+
+ -- Proxmox Support Team <support@proxmox.com> Mon, 28 Apr 2025 12:34:56 +0200
diff --git a/proxmox-oci/debian/control b/proxmox-oci/debian/control
new file mode 100644
index 00000000..5f116817
--- /dev/null
+++ b/proxmox-oci/debian/control
@@ -0,0 +1,45 @@
+Source: rust-proxmox-oci
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.82) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-flate2-1+default-dev <!nocheck>,
+ librust-oci-spec-0.8+default-dev (>= 0.8.3-~~) <!nocheck>,
+ librust-proxmox-io-1+default-dev (>= 1.2.0-~~) <!nocheck>,
+ librust-sha2-0.10+default-dev <!nocheck>,
+ librust-tar-0.4+default-dev <!nocheck>,
+ librust-thiserror-1+default-dev <!nocheck>,
+ librust-zstd-0.13+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.0
+Vcs-Git: git://git.proxmox.com/git/proxmox.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: proxmox-oci
+Rules-Requires-Root: no
+
+Package: librust-proxmox-oci-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-flate2-1+default-dev,
+ librust-oci-spec-0.8+default-dev (>= 0.8.3-~~),
+ librust-proxmox-io-1+default-dev (>= 1.2.0-~~),
+ librust-sha2-0.10+default-dev,
+ librust-tar-0.4+default-dev,
+ librust-thiserror-1+default-dev,
+ librust-zstd-0.13+default-dev
+Provides:
+ librust-proxmox-oci+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0-dev (= ${binary:Version}),
+ librust-proxmox-oci-0+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1.0-dev (= ${binary:Version}),
+ librust-proxmox-oci-0.1.0+default-dev (= ${binary:Version})
+Description: OCI image parsing and extraction - Rust source code
+ Source code for Debianized Rust crate "proxmox-oci"
diff --git a/proxmox-oci/debian/debcargo.toml b/proxmox-oci/debian/debcargo.toml
new file mode 100644
index 00000000..b7864cdb
--- /dev/null
+++ b/proxmox-oci/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox.git"
diff --git a/proxmox-oci/src/lib.rs b/proxmox-oci/src/lib.rs
new file mode 100644
index 00000000..cf0e4271
--- /dev/null
+++ b/proxmox-oci/src/lib.rs
@@ -0,0 +1,324 @@
+use std::collections::HashMap;
+use std::fs::{read_dir, remove_dir_all, remove_file, File};
+use std::io::{Read, Seek, SeekFrom};
+use std::path::{Path, PathBuf};
+use std::str::FromStr;
+
+use flate2::read::GzDecoder;
+pub use oci_spec::image::{Arch, Config};
+use oci_spec::image::{ImageConfiguration, ImageManifest, MediaType};
+use oci_spec::OciSpecError;
+use sha2::digest::generic_array::GenericArray;
+use sha2::{Digest, Sha256};
+use tar::{Archive, EntryType};
+use thiserror::Error;
+
+mod oci_tar_image;
+use oci_tar_image::{OciTarImage, OciTarImageBlob};
+
+const WHITEOUT_PREFIX: &str = ".wh.";
+const OPAQUE_WHITEOUT_NAME: &str = ".wh..wh..opq";
+
+fn compute_digest<R: Read, H: Digest>(
+ mut reader: R,
+ mut hasher: H,
+) -> std::io::Result<GenericArray<u8, H::OutputSize>> {
+ let mut buf = proxmox_io::boxed::zeroed(32768);
+
+ loop {
+ let bytes_read = reader.read(&mut buf)?;
+ if bytes_read == 0 {
+ break Ok(hasher.finalize());
+ }
+
+ hasher.update(&buf[..bytes_read]);
+ }
+}
+
+fn compute_sha256<R: Read>(reader: R) -> std::io::Result<oci_spec::image::Sha256Digest> {
+ let digest = compute_digest(reader, Sha256::new())?;
+ Ok(oci_spec::image::Sha256Digest::from_str(&format!("{digest:x}")).expect("valid digest"))
+}
+
+/// Build a mapping from uncompressed layer digests (as found in the image config's `rootfs.diff_ids`)
+/// to their corresponding compressed-layer digests (i.e. the filenames under `blobs/<algorithm>/<digest>`)
+fn build_layer_map<R: Read + Seek>(
+ mut oci_tar_image: OciTarImage<R>,
+ image_manifest: &ImageManifest,
+) -> Result<
+ (
+ OciTarImage<R>,
+ HashMap<oci_spec::image::Digest, oci_spec::image::Descriptor>,
+ ),
+ ExtractError,
+> {
+ let mut layer_mapping = HashMap::new();
+
+ for layer in image_manifest.layers() {
+ let digest = match layer.media_type() {
+ MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => layer.digest().clone(),
+ MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
+ let mut compressed_blob = oci_tar_image
+ .open_blob(layer.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer.digest().clone()))?;
+ let decoder = GzDecoder::new(&mut compressed_blob);
+ let hash = compute_sha256(decoder)?.into();
+ oci_tar_image = compressed_blob.into_oci_tar_image();
+ hash
+ }
+ MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
+ let mut compressed_blob = oci_tar_image
+ .open_blob(layer.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer.digest().clone()))?;
+ let decoder = zstd::Decoder::new(&mut compressed_blob)?;
+ let hash = compute_sha256(decoder)?.into();
+ oci_tar_image = compressed_blob.into_oci_tar_image();
+ hash
+ }
+ // Skip any other non-ImageLayer related media types.
+ // Match explicitly to avoid missing new image layer types when oci-spec updates.
+ MediaType::Descriptor
+ | MediaType::LayoutHeader
+ | MediaType::ImageManifest
+ | MediaType::ImageIndex
+ | MediaType::ImageConfig
+ | MediaType::ArtifactManifest
+ | MediaType::EmptyJSON
+ | MediaType::Other(_) => continue,
+ };
+
+ layer_mapping.insert(digest, layer.clone());
+ }
+
+ Ok((oci_tar_image, layer_mapping))
+}
+
+#[derive(Debug, Error)]
+pub enum ProxmoxOciError {
+ #[error("Error while parsing OCI image: {0}")]
+ ParseError(#[from] ParseError),
+ #[error("Error while extracting OCI image: {0}")]
+ ExtractError(#[from] ExtractError),
+}
+
+/// Extract the rootfs of an OCI image tar and return the image config.
+///
+/// # Arguments
+///
+/// * `oci_tar_path` - Path to the OCI image tar archive
+/// * `rootfs_path` - Destination path where the rootfs will be extracted to
+/// * `arch` - Optional CPU architecture used to pick the first matching manifest from a multi-arch
+/// image index. If `None`, the first manifest will be used.
+pub fn parse_and_extract_image<P: AsRef<Path>>(
+ oci_tar_path: P,
+ rootfs_path: P,
+ arch: Option<&Arch>,
+) -> Result<Option<Config>, ProxmoxOciError> {
+ let (oci_tar_image, image_manifest, image_config) = parse_image(oci_tar_path, arch)?;
+
+ extract_image_rootfs(oci_tar_image, &image_manifest, &image_config, rootfs_path)?;
+
+ Ok(image_config.config().clone())
+}
+
+#[derive(Debug, Error)]
+pub enum ParseError {
+ #[error("OCI spec error: {0}")]
+ OciSpec(#[from] OciSpecError),
+ #[error("Wrong media type")]
+ WrongMediaType,
+ #[error("IO error: {0}")]
+ Io(#[from] std::io::Error),
+ #[error("Unsupported CPU architecture")]
+ UnsupportedArchitecture,
+ #[error("Missing image config")]
+ MissingImageConfig,
+}
+
+fn parse_image<P: AsRef<Path>>(
+ oci_tar_path: P,
+ arch: Option<&Arch>,
+) -> Result<(OciTarImage<File>, ImageManifest, ImageConfiguration), ParseError> {
+ let oci_tar_file = File::open(oci_tar_path)?;
+ let mut oci_tar_image = OciTarImage::new(oci_tar_file)?;
+
+ let image_manifest = oci_tar_image
+ .image_manifest(arch)
+ .ok_or(ParseError::UnsupportedArchitecture)??;
+
+ let image_config_descriptor = image_manifest.config();
+
+ if image_config_descriptor.media_type() != &MediaType::ImageConfig {
+ return Err(ParseError::WrongMediaType);
+ }
+
+ let mut image_config_file = oci_tar_image
+ .open_blob(image_config_descriptor.digest())
+ .ok_or(ParseError::MissingImageConfig)?;
+ let image_config = ImageConfiguration::from_reader(&mut image_config_file)?;
+
+ Ok((
+ image_config_file.into_oci_tar_image(),
+ image_manifest,
+ image_config,
+ ))
+}
+
+#[derive(Debug, Error)]
+pub enum ExtractError {
+ #[error("Incorrectly formatted digest: \"{0}\"")]
+ InvalidDigest(String),
+ #[error("Unknown layer digest {0} found in rootfs.diff_ids")]
+ UnknownLayerDigest(oci_spec::image::Digest),
+ #[error("Layer file {0} mentioned in image manifest is missing")]
+ MissingLayerFile(oci_spec::image::Digest),
+ #[error("IO error: {0}")]
+ Io(#[from] std::io::Error),
+ #[error("Layer has wrong media type: {0}")]
+ WrongMediaType(String),
+}
+
+fn extract_image_rootfs<R: Read + Seek, P: AsRef<Path>>(
+ oci_tar_image: OciTarImage<R>,
+ image_manifest: &ImageManifest,
+ image_config: &ImageConfiguration,
+ target_path: P,
+) -> Result<(), ExtractError> {
+ let (mut oci_tar_image, layer_map) = build_layer_map(oci_tar_image, image_manifest)?;
+
+ for layer in image_config.rootfs().diff_ids() {
+ let layer_digest = oci_spec::image::Digest::from_str(layer)
+ .map_err(|_| ExtractError::InvalidDigest(layer.to_string()))?;
+ let layer_descriptor = layer_map
+ .get(&layer_digest)
+ .ok_or(ExtractError::UnknownLayerDigest(layer_digest.clone()))?;
+ let mut layer_file = oci_tar_image
+ .open_blob(layer_descriptor.digest())
+ .ok_or(ExtractError::MissingLayerFile(layer_digest))?;
+
+ type DecodeFn<T> = Box<dyn for<'a> Fn(&'a mut T) -> std::io::Result<Box<dyn Read + 'a>>>;
+ let decode_fn: DecodeFn<OciTarImageBlob<R>> = match layer_descriptor.media_type() {
+ MediaType::ImageLayer | MediaType::ImageLayerNonDistributable => {
+ Box::new(|file| Ok(Box::new(file)))
+ }
+ MediaType::ImageLayerGzip | MediaType::ImageLayerNonDistributableGzip => {
+ Box::new(|file| Ok(Box::new(GzDecoder::new(file))))
+ }
+ MediaType::ImageLayerZstd | MediaType::ImageLayerNonDistributableZstd => {
+ Box::new(|file| Ok(Box::new(zstd::Decoder::new(file)?)))
+ }
+ // Error on any other non-ImageLayer related media types.
+ // Match explicitly to avoid missing new image layer types when oci-spec updates.
+ media_type @ (MediaType::Descriptor
+ | MediaType::LayoutHeader
+ | MediaType::ImageManifest
+ | MediaType::ImageIndex
+ | MediaType::ImageConfig
+ | MediaType::ArtifactManifest
+ | MediaType::EmptyJSON
+ | MediaType::Other(_)) => {
+ return Err(ExtractError::WrongMediaType(media_type.to_string()))
+ }
+ };
+
+ apply_whiteouts(&mut decode_fn(&mut layer_file)?, &target_path)?;
+ layer_file.seek(SeekFrom::Start(0))?;
+ extract_archive(&mut decode_fn(&mut layer_file)?, &target_path)?;
+
+ oci_tar_image = layer_file.into_oci_tar_image();
+ }
+
+ Ok(())
+}
+
+/// Apply whiteouts on previous layers
+fn apply_whiteouts<R: Read, P: AsRef<Path>>(reader: &mut R, target_path: P) -> std::io::Result<()> {
+ let mut archive = Archive::new(reader);
+
+ for entry in archive.entries()? {
+ let file = entry?;
+ if file.header().entry_type() != EntryType::Regular {
+ continue;
+ }
+
+ let filepath = file.path()?;
+ if let Some(filename) = filepath.file_name() {
+ if filename == OPAQUE_WHITEOUT_NAME {
+ if let Some(parent) = filepath.parent() {
+ let whiteout_abs_path = target_path.as_ref().join(parent);
+ if whiteout_abs_path.exists() {
+ for direntry in read_dir(whiteout_abs_path)? {
+ remove_path(direntry?.path())?;
+ }
+ }
+ }
+ } else if let Some(filename) = filename.to_str() {
+ // TODO: Simplify this once OsStr::strip_prefix is implemented
+ if let Some(filename_stripped) = filename.strip_prefix(WHITEOUT_PREFIX) {
+ let whiteout_path = match filename_stripped {
+ "." => match filepath.parent() {
+ Some(p) if p.parent().is_some() => p,
+ _ => continue, // Prevent whiteout of root directory
+ },
+ ".." => continue, // Prevent whiteout of grandparent directory
+ fname => &filepath.with_file_name(fname),
+ };
+ let whiteout_abs_path = target_path.as_ref().join(whiteout_path);
+ if whiteout_abs_path.exists() {
+ remove_path(whiteout_abs_path)?;
+ }
+ }
+ }
+ }
+ }
+
+ Ok(())
+}
+
+fn extract_archive<R: Read, P: AsRef<Path>>(reader: &mut R, target_path: P) -> std::io::Result<()> {
+ let mut archive = Archive::new(reader);
+ archive.set_preserve_ownerships(true);
+ archive.set_preserve_permissions(true);
+ archive.set_unpack_xattrs(true);
+
+ // Delay directory entries until the end (they will be created if needed by descendants),
+ // to ensure that directory permissions do not interfere with descendant extraction.
+ let mut directories = Vec::new();
+ for entry in archive.entries()? {
+ let mut file = entry?;
+ if file.header().entry_type() == EntryType::Directory {
+ directories.push(file);
+ continue;
+ } else if file.header().entry_type() == EntryType::Regular {
+ // Skip whiteout files
+ if let Some(filename) = file.path()?.file_name() {
+ if filename == OPAQUE_WHITEOUT_NAME {
+ continue;
+ } else if let Some(filename) = filename.to_str() {
+ if filename.starts_with(WHITEOUT_PREFIX) {
+ continue;
+ }
+ }
+ }
+ }
+
+ file.unpack_in(&target_path)?;
+ }
+
+ // Apply the directories in reverse topological order,
+ // to avoid failure on restrictive parent directory permissions.
+ directories.sort_by(|a, b| b.path_bytes().cmp(&a.path_bytes()));
+ for mut dir in directories {
+ dir.unpack_in(&target_path)?;
+ }
+
+ Ok(())
+}
+
+fn remove_path(path: PathBuf) -> std::io::Result<()> {
+ if path.metadata()?.is_dir() {
+ remove_dir_all(path)
+ } else {
+ remove_file(path)
+ }
+}
diff --git a/proxmox-oci/src/oci_tar_image.rs b/proxmox-oci/src/oci_tar_image.rs
new file mode 100644
index 00000000..23e1bfe0
--- /dev/null
+++ b/proxmox-oci/src/oci_tar_image.rs
@@ -0,0 +1,144 @@
+use std::collections::HashMap;
+use std::io::{Read, Seek, SeekFrom};
+use std::ops::Range;
+use std::path::{Path, PathBuf};
+
+use oci_spec::image::{Arch, Digest, ImageIndex, ImageManifest, MediaType};
+use oci_spec::OciSpecError;
+use tar::Archive;
+
+use proxmox_io::RangeReader;
+
+#[derive(Clone)]
+struct TarEntry {
+ range: Range<u64>,
+}
+
+impl TarEntry {
+ fn new(range: Range<u64>) -> Self {
+ Self { range }
+ }
+}
+
+pub struct OciTarImage<R: Read + Seek> {
+ reader: R,
+ entries: HashMap<PathBuf, TarEntry>,
+ image_index: ImageIndex,
+}
+
+impl<R: Read + Seek> OciTarImage<R> {
+ pub fn new(reader: R) -> oci_spec::Result<Self> {
+ let mut archive = Archive::new(reader);
+ let entries = archive.entries_with_seek()?;
+ let mut entries_index = HashMap::new();
+ let mut image_index = None;
+
+ for entry in entries {
+ let mut entry = entry?;
+ let offset = entry.raw_file_position();
+ let size = entry.size();
+ let path = entry.path()?.into_owned();
+
+ if path.as_path() == Path::new("index.json") {
+ image_index = Some(ImageIndex::from_reader(&mut entry)?);
+ }
+
+ let tar_entry = TarEntry::new(offset..(offset + size));
+ entries_index.insert(path, tar_entry);
+ }
+
+ if let Some(image_index) = image_index {
+ Ok(Self {
+ reader: archive.into_inner(),
+ entries: entries_index,
+ image_index,
+ })
+ } else {
+ Err(OciSpecError::Other("Missing index.json file".into()))
+ }
+ }
+
+ pub fn image_index(&self) -> &ImageIndex {
+ &self.image_index
+ }
+
+ fn get_blob_entry(&self, digest: &Digest) -> Option<TarEntry> {
+ let path = get_blob_path(digest);
+ self.entries.get(&path).cloned()
+ }
+
+ pub fn open_blob(self, digest: &Digest) -> Option<OciTarImageBlob<R>> {
+ if let Some(entry) = self.get_blob_entry(digest) {
+ Some(OciTarImageBlob::new(self, entry.range))
+ } else {
+ None
+ }
+ }
+
+ pub fn image_manifest(
+ &mut self,
+ architecture: Option<&Arch>,
+ ) -> Option<oci_spec::Result<ImageManifest>> {
+ let digest = match self.image_index.manifests().iter().find(|d| {
+ d.media_type() == &MediaType::ImageManifest
+ && architecture
+ .is_none_or(|a| d.platform().as_ref().is_none_or(|p| p.architecture() == a))
+ }) {
+ Some(descriptor) => descriptor.digest(),
+ None => return None,
+ };
+
+ if let Some(entry) = self.get_blob_entry(digest) {
+ let mut range_reader = RangeReader::new(&mut self.reader, entry.range);
+ Some(ImageManifest::from_reader(&mut range_reader))
+ } else {
+ Some(Err(OciSpecError::Other(format!(
+ "Image manifest with digest {digest} mentioned in image index is missing"
+ ))))
+ }
+ }
+}
+
+fn get_blob_path(digest: &Digest) -> PathBuf {
+ let algorithm = digest.algorithm();
+ let digest = digest.digest();
+ format!("blobs/{algorithm}/{digest}").into()
+}
+
+pub struct OciTarImageBlob<R: Read + Seek> {
+ range_reader: RangeReader<R>,
+ entries: HashMap<PathBuf, TarEntry>,
+ image_index: ImageIndex,
+}
+
+impl<R: Read + Seek> OciTarImageBlob<R> {
+ fn new(archive: OciTarImage<R>, range: Range<u64>) -> Self {
+ let range_reader = RangeReader::new(archive.reader, range);
+
+ Self {
+ range_reader,
+ entries: archive.entries,
+ image_index: archive.image_index,
+ }
+ }
+
+ pub fn into_oci_tar_image(self) -> OciTarImage<R> {
+ OciTarImage {
+ reader: self.range_reader.into_inner(),
+ entries: self.entries,
+ image_index: self.image_index,
+ }
+ }
+}
+
+impl<R: Read + Seek> Read for OciTarImageBlob<R> {
+ fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
+ self.range_reader.read(buf)
+ }
+}
+
+impl<R: Read + Seek> Seek for OciTarImageBlob<R> {
+ fn seek(&mut self, pos: SeekFrom) -> std::io::Result<u64> {
+ self.range_reader.seek(pos)
+ }
+}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH proxmox v5 03/17] proxmox-oci: add tests for whiteout handling
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 01/17] io: introduce RangeReader for bounded reads Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 02/17] add proxmox-oci crate Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox-perl-rs v5 04/17] add Perl mapping for OCI container image parser/extractor Filip Schauer
` (13 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Test extracting OCI images with whiteout special cases.
Test cases inspired by:
https://github.com/containerd/containerd/blob/4312e076a8a3/pkg/archive/tar_test.go
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v4
proxmox-oci/Cargo.toml | 3 +
| 92 ++++++++++++++++++
.../oci_test_whiteout_current_directory.tar | Bin 0 -> 8704 bytes
.../oci_test_whiteout_dead_symlink_parent.tar | Bin 0 -> 8704 bytes
.../oci_test_whiteout_root_breakout.tar | Bin 0 -> 7168 bytes
...oci_test_whiteout_root_parent_breakout.tar | Bin 0 -> 7168 bytes
.../oci_test_whiteout_symlink.tar | Bin 0 -> 8704 bytes
7 files changed, 95 insertions(+)
create mode 100644 proxmox-oci/tests/extract_whiteouts.rs
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar
create mode 100644 proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar
diff --git a/proxmox-oci/Cargo.toml b/proxmox-oci/Cargo.toml
index 6a87c0bd..8d29da73 100644
--- a/proxmox-oci/Cargo.toml
+++ b/proxmox-oci/Cargo.toml
@@ -20,3 +20,6 @@ thiserror = "1"
zstd.workspace = true
proxmox-io.workspace = true
+
+[dev-dependencies]
+proxmox-sys.workspace = true
--git a/proxmox-oci/tests/extract_whiteouts.rs b/proxmox-oci/tests/extract_whiteouts.rs
new file mode 100644
index 00000000..71ec4dea
--- /dev/null
+++ b/proxmox-oci/tests/extract_whiteouts.rs
@@ -0,0 +1,92 @@
+use std::fs::remove_dir_all;
+
+use proxmox_oci::{parse_and_extract_image, Arch};
+use proxmox_sys::fs::make_tmp_dir;
+
+#[test]
+fn test_whiteout_root_breakout() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_root_breakout.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ // Check that the whiteout did not remove the root directory
+ assert!(extract_dir.exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_root_parent_breakout() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ // Check that the whiteout did not remove the root directory
+ assert!(extract_dir.exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_current_directory() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_current_directory.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(!extract_dir.join("etc").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_symlink() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_symlink.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(extract_dir.join("etc/passwd").exists());
+ assert!(!extract_dir.join("localetc").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
+
+#[test]
+fn test_whiteout_dead_symlink_parent() {
+ let extract_dir = make_tmp_dir("/tmp/", None).unwrap();
+
+ parse_and_extract_image(
+ &"tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar".into(),
+ &extract_dir,
+ Some(&Arch::Amd64),
+ )
+ .unwrap();
+
+ assert!(extract_dir.join("etc/passwd").exists());
+
+ // Cleanup
+ remove_dir_all(extract_dir).unwrap();
+}
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_current_directory.tar
new file mode 100644
index 0000000000000000000000000000000000000000..56b34cfd774b74b8410839bb48333a38409b70d9
GIT binary patch
literal 8704
zcmeHLO^6&t6mB*q#3g!342vKc1`LtSPO9p4Rdp|tlb4`}#RLxti(9`lz0A)tGaIrA
zYd}yC&DoGcSn(o)H&34Q5IiafdhsfF^&%b&@m0@cC$r;j;?B&%V%MR6URS+(^<I77
zd)2z$(wz&@R2#v@35miT+s^xwvfeW#OtPrNr)pJS`Ff|T+Gx_aa|O$>Z7*(IeFrpN
z=kf2<6ysuA+aUPmzr~*dDdK-1J76gOh7zk%sbfF{aw3>ad7PwPLP`@Qq@!SgE5ekv
z95|;$0+&h%pK!~q-o@e&|M&@Lgm-L@p+XqY?dM$1C<q>v#>V6>_6&*t=lDPK@$utd
z-219Fb3_3M3OoUoKuTCDXyUnZi2}*EW&&G^`@j=rg-nEuDL9kr*gC^}0ua5qaOvk?
z=70F{C$semiDuyBN%F#TzsGkQwbhl?dc9WHCr-(jaPF4K%GQx{3zbXfXBIzr`TiS!
z{(5fi=;te^PPC)w{`@zuM#qLiRJ=^j14H=Nf+4Dj2PzE}OgP|H#lW>pxC8?>rkW^W
zK&3(%%?T5!L1-KB+6mY<{>6X``uLX=3j9wm>I2OI|7X7p_}8GBN?FRZrHNoj?x`lR
zaUMj>ElG*dNcSKGc;cAQQgDt}#vzf;Tc6{<x<wKCoy(Vd(0_R5#f$GD`3nKnCF1&^
ze|u?V?z7TMbB~UG^x(m7uYG;zHgd)5f3!Y4{phSe_U&JC?#}IdN3MSVu*V9Yy#L+v
z(Z#>-ej3dV>@J=r<AEXkQ|B3mR(PeX3aNk+ZG^{N5Ku2uP$psCSR+-;4fj$zNW9Tl
zA(&XH{o<cfxg-BGDX{)4^8aLFyqA_+v!zm5wM{MSI@4WmyYeEwH>`-u=ffw{cB<=a
zkTR?Z7f7<e_!UYQNxDdQ1q7xH#^U&DuX`=)bX)BW?0#)IAK&VPq4lh}exui;-D-7x
z5Qdp?g6_r|&RMRj4cG4ET~KH<@88WVcDt7MLQ(ICvN$eF^=B9;9JlN#=LRn0b#r#}
z@DbDy{*{P5!E#Rv=egz~scWSrW_Xrb=DeXPagI~Py}@)(EMbnOoZ8rtSaG(?MhnIM
z9`!#Rt^XO5w7~x^?wqJN$A8DvTtmHs0L)rVgeJ?O+i*5hS8{kFC9JL0Go!lL;llN%
zt+Y&5$r`HaDh<`lyz6u;*XjIf2gmn#qt*0TbpZ2cRA%yFL#lE*sNw{HNct==ZlZe3
zGo9>aAUpsb@S7XnCn6-U9N0aULK~iO0r_yWzk+|2=BqcewE^MSY{x3bJ#G(F_t!x=
z<VNQ!DfJn4I<}8{R2a2gik|IwT|SIVI#F02n6j*C-Hl4X?ewO63jY(}JO2MrN@ZN|
z|MVg<(7YG__lZ3eYx^U^e@zW})i=V0<CY#Hd?G~P^iY)*d@>04$P?!A1<YBk_3p^0
zi3y&De*$9o{6{ICH46MsEC72bx>e4rxVQ2xuw4O`X@yisd2^3}#izmpg$D``6dpL#
G9{2}Zftk|)
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_dead_symlink_parent.tar
new file mode 100644
index 0000000000000000000000000000000000000000..b1b5094689402cce0433174f79c4595c10dae392
GIT binary patch
literal 8704
zcmeHLU5Hdw93K=H8)k*T@*$~h?Bl_9JfHX6yDu7)U}~C5LLuw<+`YOpv(DTVM_baO
zAYADo>Y-?%5A&&njE$_fC=7)xq8@w*q`)2`33^Df?SJo#wzKQBqq8eW_dMLW_dn<3
zcmC)6e&>JwYN)KLt1|Oi5EPD3CjNyo{dcOZ{*Dn5B-6v^6;-42k!mC>0IAclV&qV+
zPOj7bZc=-dtbcV-67J2{Hi-Rled~`YO4t9U?0}~AR}9ukVro2PJ>_}MNyM#2fwDeB
z#Bt(dC#AHOAdh=SaxN&>0aWz_30w=r@%pnV)QG>KT%igk6kmC-+X=xQtbfOxY=NiA
z{6FXafoxAtzN!apQ9fxa)v#R3_w|s6Vr$WwP;RRYTd?Hi@bFNmWds#g?Jb!?S%-zN
zC<knzC`+NU)u^x+uiRCI_DN%1F1c_3TE+vk-&ws`Q&-T~gaa@@45KboeLm6>Nx`fV
z#%e#XlpA5K@QAXKD<iQMJfMNoL}&>WxUkwGB1yoBQY^t3PdwZo&yG?KC`S|EL$U^b
z0OQjM@H+hVCy+D;`~^5wRu1eBhZ{p(YZjg&fKvya(@X$DBb*b+X(N@4>Eswuz&hbH
zwH#qfg+)GPK2yeUYuvwGH|BvENkAtv>1nurez03(F)$yP>LeG+cvx{PsHKi6t&ugv
zC*X2VBdwiKm=i%2p^SQ#IAwyF*r}*vhv$y$Xts%@>BoF|ph^EbrBw@%v8ext{v+5p
z)AgTwBGvyE?wnI|qW`j@2gAtfXrxjX`=asc8kyG>D<f-S?InotUgY-@z769(6!alp
zpq@u4A&fnh5IR2?R-<yI7R%?3L{7`Gl;(%R(#XDgjY_#3x!Bk^7^38W$v0#-Zj(r*
zbN!)#YIPL`Q$zV7Bq0nGQwClTXhS6gCmacewWL@&%(!q6#xQF*NE0`V351!mL4YK4
zN=(t6B<AhYR2$FZ92A!Y4B}1(T4g|kaUeA%Txp+!;ChrGUm=0H&$-r?_)J^h1YSNt
zyaS%2x1wq_@xRjpsm0EmjsFns&EUVlDgRrzb570o9A3P*cHx7;1q+p|+Bj;-{Q4cv
z9z?lCXO|xp=$nc0V}l>Rt?oEp9^SI_!4=us`janQ{N?4XTMr_$WX~J>#+Utea^lXh
z=a-$SWmg=3=KQJ4dsg2bOy2v`XCHm|=ecu#UB=Jmf4@7o>+kdLp4k55YnL9luW)G7
zAFH4E`IE`h6NfKNzPjbHFSFwp-&&u^&FRF_&f4-o6aPC6>ROCAJO5|g{}^K_|690o
zPR%j@FTDPK%zrrPcb7+^{s`_`{oCV!0KOiv(rm32;Q-&gSypT*N+ja+SDPL`v#xjD
z_dl%s`s~7Qb1c)fW~WmN-aC|ev+G3n#_kPAj&vW{ICkj#i)+_*xm>m@7iErqasNw?
zt~s{(^H)xd-?iwSolD1_dw5Ca)NP+G&veb{oYKzP_dr-Oc3&aBx3}NsdTY+cf6Axv
zA7d<1{$H<~Z>W9}|F_8hG!i>Eo(#EWYN!$5Ps`|ZntGfP*Vl785Ys1-`6xWod_v2;
z0eQS$xO;)px2)3fAG7fv(P{BN#&Fk6&;O1c-$n-q@<|qVL$U>~W`Og!fC?x-+C~Ry
RmDB^N2T~8D9=Ihv@DEuzwG;pV
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_breakout.tar
new file mode 100644
index 0000000000000000000000000000000000000000..36e25851148064fdc39d0fee90bdd5762e187989
GIT binary patch
literal 7168
zcmeHL%ZeLE6m35+X2`5;jCT=D7J<pM>)BPMW!8aU7_vwhl6sDMkko_JV>}+TF=Uf?
zX^1fd@&oya{6Gx;iF`tqxm8j|mNN2;qmc=5*P_*Zse9`_>eRU<)343=LAKF>U~q)8
z_zPqC_f6UK9U~%0*5MoVYG(X++~^UwwC-Bba~NmAt!wW+>6=vlaZeLoZfYCEyu4fe
zF-2|t-^&hIsK0ewIa~(kC^gD)jhPqHV?|sUj0s-DgHVd8@|H{M5n&2Di<~x$yK;rc
zvHt80XvA+Ae?tWc%Fo`DoRE^ztg|*ISNO9a{=o5ny4C6AW83os{goe$t9qDsJLIz{
z9e7viy^|vkl3X7h^((6znDF4y&=s|<ifW(_d@<0&D)@0zJi<GNW6)2^wjPG+05szQ
zmp3+5TQCL6t~!7Pq8rnLs#_ApFcZo$;%Mn4a{+5CEi+0A%T>VEawlC$L<kD@rqwba
zZIF;9Gu|X0W8l@37#-mZyqqM*`}zd(0K#vQ<5Tx9A3#zE{|jK1or5RU(QK*nqOfI<
zAq1p?S2zH+N`(PU97K#sf+?VmDW^Tv)=<wF^IS>`N+2{r&P~taz)U8mC-B_gnrtm>
z^z%O!@xP`+=P!}4{Wkm)!j|zbIBM~~BAr(%{`kk+x4(b+Yw!9E&kz<$Ifton0TFHq
z6h=FQgAiJ1!%JIg*npCIg(TB)<5DAm2pEwDXRmjDfA!k_vWv3o^v)f$ee)0YV$eJM
z?y%qQ_030j#myh?<4^CSSJ$53`{|Q=*M9zd`^)@vc=y@UhiL1T%Cf!R{@l)PU1@l&
zu?sS=fd6%m)fH<+vVR-?sVJB64^2jk{}t)HQgPycJ+i&3@wPc0P04=XN;c#1k=KoP
zk$VZ83A;${626CV7pX4d1uaWL;h11wPp9{)aZ`^@B7N>BI;|s>=KX4T{CGNJRM$<2
z!dYX`qy{6O(Qed9R2jmzm4j(h7LHV%t|(n5EM?Af?X^^f2%)&P*a72vhdB<$y5KR)
z2o4KgBBr>Y5(zl0IEB+IPYdkBb8<O?ywYs{_l#9T=N}ibZ`+)27vi7eMf_uo1#9uY
zxr)p!C;YGI|7VT;zi)=my&B8_zSPmLd-eQBFU*8UP=AMBc%GFH=MC<0yKwf_$Ia$q
zz6k$_F7ZFcz%?!YH&>Cl<<mUf;=f2+;OrlO92clS`D8w?eQPt&W}wYLn}PpV2L1xB
CF8omd
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_root_parent_breakout.tar
new file mode 100644
index 0000000000000000000000000000000000000000..12421d00acc42a589f2ccaf50cd085ceb34c4ad4
GIT binary patch
literal 7168
zcmeHKOK%iM5MCK4LKd9h5OKlU6Ucbz_hS#7azWyd1L6R(a8EzHW9{x(J7a`FK61)A
zQbeQ(DYr=ZEs5kSKOny#XR3P_n>DPxWV{?;TH2kNn(FH6>iWK_$p>X#-R&%OAQ&8>
zPQ1c6ef>~2xyOh|(pl$A^=fK-zpBj`Txz;j^c=?V=9=33Li6)<{#D-)E|!%BF)p8X
z{+Ocn{J%&B%$&bRURY+OVT36{xz@(B;JGANGR+ZIG&fdR61byMGR3^l9aYo@NFQqE
zcpT54eE^O44C6gi5S8NiKFJA*m-WV&oa4`o_#4On%eD3Ot{K~YQF~kOkA1fX&qF6z
z_fC9e$KKT5MJYjqZz8ox_(P0)NcRxWs1Sq_!r0@9dcUaZa(n>lPqxzhvWim&#c2Qe
zL}Of*b%?^aGAvLZ?7^DdfjRKwDye|b6(Kw=9H>1>S|FNGR8K%^Qj9E;!C<Ym@_{3Z
zEz=5HDRU}_7M5efl)#EeYLK)H(pFgoa@|e#w&4G8?Qm@-6TZoZ8T^NUbFNS>1f^Cp
zs&nhDM<`&;92F2s=Q!t-IW3G*7;7WAl))KE16E>A1S8nb!oQ^aWBk(=|8v55q2h%9
z%J%)xJOTg~<p`ij5_E^&73N`rCs4xP-k`9i28FxNMlLIDkrhL;<Fla|6~R|^_6%?A
zRxp3U8|5ezJ0~z-(3nlrW=thI*ntHikxW^jX`y=0Gga{-5+30!5<+r(N<=_li76&5
z1wJgefixZb&!DT^*m+UxO_%!AE;!{zX>CJp19l2wL0JeWXsn#nNWf19flzKVWr%Z)
zCD)vq96KJMN}rgX?gKNOoIZsAt;OhV2LDYRnv<F5;h&Jx_=iZy7XNd?d7<K)KVQD=
zU;ou#xzx0>c8)gw{^#h(zPXJ$EA-kmbanM#_Uo{}yS+OY^auHs>vHu^x&0H`esk&8
zW_JJ1O7GQ;-+%n??X4fa`(yk1m2s!@%WtdqI^SRD{#s*aN1zxv|2&JleYRecx98!X
zay(oAF^2Dr7XM9RzUbVu>i@K`$7cQi6~j-q2C3_J#i@$kwDtU`7BVS6w-zoMv#&`H
z`nXN#S<3x-dAm9f|A?OY{>K=;HCy~I??g^4zwD+i{y|EC<1c_N&QONBhbMaNT^oTm
N0&N7^2z*@;cn3S&@<;#x
literal 0
HcmV?d00001
diff --git a/proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar b/proxmox-oci/tests/oci_image_data/oci_test_whiteout_symlink.tar
new file mode 100644
index 0000000000000000000000000000000000000000..7e6835cab889566cdf1d8dc1a42b69642b9d0463
GIT binary patch
literal 8704
zcmeHL-D_M$6u(Kdl$bo!hEVjuuoMlZ+0OUeSrC0Fg`j95ML`M8d}eRDA8~h6l9oV0
z{D3?Og28|WKVF0iz6t&VN?QyFeG)_p;;RUKlGd5K+nSBtW|K_{#XApo?>&3Y%$YOu
z`^}l7N>x=q869W?zEJ?8@ax<5->tIdI|PWMXpA1HSFO$$>kV1+F7-NA@Z7h1lk2s=
zhh(qP^{*EtqU@lyL5R!l)*oV!t^Y&W0d4EAq*e@AV2ISxafCP!MiL+ew}v2zfmFc8
zmRT875JP7PHc&IHsWMWU4jzZ~CtIKqzJ0q36^Jmr``(lz#4wM>dd6f2f7-<VbNpW#
z92?8kb<vjP>$X-;s+C+}44nwlvNef3yS!k%BqtXZN{N;YuW<50#pJ6x$tPtwXY*xQ
zNt~@W@)zL5eBIlp#=2T@$(*+g19ab6)2yv4Z)}n|KR|G!NmK>R9T6I92_W<vRT3`!
z282ooY$SukQ7a9B8WE$d<{Sz|B_x(;MF@2~^)XC5xfo`LG4)=qq>h*5vX6rozLh%O
zbiexvq~_lLygyYY<}M}+t)cEWi`WSvIOA3d<6#DcWJDN?Bw@tH0>uhqOtdrrN+hHM
zkj26xU~vqv=IGz9Tl0W8PMuC?GS&+H!T64ViMg{%OR1cSjmE}uj$v$L=vRUPg#^<a
zBO@3i$V)L$R1hGvl@5Xb`i{aH_*L-FnZcF8Hjri?2Ni)f{_ho59h?lL{s8_56!Pu-
zPchE;ze70pRh;s_tm$IXu)48WYm$B0Fp?rPr)yR=tO?wU0HTv1o<#I4gasf9faWn{
z2qQ$un@#m%Qg2ji%Yi;;QaY^$D$SLW%HmQ}qgJgpToASnhDLh8<XW^FY*MOp?p$KL
zT9bvnsKxwTJIDluK%$sAE@BN7r^-_$@~$8zrFKFwEQkY?P>rde3L~MNCW^%I7T!s7
z-h%a7ELMQ|Kc!?wX-~aA*P%ef2KfVpSVlQyRzM4lLR@j@6CbT(M|`4$bE(HO{<-of
zvQ``Zdp(dk?983;k09EHf5;)s@ZTYv`zn5B_3+`%>)#an2b8Sa=;g_&8*>{FME%nP
z1E73)O3U^2^))cQhBo^?8vXeAXy27D-+S)E7x<}{o(H#|cye|1>U8nxJI^lN{B?-l
z9v&XPe(l)E(CM3>{eJnz+b<3t;y?WO+N;<4M*H9S`fpAM<K_(b{M+wO|8-~l*yCT!
z9v_Hi|NQE=XlP$2m37v21lsW5Yf{%?)Sd7TIo*bTglLBU4$<6Salrp~pIiy}_b2`N
z>SE(u!{4>ey%7=u|24f__J{ZRaGi;Mp7?3=YJl_9<doYu;z<G@Jqm^&`-7~Ni}SPd
zrBbn^hDJQhA#-QI?8d<OWPa*If8pJyK0S5omlW_LL$xUS;MV&`qC@*Tk*v2pMIfmd
zyOa<2_C1+*WwaCiu?Kt5X|>2->LDlD`QKG=dsIsE{|@z^R$*uF#gO}!hFS^!wu(-#
zrN=FKeY2*+SXNKyVsH35-lV^Kd8Y3|d5<aq*AKe?1I)JL-`_Pe{O>W`yE40!ORKoA
hq+8%#1vm%uAP;gYyXq=a&LWUSAd5g2fk!9;{{UmyntT8N
literal 0
HcmV?d00001
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH proxmox-perl-rs v5 04/17] add Perl mapping for OCI container image parser/extractor
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (2 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 03/17] proxmox-oci: add tests for whiteout handling Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 05/17] config: add `lxc.environment.runtime`/`hooks` Filip Schauer
` (12 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This patch depends on the proxmox-oci crate added in patch 02/15.
Changed since v3:
* improve rustdoc comments
* parse_and_extract_image: add argument for CPU architecture
Changed since v2:
* rebase onto newest master (6132d4d36cbd)
* forward all errors to Perl
* remove oci-spec dependency
Changed since v1:
* rebase on latest master (3d9806cb3c7f)
* add new dependencies to debian/control
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 1 +
pve-rs/debian/control | 1 +
pve-rs/src/bindings/mod.rs | 3 +++
pve-rs/src/bindings/oci.rs | 29 +++++++++++++++++++++++++++++
5 files changed, 35 insertions(+)
create mode 100644 pve-rs/src/bindings/oci.rs
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index e40c55c..8d87399 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -39,6 +39,7 @@ proxmox-http-error = "1"
proxmox-log = "1"
proxmox-network-types = "0.1"
proxmox-notify = { version = "1", features = ["pve-context"] }
+proxmox-oci = "0.1.0"
proxmox-openid = "1.0.2"
proxmox-resource-scheduling = "1"
proxmox-section-config = "3"
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index 13a5418..aa7181e 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -27,6 +27,7 @@ PERLMOD_GENPACKAGE := /usr/lib/perlmod/genpackage.pl \
PERLMOD_PACKAGES := \
PVE::RS::Firewall::SDN \
+ PVE::RS::OCI \
PVE::RS::OpenId \
PVE::RS::ResourceScheduling::Static \
PVE::RS::SDN::Fabrics \
diff --git a/pve-rs/debian/control b/pve-rs/debian/control
index e5c2fc8..4b892a8 100644
--- a/pve-rs/debian/control
+++ b/pve-rs/debian/control
@@ -28,6 +28,7 @@ Build-Depends: cargo:native <!nocheck>,
librust-proxmox-network-types-0.1+default-dev,
librust-proxmox-notify-1+default-dev,
librust-proxmox-notify-1+pve-context-dev,
+ librust-proxmox-oci-0.1+default-dev,
librust-proxmox-openid-1+default-dev (>= 1.0.2-~~),
librust-proxmox-resource-scheduling-1+default-dev,
librust-proxmox-section-config-3+default-dev,
diff --git a/pve-rs/src/bindings/mod.rs b/pve-rs/src/bindings/mod.rs
index 7730de3..c21b328 100644
--- a/pve-rs/src/bindings/mod.rs
+++ b/pve-rs/src/bindings/mod.rs
@@ -1,5 +1,8 @@
//! This contains all the perl bindings.
+mod oci;
+pub use oci::pve_rs_oci;
+
mod resource_scheduling_static;
pub use resource_scheduling_static::pve_rs_resource_scheduling_static;
diff --git a/pve-rs/src/bindings/oci.rs b/pve-rs/src/bindings/oci.rs
new file mode 100644
index 0000000..7f5acaf
--- /dev/null
+++ b/pve-rs/src/bindings/oci.rs
@@ -0,0 +1,29 @@
+#[perlmod::package(name = "PVE::RS::OCI", lib = "pve_rs")]
+pub mod pve_rs_oci {
+ //! The `PVE::RS::OCI` package.
+ //!
+ //! Provides bindings for the [`proxmox_oci`] crate.
+
+ use anyhow::Error;
+ use proxmox_oci::Config;
+
+ /// Extract the rootfs of an OCI image tar and return the image config.
+ ///
+ /// # Arguments
+ ///
+ /// * `oci_tar_path` - Path to the OCI image tar archive
+ /// * `rootfs_path` - Destination path where the rootfs will be extracted to
+ /// * `arch` - Optional CPU architecture used to pick the first matching manifest from a multi-arch
+ /// image index. If `None`, the first manifest will be used.
+ #[export]
+ pub fn parse_and_extract_image(
+ oci_tar_path: &str,
+ rootfs_path: &str,
+ arch: Option<&str>,
+ ) -> Result<Config, Error> {
+ let arch = arch.map(Into::into);
+ proxmox_oci::parse_and_extract_image(oci_tar_path, rootfs_path, arch.as_ref())
+ .map(|config| config.unwrap_or_default())
+ .map_err(Into::into)
+ }
+}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH container v5 05/17] config: add `lxc.environment.runtime`/`hooks`
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (3 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox-perl-rs v5 04/17] add Perl mapping for OCI container image parser/extractor Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 06/17] add support for OCI images as container templates Filip Schauer
` (11 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add `lxc.environment.runtime` and `lxc.environment.hooks` config keys.
These allow setting environment variables separately for the container
init process and for LXC hooks.
This will be needed by containers created from OCI images with custom
environment variables which could interfere with hook execution.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This patch depends on the upstream lxc commit df8cf80e960e, which is
expected to be included in the next lxc release 6.0.6.
Introduced in v4
src/PVE/LXC/Config.pm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 5d3749e..1038fa7 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -736,6 +736,8 @@ my $valid_lxc_conf_keys = {
'lxc.start.order' => 1,
'lxc.group' => 1,
'lxc.environment' => 1,
+ 'lxc.environment.runtime' => 1,
+ 'lxc.environment.hooks' => 1,
# All these are namespaced via CLONE_NEWIPC (see namespaces(7)).
'lxc.sysctl.fs.mqueue' => 1,
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH container v5 06/17] add support for OCI images as container templates
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (4 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 05/17] config: add `lxc.environment.runtime`/`hooks` Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 07/17] config: add entrypoint parameter Filip Schauer
` (10 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This aims to add basic support for the Open Container Initiative image
format according to the specification. [0]
[0] https://github.com/opencontainers/image-spec/blob/main/spec.md
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This patch depends on changes made to proxmox-perl-rs in patch 04/17.
Meaning that proxmox-perl-rs needs to be bumped and a dependency & build
dependency to libpve-rs-perl needs to be added to debian/control.
Changed since v3:
* correctly handle case where $archive is '-'
* replace unnecessary regex comparisons with `eq`
* pass environment variables to container via `lxc.container.runtime`
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* check whether archive is an OCI image before trying to parse it as one
Changed since v1:
* fix entrypoint command missing Cmd
* set lxc.signal.halt according to StopSignal (Fixes container shutdown)
src/PVE/API2/LXC.pm | 94 ++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 84 insertions(+), 10 deletions(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index e53b388..7a44547 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -19,9 +19,11 @@ use PVE::Storage;
use PVE::RESTHandler;
use PVE::RPCEnvironment;
use PVE::ReplicationConfig;
+use PVE::RS::OCI;
use PVE::LXC;
use PVE::LXC::Create;
use PVE::LXC::Migrate;
+use PVE::LXC::Namespaces;
use PVE::GuestHelpers;
use PVE::VZDump::Plugin;
use PVE::API2::LXC::Config;
@@ -536,19 +538,91 @@ __PACKAGE__->register_method({
eval {
my $rootdir = PVE::LXC::mount_all($vmid, $storage_cfg, $conf, 1);
+ my $archivepath = '-';
+ $archivepath = PVE::Storage::abs_filesystem_path($storage_cfg, $archive)
+ if ($archive ne '-');
$bwlimit = PVE::Storage::get_bandwidth_limit(
'restore', [keys %used_storages], $bwlimit,
);
- print "restoring '$archive' now..\n"
- if $restore && $archive ne '-';
- PVE::LXC::Create::restore_archive(
- $storage_cfg,
- $archive,
- $rootdir,
- $conf,
- $ignore_unpack_errors,
- $bwlimit,
- );
+ my $is_oci = 0;
+
+ if ($restore && $archive ne '-') {
+ print "restoring '$archive' now..\n";
+ } elsif ($archivepath =~ /\.tar$/) {
+ # Check whether archive is an OCI image
+ my ($has_oci_layout, $has_index_json, $has_blobs) = (0, 0, 0);
+ PVE::Tools::run_command(
+ ['tar', '-tf', $archivepath],
+ outfunc => sub {
+ my $line = shift;
+ $has_oci_layout = 1 if $line eq 'oci-layout';
+ $has_index_json = 1 if $line eq 'index.json';
+ $has_blobs = 1 if $line =~ /^blobs\//m;
+ },
+ );
+
+ $is_oci = 1 if $has_oci_layout && $has_index_json && $has_blobs;
+ }
+
+ if ($is_oci) {
+ # Extract the OCI image
+ my ($id_map, undef, undef) = PVE::LXC::parse_id_maps($conf);
+ my $oci_config = PVE::LXC::Namespaces::run_in_userns(
+ sub {
+ PVE::RS::OCI::parse_and_extract_image($archivepath, $rootdir);
+ },
+ $id_map,
+ );
+
+ # Set the entrypoint and arguments if specified by the OCI image
+ my @init_cmd = ();
+ push(@init_cmd, @{ $oci_config->{Entrypoint} })
+ if $oci_config->{Entrypoint};
+ push(@init_cmd, @{ $oci_config->{Cmd} }) if $oci_config->{Cmd};
+ if (@init_cmd) {
+ my $init_cmd_str = shift(@init_cmd);
+ if (@init_cmd) {
+ $init_cmd_str .= ' ';
+ $init_cmd_str .= join(
+ ' ',
+ map {
+ my $s = $_;
+ $s =~ s/"/\\"/g;
+ qq{"$_"}
+ } @init_cmd,
+ );
+ }
+ if ($init_cmd_str ne '/sbin/init') {
+ push @{ $conf->{lxc} }, ['lxc.init.cmd', $init_cmd_str];
+
+ # An entrypoint other than /sbin/init breaks the tty console mode.
+ # This is fixed by setting cmode: console
+ $conf->{cmode} = 'console';
+ }
+ }
+
+ push @{ $conf->{lxc} }, ['lxc.init.cwd', $oci_config->{WorkingDir}]
+ if ($oci_config->{WorkingDir});
+
+ if (my $envs = $oci_config->{Env}) {
+ for my $env (@{$envs}) {
+ push @{ $conf->{lxc} }, ['lxc.environment.runtime', $env];
+ }
+ }
+
+ my $stop_signal = $oci_config->{StopSignal} // "SIGTERM";
+ push @{ $conf->{lxc} }, ['lxc.signal.halt', $stop_signal];
+ } else {
+ # Not an OCI image, so restore it as an LXC image instead
+ PVE::LXC::Create::restore_archive(
+ $storage_cfg,
+ $archive,
+ $rootdir,
+ $conf,
+ $ignore_unpack_errors,
+ $bwlimit,
+ );
+ }
if ($restore) {
print "merging backed-up and given configuration..\n";
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH container v5 07/17] config: add entrypoint parameter
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (5 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 06/17] add support for OCI images as container templates Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 08/17] configure static IP in LXC config for custom entrypoint Filip Schauer
` (9 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
src/PVE/API2/LXC.pm | 2 +-
src/PVE/LXC.pm | 2 ++
src/PVE/LXC/Config.pm | 12 ++++++++++++
3 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 7a44547..546f4ee 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -593,7 +593,7 @@ __PACKAGE__->register_method({
);
}
if ($init_cmd_str ne '/sbin/init') {
- push @{ $conf->{lxc} }, ['lxc.init.cmd', $init_cmd_str];
+ $conf->{entrypoint} = $init_cmd_str;
# An entrypoint other than /sbin/init breaks the tty console mode.
# This is fixed by setting cmode: console
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index a445a85..5eaa57c 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -862,6 +862,8 @@ sub update_lxc_config {
$raw .= "lxc.rootfs.path = $dir/rootfs\n";
+ $raw .= "lxc.init.cmd = $conf->{entrypoint}\n" if defined($conf->{entrypoint});
+
foreach my $k (sort keys %$conf) {
next if $k !~ m/^net(\d+)$/;
my $ind = $1;
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 1038fa7..56cb01c 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -638,6 +638,12 @@ my $confdesc = {
enum => ['shell', 'console', 'tty'],
default => 'tty',
},
+ entrypoint => {
+ optional => 1,
+ type => 'string',
+ description => "Absolute path from container rootfs to the binary to use as init.",
+ default => '/sbin/init',
+ },
protection => {
optional => 1,
type => 'boolean',
@@ -1861,6 +1867,12 @@ sub get_cmode {
return $conf->{cmode} // $confdesc->{cmode}->{default};
}
+sub get_entrypoint {
+ my ($class, $conf) = @_;
+
+ return $conf->{entrypoint} // $confdesc->{entrypoint}->{default};
+}
+
sub valid_volume_keys {
my ($class, $reverse) = @_;
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH container v5 08/17] configure static IP in LXC config for custom entrypoint
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (6 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 07/17] config: add entrypoint parameter Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 09/17] setup: debian: create /etc/network path if missing Filip Schauer
` (8 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
When a container uses the default `/sbin/init` entrypoint, network
interface configuration is usually managed by processes within the
container. However, containers with a different entrypoint might not
have any internal network management process. Consequently, IP addresses
might not be assigned.
This change ensures that a static IP address is explicitly set in the
LXC config for the container.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* add an "ipmanagehost" property to pct.conf to control whether network
interface IP configuration should be handled by the host.
src/PVE/API2/LXC.pm | 4 ++++
src/PVE/LXC.pm | 15 +++++++++++++++
src/PVE/LXC/Config.pm | 14 ++++++++++++++
3 files changed, 33 insertions(+)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 546f4ee..c8aa984 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -598,6 +598,10 @@ __PACKAGE__->register_method({
# An entrypoint other than /sbin/init breaks the tty console mode.
# This is fixed by setting cmode: console
$conf->{cmode} = 'console';
+
+ # Manage the IP configuration for the container. A container with a
+ # custom entrypoint likely lacks internal network management.
+ $conf->{ipmanagehost} = 1;
}
}
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 5eaa57c..6fdef79 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -886,6 +886,21 @@ sub update_lxc_config {
if ($lxc_major >= 4) {
$raw .= "lxc.net.$ind.script.up = /usr/share/lxc/lxcnetaddbr\n";
}
+
+ if ((!defined($d->{link_down}) || $d->{link_down} != 1) && $conf->{ipmanagehost}) {
+ if (defined($d->{ip})) {
+ die "$k: DHCP is not supported with a custom entrypoint\n" if $d->{ip} eq 'dhcp';
+ $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n" if $d->{ip} ne 'manual';
+ }
+ $raw .= "lxc.net.$ind.ipv4.gateway = $d->{gw}\n" if defined($d->{gw});
+ if (defined($d->{ip6})) {
+ die "$k: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
+ if $d->{ip6} =~ /^(auto|dhcp)$/;
+ $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} ne 'manual';
+ }
+ $raw .= "lxc.net.$ind.ipv6.gateway = $d->{gw6}\n" if defined($d->{gw6});
+ $raw .= "lxc.net.$ind.flags = up\n";
+ }
}
my $had_cpuset = 0;
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 56cb01c..afa2fcf 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -594,6 +594,12 @@ my $confdesc = {
. " This is saved as comment inside the configuration file.",
maxLength => 1024 * 8,
},
+ ipmanagehost => {
+ type => 'boolean',
+ description =>
+ "Whether this interface's IP configuration should be managed by the host.",
+ optional => 1,
+ },
searchdomain => {
optional => 1,
type => 'string',
@@ -1288,6 +1294,14 @@ sub update_pct_config {
die "$opt: MTU size '$mtu' is bigger than bridge MTU '$bridge_mtu'\n"
if ($mtu > $bridge_mtu);
}
+
+ if ((!defined($res->{link_down}) || $res->{link_down} != 1) && $conf->{ipmanagehost}) {
+ die "$opt: DHCP is not supported with a custom entrypoint\n"
+ if defined($res->{ip}) && $res->{ip} eq 'dhcp';
+
+ die "$opt: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
+ if defined($res->{ip6}) && $res->{ip6} =~ /^(auto|dhcp)$/;
+ }
} elsif ($opt =~ m/^dev(\d+)$/) {
my $device = $class->parse_device($value);
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH container v5 09/17] setup: debian: create /etc/network path if missing
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (7 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 08/17] configure static IP in LXC config for custom entrypoint Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 10/17] setup: recursively mkdir /etc/systemd/{network, system-preset} Filip Schauer
` (7 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This prevents an error during Debian container setup when the
/etc/network directory is missing. This fixes container creation from
Debian based OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
src/PVE/LXC/Setup/Debian.pm | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/PVE/LXC/Setup/Debian.pm b/src/PVE/LXC/Setup/Debian.pm
index 030d934..fa99ae4 100644
--- a/src/PVE/LXC/Setup/Debian.pm
+++ b/src/PVE/LXC/Setup/Debian.pm
@@ -487,6 +487,7 @@ sub setup_network {
"auto lo\niface lo inet loopback\n" . "iface lo inet6 loopback\n\n" . $interfaces;
}
+ $self->ct_make_path('/etc/network');
$self->ct_file_set_contents($filename, $interfaces);
}
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH container v5 10/17] setup: recursively mkdir /etc/systemd/{network, system-preset}
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (8 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 09/17] setup: debian: create /etc/network path if missing Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 11/17] implement host-managed DHCP for containers with `ipmanagehost` Filip Schauer
` (6 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Ensure that both /etc/systemd/network and /etc/systemd/system-preset
exist before writing files into them. This fixes container creation from
the docker.io/fedora & docker.io/ubuntu OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
Introduced in v2
src/PVE/LXC/Setup/Base.pm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/PVE/LXC/Setup/Base.pm b/src/PVE/LXC/Setup/Base.pm
index a2c88ed..4442727 100644
--- a/src/PVE/LXC/Setup/Base.pm
+++ b/src/PVE/LXC/Setup/Base.pm
@@ -319,6 +319,7 @@ DATA
$data .= "IPv6AcceptRA = $accept_ra\n";
$data .= $routes if $routes;
+ $self->ct_make_path('/etc/systemd/network');
$self->ct_file_set_contents($filename, $data);
}
}
@@ -358,7 +359,7 @@ sub setup_systemd_preset {
}
}
- $self->ct_mkdir('/etc/systemd/system-preset', 0755);
+ $self->ct_make_path('/etc/systemd/system-preset');
$self->ct_file_set_contents(
'/etc/systemd/system-preset/00-pve.preset', $preset_data,
);
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH container v5 11/17] implement host-managed DHCP for containers with `ipmanagehost`
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (9 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 10/17] setup: recursively mkdir /etc/systemd/{network, system-preset} Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 12/17] allow .tar container templates Filip Schauer
` (5 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Use `dhclient` on the host to manage DHCP for container network
interfaces when `ipmanagehost` is enabled in the container config.
This is useful for containers that lack an internal network management
stack, like many containers based on OCI images.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changes since v3:
* keep lxc pidfd around to ensure the existence of $rootdir until
dhclients are started
* update commit message to reflect changes from v3
Changed since v2:
* rebase onto newest master (5a8b3f962f16) and re-format with
proxmox-perltidy
* add an "ipmanagehost" property to pct.conf to control whether network
interface IP configuration should be handled by the host.
* manage_dhclient: add a FIXME comment regarding the AppArmor profile:
"use a profile that confines writes to /var/lib/lxc/$vmid and rootfs"
* kill_dhclients: untaint pid from pidfile
* fix manage_dhclient called with 'stop' instead of 'start' for IPv6
when container is started
src/PVE/LXC.pm | 103 +++++++++++++++++++++++++++++++++++++-----
src/PVE/LXC/Config.pm | 17 ++++---
2 files changed, 101 insertions(+), 19 deletions(-)
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 6fdef79..69bbd24 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -888,15 +888,12 @@ sub update_lxc_config {
}
if ((!defined($d->{link_down}) || $d->{link_down} != 1) && $conf->{ipmanagehost}) {
- if (defined($d->{ip})) {
- die "$k: DHCP is not supported with a custom entrypoint\n" if $d->{ip} eq 'dhcp';
- $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n" if $d->{ip} ne 'manual';
- }
+ $raw .= "lxc.net.$ind.ipv4.address = $d->{ip}\n"
+ if defined($d->{ip}) && $d->{ip} !~ /^(dhcp|manual)$/;
$raw .= "lxc.net.$ind.ipv4.gateway = $d->{gw}\n" if defined($d->{gw});
if (defined($d->{ip6})) {
- die "$k: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
- if $d->{ip6} =~ /^(auto|dhcp)$/;
- $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} ne 'manual';
+ die "$k: SLAAC is not supported with ipmanagehost\n" if $d->{ip6} eq 'auto';
+ $raw .= "lxc.net.$ind.ipv6.address = $d->{ip6}\n" if $d->{ip6} !~ /^(dhcp|manual)$/;
}
$raw .= "lxc.net.$ind.ipv6.gateway = $d->{gw6}\n" if defined($d->{gw6});
$raw .= "lxc.net.$ind.flags = up\n";
@@ -1079,6 +1076,8 @@ sub vm_stop_cleanup {
PVE::Storage::deactivate_volumes($storage_cfg, $vollist);
};
warn $@ if $@; # avoid errors - just warn
+
+ kill_dhclients($vmid, '*') if $conf->{ipmanagehost};
}
sub net_tap_plug : prototype($$) {
@@ -1310,6 +1309,52 @@ sub get_interfaces {
return $res;
}
+sub manage_dhclient {
+ my ($action, $vmid, $ipversion, $eth, $rootdir) = @_;
+
+ File::Path::make_path("/var/lib/lxc/$vmid/hook") if $action eq 'start';
+ my $pidfile = "/var/lib/lxc/$vmid/hook/dhclient$ipversion-$eth.pid";
+ my $leasefile = "/var/lib/lxc/$vmid/hook/dhclient$ipversion-$eth.leases";
+ my $scriptfile = '/usr/share/lxc/hooks/dhclient-script';
+ PVE::Tools::run_command([
+ 'lxc-attach',
+ '-n',
+ $vmid,
+ '-s',
+ 'NETWORK|UTSNAME',
+ '--',
+ 'aa-exec',
+ '-p',
+ 'unconfined', # FIXME: use a profile that confines writes to /var/lib/lxc/$vmid and rootfs
+ '/sbin/dhclient',
+ $action eq 'start' ? '-1' : '-r',
+ "-$ipversion",
+ '-pf',
+ $pidfile,
+ '-lf',
+ $leasefile,
+ '-e',
+ "ROOTFS=$rootdir",
+ '-sf',
+ $scriptfile,
+ $eth,
+ ]);
+}
+
+sub kill_dhclients {
+ my ($vmid, $eth) = @_;
+
+ foreach my $pidfile (glob("/var/lib/lxc/$vmid/hook/dhclient*-$eth.pid")) {
+ my $pid = eval { file_get_contents($pidfile) };
+ if (!$@) {
+ chomp $pid;
+ next if $pid !~ /^(\d+)$/;
+ kill 9, $1;
+ unlink($pidfile);
+ }
+ }
+}
+
sub update_ipconfig {
my ($vmid, $conf, $opt, $eth, $newnet, $rootdir) = @_;
@@ -1346,11 +1391,21 @@ sub update_ipconfig {
# step 1: add new IP, if this fails we cancel
my $is_real_ip = ($newip && $newip !~ /^(?:auto|dhcp|manual)$/);
- if ($change_ip && $is_real_ip) {
- eval { &$ipcmd($family_opt, 'addr', 'add', $newip, 'dev', $eth); };
- if (my $err = $@) {
- warn $err;
- return;
+ if ($change_ip) {
+ if ($conf->{ipmanagehost}) {
+ if ($newip && $newip eq 'dhcp') {
+ manage_dhclient('start', $vmid, $ipversion, $eth, $rootdir);
+ } elsif ($oldip && $oldip eq 'dhcp') {
+ manage_dhclient('stop', $vmid, $ipversion, $eth, $rootdir);
+ }
+ }
+
+ if ($is_real_ip) {
+ eval { &$ipcmd($family_opt, 'addr', 'add', $newip, 'dev', $eth); };
+ if (my $err = $@) {
+ warn $err;
+ return;
+ }
}
}
@@ -3001,6 +3056,30 @@ sub vm_start {
}
PVE::GuestHelpers::exec_hookscript($conf, $vmid, 'post-start');
+ if ($conf->{ipmanagehost}) {
+ my @dhcpv4_interfaces = ();
+ my @dhcpv6_interfaces = ();
+ foreach my $k (sort keys %$conf) {
+ next if $k !~ m/^net(\d+)$/;
+ my $d = PVE::LXC::Config->parse_lxc_network($conf->{$k});
+ push @dhcpv4_interfaces, $d->{name} if $d->{ip} && $d->{ip} eq 'dhcp';
+ push @dhcpv6_interfaces, $d->{name} if $d->{ip6} && $d->{ip6} eq 'dhcp';
+ }
+
+ my ($pid, $pidfd) = PVE::LXC::open_lxc_pid($vmid);
+ my $rootdir = "/proc/$pid/root";
+
+ foreach my $eth (@dhcpv4_interfaces) {
+ eval { manage_dhclient('start', $vmid, 4, $eth, $rootdir) };
+ PVE::RESTEnvironment::log_warn("DHCP failed - $@") if $@;
+ }
+
+ foreach my $eth (@dhcpv6_interfaces) {
+ eval { manage_dhclient('start', $vmid, 6, $eth, $rootdir) };
+ PVE::RESTEnvironment::log_warn("DHCP failed - $@") if $@;
+ }
+ }
+
return;
}
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index afa2fcf..3797ca5 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -1295,12 +1295,13 @@ sub update_pct_config {
if ($mtu > $bridge_mtu);
}
- if ((!defined($res->{link_down}) || $res->{link_down} != 1) && $conf->{ipmanagehost}) {
- die "$opt: DHCP is not supported with a custom entrypoint\n"
- if defined($res->{ip}) && $res->{ip} eq 'dhcp';
-
- die "$opt: DHCPv6 and SLAAC are not supported with a custom entrypoint\n"
- if defined($res->{ip6}) && $res->{ip6} =~ /^(auto|dhcp)$/;
+ if (
+ (!defined($res->{link_down}) || $res->{link_down} != 1)
+ && $conf->{ipmanagehost}
+ && defined($res->{ip6})
+ && $res->{ip6} eq 'auto'
+ ) {
+ die "$opt: SLAAC is not supported with ipmanagehost\n";
}
} elsif ($opt =~ m/^dev(\d+)$/) {
my $device = $class->parse_device($value);
@@ -1585,9 +1586,11 @@ sub vmconfig_hotplug_pending {
$cgroup->change_cpu_shares(undef);
} elsif ($opt =~ m/^net(\d)$/) {
my $netid = $1;
+ my $net = parse_lxc_network($conf->{$opt});
+ PVE::LXC::kill_dhclients($vmid, $net->{name}) if $conf->{ipmanagehost};
+
PVE::Network::veth_delete("veth${vmid}i$netid");
if ($have_sdn) {
- my $net = PVE::LXC::Config->parse_lxc_network($conf->{$opt});
print "delete ips from $opt\n";
eval {
PVE::Network::SDN::Vnets::del_ips_from_mac(
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH storage v5 12/17] allow .tar container templates
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (10 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 11/17] implement host-managed DHCP for containers with `ipmanagehost` Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 13/17] api: add storage/{storage}/oci-registry-pull method Filip Schauer
` (4 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This is needed for OCI container images bundled as tar files, as
generated by `docker save`. OCI images do not need additional
compression, since the content is usually compressed already.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v2:
* Modify VZTMPL_EXT_RE_1 regex to put "tar" into capture group when
matching on a .tar file.
src/PVE/Storage.pm | 2 +-
src/PVE/Storage/Plugin.pm | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 1dde2b7..0bab945 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -115,7 +115,7 @@ PVE::Storage::Plugin->init();
our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i;
-our $VZTMPL_EXT_RE_1 = qr/\.tar\.(gz|xz|zst|bz2)/i;
+our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i;
our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 8acd214..767087b 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -1618,7 +1618,7 @@ my $get_subdir_files = sub {
} elsif ($tt eq 'vztmpl') {
next if $fn !~ m!/([^/]+$PVE::Storage::VZTMPL_EXT_RE_1)$!;
- $info = { volid => "$sid:vztmpl/$1", format => "t$2" };
+ $info = { volid => "$sid:vztmpl/$1", format => $2 eq 'tar' ? $2 : "t$2" };
} elsif ($tt eq 'backup') {
next if $fn !~ m!/([^/]+$PVE::Storage::BACKUP_EXT_RE_2)$!;
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH storage v5 13/17] api: add storage/{storage}/oci-registry-pull method
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (11 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 12/17] allow .tar container templates Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 14/17] ui: storage upload: accept *.tar files as vztmpl Filip Schauer
` (3 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add a storage API method to pull an OCI image from a registry using
skopeo.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v5.
debian/control | 1 +
src/PVE/API2/Storage/Status.pm | 70 ++++++++++++++++++++++++++++++++++
2 files changed, 71 insertions(+)
diff --git a/debian/control b/debian/control
index 5341317..6bd55a2 100644
--- a/debian/control
+++ b/debian/control
@@ -54,6 +54,7 @@ Depends: bzip2,
${misc:Depends},
${perl:Depends},
Recommends: pve-esxi-import-tools (>= 0.6.0),
+ skopeo,
zfs-zed,
Description: Proxmox VE storage management library
This package contains the storage management library used by Proxmox VE.
diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index 7bde4ec..12b7341 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -265,6 +265,7 @@ __PACKAGE__->register_method({
{ subdir => 'download-url' },
{ subdir => 'file-restore' },
{ subdir => 'import-metadata' },
+ { subdir => 'oci-registry-pull' },
{ subdir => 'prunebackups' },
{ subdir => 'rrd' },
{ subdir => 'rrddata' },
@@ -864,6 +865,75 @@ __PACKAGE__->register_method({
},
});
+__PACKAGE__->register_method({
+ name => 'oci_registry_pull',
+ path => '{storage}/oci-registry-pull',
+ method => 'POST',
+ description => "Pull an OCI image from a registry.",
+ proxyto => 'node',
+ permissions => {
+ check => [
+ 'and',
+ ['perm', '/storage/{storage}', ['Datastore.AllocateTemplate']],
+ ['perm', '/nodes/{node}', ['Sys.AccessNetwork']],
+ ],
+ },
+ protected => 1,
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ storage => get_standard_option('pve-storage-id'),
+ reference => {
+ description => "The reference to the OCI image to download.",
+ type => 'string',
+ pattern =>
+ '^(?:(?:[a-zA-Z\d]|[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d])'
+ . '(?:\.(?:[a-zA-Z\d]|[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d]))*(?::\d+)?/)?[a-z\d]+'
+ . '(?:/[a-z\d]+(?:(?:(?:[._]|__|[-]*)[a-z\d]+)+)?)*:\w[\w.-]{0,127}$',
+ },
+ },
+ },
+ returns => {
+ type => "string",
+ },
+ code => sub {
+ my ($param) = @_;
+
+ die "Install 'skopeo' to pull OCI images from registries.\n" if (!-f '/usr/bin/skopeo');
+
+ my $rpcenv = PVE::RPCEnvironment::get();
+ my $user = $rpcenv->get_user();
+
+ my $cfg = PVE::Storage::config();
+
+ my ($node, $storage) = $param->@{qw(node storage)};
+ my $scfg = PVE::Storage::storage_check_enabled($cfg, $storage, $node);
+
+ die "can't upload to storage type '$scfg->{type}', not a file based storage!\n"
+ if !defined($scfg->{path});
+
+ my $reference = $param->{reference};
+
+ die "storage '$storage' is not configured for content-type 'vztmpl'\n"
+ if !$scfg->{content}->{vztmpl};
+
+ my $filename = PVE::Storage::normalize_content_filename($reference);
+ my $path = PVE::Storage::get_vztmpl_dir($cfg, $storage);
+ PVE::Storage::activate_storage($cfg, $storage);
+
+ my $worker = sub {
+ PVE::Tools::run_command(
+ ["skopeo", "copy", "docker://$reference", "oci-archive:$path/$filename.tar"],
+ );
+ };
+
+ my $worker_id = PVE::Tools::encode_text($filename); # must not pass : or the like as w-ID
+
+ return $rpcenv->fork_worker('ociregistrypull', $worker_id, $user, $worker);
+ },
+});
+
__PACKAGE__->register_method({
name => 'get_import_metadata',
path => '{storage}/import-metadata',
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH manager v5 14/17] ui: storage upload: accept *.tar files as vztmpl
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (12 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 13/17] api: add storage/{storage}/oci-registry-pull method Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 15/17] api: add nodes/{node}/query-oci-repo-tags method Filip Schauer
` (2 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
This is needed for OCI container images bundled as tar files.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
This depends on the change made to pve-storage in patch 12/17.
It might make sense to bump pve-storage and with it bump the dependency
to libpve-storage-perl in debian/control.
Changed since v2:
* rebase onto newest master (84b22751f211) and re-format
Introduced in v2
www/manager6/window/UploadToStorage.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
index cc53596d..df6a9178 100644
--- a/www/manager6/window/UploadToStorage.js
+++ b/www/manager6/window/UploadToStorage.js
@@ -11,7 +11,7 @@ Ext.define('PVE.window.UploadToStorage', {
acceptedExtensions: {
import: ['.ova', '.qcow2', '.raw', '.vmdk'],
iso: ['.img', '.iso'],
- vztmpl: ['.tar.gz', '.tar.xz', '.tar.zst'],
+ vztmpl: ['.tar', '.tar.gz', '.tar.xz', '.tar.zst'],
},
// accepted for file selection, will be renamed to real extension
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH manager v5 15/17] api: add nodes/{node}/query-oci-repo-tags method
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (13 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 14/17] ui: storage upload: accept *.tar files as vztmpl Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 16/17] ui: template view: add OCI registry pull dialog Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH docs v5 17/17] ct: add OCI image docs Filip Schauer
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add an API method for querying the tags of a reference from an OCI
registry.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v5
PVE/API2/Nodes.pm | 47 +++++++++++++++++++++++++++++++++++++++++++++++
debian/control | 1 +
2 files changed, 48 insertions(+)
diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 4590b618..8f86afd9 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -279,6 +279,7 @@ __PACKAGE__->register_method({
{ name => 'netstat' },
{ name => 'network' },
{ name => 'qemu' },
+ { name => 'query-oci-repo-tags' },
{ name => 'query-url-metadata' },
{ name => 'replication' },
{ name => 'report' },
@@ -1761,6 +1762,52 @@ __PACKAGE__->register_method({
},
});
+__PACKAGE__->register_method({
+ name => 'query_oci_repo_tags',
+ path => 'query-oci-repo-tags',
+ method => 'GET',
+ description => "List all tags for an OCI repository reference.",
+ proxyto => 'node',
+ permissions => {
+ check => ['perm', '/nodes/{node}', ['Sys.AccessNetwork']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ reference => {
+ description => "The reference to the repository to query tags from.",
+ type => 'string',
+ pattern => '^(?:(?:[a-zA-Z\d]|[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d])'
+ . '(?:\.(?:[a-zA-Z\d]|[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d]))*(?::\d+)?/)?[a-z\d]+'
+ . '(?:(?:[._]|__|[-]*)[a-z\d]+)*(?:/[a-z\d]+(?:(?:[._]|__|[-]*)[a-z\d]+)*)*$',
+ },
+ },
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'string',
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+
+ die "Install 'skopeo' to list tags from OCI registries.\n" if (!-f '/usr/bin/skopeo');
+
+ my $reference = $param->{reference};
+ my $tags_json = "";
+ PVE::Tools::run_command(
+ ["skopeo", "list-tags", "docker://$reference"],
+ outfunc => sub {
+ $tags_json = $tags_json . shift;
+ },
+ );
+ my $tags = decode_json($tags_json);
+ return $tags->{Tags};
+ },
+});
+
__PACKAGE__->register_method({
name => 'query_url_metadata',
path => 'query-url-metadata',
diff --git a/debian/control b/debian/control
index 64b942ce..afd0386a 100644
--- a/debian/control
+++ b/debian/control
@@ -105,6 +105,7 @@ Depends: apt (>= 1.5~),
Recommends: proxmox-firewall,
proxmox-offline-mirror-helper,
pve-nvidia-vgpu-helper,
+ skopeo,
Conflicts: vlan, vzdump,
Replaces: vlan, vzdump,
Provides: vlan, vzdump,
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH manager v5 16/17] ui: template view: add OCI registry pull dialog
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (14 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 15/17] api: add nodes/{node}/query-oci-repo-tags method Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH docs v5 17/17] ct: add OCI image docs Filip Schauer
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Add a button and a dialog for pulling images from an OCI registry.
Behind the scenes this uses the nodes/{node}/query-oci-repo-tags and
storage/{storage}/oci-registry-pull API methods.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Introduced in v5
www/manager6/storage/TemplateView.js | 139 ++++++++++++++++++++++++++-
1 file changed, 138 insertions(+), 1 deletion(-)
diff --git a/www/manager6/storage/TemplateView.js b/www/manager6/storage/TemplateView.js
index 8a775bbd..de98d788 100644
--- a/www/manager6/storage/TemplateView.js
+++ b/www/manager6/storage/TemplateView.js
@@ -176,6 +176,130 @@ Ext.define('PVE.storage.TemplateDownload', {
},
});
+Ext.define('PVE.storage.OciRegistryPull', {
+ extend: 'Proxmox.window.Edit',
+ alias: 'widget.pveOciRegistryPull',
+ mixins: ['Proxmox.Mixin.CBind'],
+
+ method: 'POST',
+
+ showTaskViewer: true,
+
+ title: gettext('Pull from OCI Registry'),
+ submitText: gettext('Download'),
+ width: 450,
+
+ cbind: {
+ url: '/nodes/{nodename}/storage/{storage}/oci-registry-pull',
+ },
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ onReferenceChange: function (field, value) {
+ let me = this;
+ let view = me.getView();
+ let tagField = view.down('[name=tag]');
+ tagField.setComboItems([]);
+
+ let parts = value.split(':');
+ if (parts.length > 1) {
+ field.setValue(parts[0]);
+ tagField.setValue(parts[1]);
+ tagField.focus();
+ } else {
+ tagField.clearValue();
+ }
+ },
+
+ queryTags: function (field) {
+ let me = this;
+ let view = me.getView();
+ let refField = view.down('[name=reference]');
+ let reference = refField.value;
+ let tagField = view.down('[name=tag]');
+
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${view.nodename}/query-oci-repo-tags`,
+ method: 'GET',
+ params: {
+ reference,
+ },
+ waitMsgTarget: view,
+ failure: (res) => {
+ Ext.MessageBox.alert(gettext('Error'), res.htmlStatus);
+ },
+ success: function (res, opt) {
+ let tags = res.result.data;
+ tagField.clearValue();
+ tagField.setComboItems(tags.map((tag) => [tag, tag]));
+ },
+ });
+ },
+ },
+
+ items: [
+ {
+ xtype: 'inputpanel',
+ border: false,
+ onGetValues: function (values) {
+ return {
+ reference: values.reference + ':' + values.tag,
+ };
+ },
+ items: [
+ {
+ xtype: 'fieldcontainer',
+ layout: 'hbox',
+ fieldLabel: gettext('Reference'),
+ items: [
+ {
+ xtype: 'textfield',
+ name: 'reference',
+ allowBlank: false,
+ emptyText: 'registry.example.org/name',
+ flex: 1,
+ listeners: {
+ change: 'onReferenceChange',
+ },
+ },
+ {
+ xtype: 'button',
+ name: 'check',
+ text: gettext('Query Tags'),
+ margin: '0 0 0 5',
+ listeners: {
+ click: 'queryTags',
+ },
+ },
+ ],
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'tag',
+ allowBlank: false,
+ emptyText: 'latest',
+ fieldLabel: gettext('Tag'),
+ forceSelection: false,
+ editable: true,
+ typeAhead: true,
+ comboItems: [],
+ },
+ ],
+ },
+ ],
+
+ initComponent: function () {
+ var me = this;
+
+ if (!me.nodename) {
+ throw 'no node name specified';
+ }
+
+ me.callParent();
+ },
+});
+
Ext.define('PVE.storage.TemplateView', {
extend: 'PVE.storage.ContentView',
@@ -213,7 +337,20 @@ Ext.define('PVE.storage.TemplateView', {
},
});
- me.tbar = [templateButton];
+ var pullOciImageButton = Ext.create('Proxmox.button.Button', {
+ itemId: 'pull-oci-img-btn',
+ text: gettext('Pull from OCI Registry'),
+ handler: function () {
+ var win = Ext.create('PVE.storage.OciRegistryPull', {
+ nodename: nodename,
+ storage: storage,
+ taskDone: () => reload(),
+ });
+ win.show();
+ },
+ });
+
+ me.tbar = [templateButton, pullOciImageButton];
me.useUploadButton = true;
me.callParent();
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* [pve-devel] [PATCH docs v5 17/17] ct: add OCI image docs
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
` (15 preceding siblings ...)
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 16/17] ui: template view: add OCI registry pull dialog Filip Schauer
@ 2025-10-08 17:10 ` Filip Schauer
16 siblings, 0 replies; 18+ messages in thread
From: Filip Schauer @ 2025-10-08 17:10 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
Changed since v4:
* document the "Pull from OCI registry" feature
Introduced in v3
pct.adoc | 44 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 36 insertions(+), 8 deletions(-)
diff --git a/pct.adoc b/pct.adoc
index d6146eb..8212bc9 100644
--- a/pct.adoc
+++ b/pct.adoc
@@ -54,15 +54,22 @@ the cluster setup, and they can use the same network and storage resources as
virtual machines. You can also use the {pve} firewall, or manage containers
using the HA framework.
-Our primary goal is to offer an environment that provides the benefits of using a
-VM, but without the additional overhead. This means that Proxmox Containers can
-be categorized as ``System Containers'', rather than ``Application Containers''.
+Our primary goal has traditionally been to offer an environment that provides
+the benefits of using a VM, but without the additional overhead. This means that
+Proxmox Containers have been primarily categorized as ``System Containers''.
-NOTE: If you want to run application containers, for example, 'Docker' images, it
-is recommended that you run them inside a Proxmox QEMU VM. This will give you
-all the advantages of application containerization, while also providing the
-benefits that VMs offer, such as strong isolation from the host and the ability
-to live-migrate, which otherwise isn't possible with containers.
+With the introduction of OCI (**O**pen **C**ontainer **I**nitiative) image support,
+Proxmox VE now also integrates ``Application Containers''. When creating a
+container from an OCI image, the image is automatically converted to the
+LXC stack that Proxmox VE uses.
+
+This approach allows users to benefit from a wide ecosystem of pre-packaged
+applications while retaining the robust management features of Proxmox VE.
+
+While running lightweight ``Application Containers'' directly offers significant
+advantages over a full VM, for use cases demanding maximum isolation and
+the ability to live-migrate, nesting containers inside a Proxmox QEMU VM remains
+a recommended practice.
Technology Overview
@@ -256,6 +263,12 @@ Container Images
Container images, sometimes also referred to as ``templates'' or
``appliances'', are `tar` archives which contain everything to run a container.
+Proxmox VE can utilize two main types of images: *System Container Templates*
+for creating full virtual environments, and *Application Container Images* based
+on the OCI standard for running specific applications.
+
+System Container Templates
+~~~~~~~~~~~~~~~~~~~~~~~~~~
{pve} itself provides a variety of basic templates for the
xref:pct_supported_distributions[most common Linux distributions]. They can be
@@ -336,6 +349,21 @@ delete that image later with:
# pveam remove local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz
----
+Open Container Initiative (OCI) Images (technology preview)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Proxmox VE can also use OCI images to create containers. This makes it easy to
+run pre-packaged applications. A container created from an OCI image still uses
+the existing LXC framework.
+
+Obtaining OCI Images
+~~~~~~~~~~~~~~~~~~~~
+
+In the web interface an OCI image can be pulled from a registry using the *Pull
+from OCI registry* button on the container template view of a storage.
+
+Once the template is on a storage, you can create the container with
+`pct create` or use the wizard in the web interface.
[[pct_settings]]
Container Settings
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2025-10-08 17:15 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-10-08 17:10 [pve-devel] [PATCH container/docs/manager/proxmox{, -perl-rs}/storage v5 00/17] support OCI images as container templates Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 01/17] io: introduce RangeReader for bounded reads Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 02/17] add proxmox-oci crate Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox v5 03/17] proxmox-oci: add tests for whiteout handling Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH proxmox-perl-rs v5 04/17] add Perl mapping for OCI container image parser/extractor Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 05/17] config: add `lxc.environment.runtime`/`hooks` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 06/17] add support for OCI images as container templates Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 07/17] config: add entrypoint parameter Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 08/17] configure static IP in LXC config for custom entrypoint Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 09/17] setup: debian: create /etc/network path if missing Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 10/17] setup: recursively mkdir /etc/systemd/{network, system-preset} Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH container v5 11/17] implement host-managed DHCP for containers with `ipmanagehost` Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 12/17] allow .tar container templates Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH storage v5 13/17] api: add storage/{storage}/oci-registry-pull method Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 14/17] ui: storage upload: accept *.tar files as vztmpl Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 15/17] api: add nodes/{node}/query-oci-repo-tags method Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH manager v5 16/17] ui: template view: add OCI registry pull dialog Filip Schauer
2025-10-08 17:10 ` [pve-devel] [PATCH docs v5 17/17] ct: add OCI image docs Filip Schauer
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.