* [PATCH proxmox-ebpf v4 1/5] tc: add the link plumbing
2026-10-19 14:24 [PATCH proxmox-ebpf v4 0/5] add proxmox-ebpf library Hannes Laimer
@ 2026-10-19 14:24 ` Hannes Laimer
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 2/5] subsystem: add the shared tc program handling Hannes Laimer
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Hannes Laimer @ 2026-10-19 14:24 UTC (permalink / raw)
To: pve-devel
Attach, swap and detach of tcx links on an interface, and the naming
of their pins in bpffs. A swap rebinds the pinned link to a new
program in place, so an update never interrupts traffic.
What a failed step means for the interface is the caller's decision.
So a lost race against another attach, a vanished interface and a pin
already gone each come back as their own error, read from the
kernel's answer.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/lib.rs | 2 +
src/tc.rs | 303 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 305 insertions(+)
create mode 100644 src/tc.rs
diff --git a/src/lib.rs b/src/lib.rs
index 244f44c..8d4420a 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,2 +1,4 @@
//! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
//! on its own, consumers pull in only the subsystem they drive.
+
+pub mod tc;
diff --git a/src/tc.rs b/src/tc.rs
new file mode 100644
index 0000000..05d823e
--- /dev/null
+++ b/src/tc.rs
@@ -0,0 +1,303 @@
+//! TC link plumbing shared by all subsystems. How a subsystem's programs are attached as tcx
+//! links (kernel 6.6 and newer), swapped and detached, and how their pins are named in bpffs,
+//! see [`LinkPin`].
+
+use std::{io::ErrorKind, path::Path, str::FromStr};
+
+use anyhow::Context;
+use aya::{
+ pin::PinError,
+ programs::{
+ ProgramError, SchedClassifier, TcAttachType,
+ links::{FdLink, LinkError, LinkOrder, PinnedLink},
+ tc::TcAttachOptions,
+ },
+};
+
+/// Why a link step did not go through, with the kernel's own answer where it gives one, so a
+/// caller can tell a lost race or a vanished interface from a real failure.
+#[derive(Debug)]
+pub enum Error {
+ /// A link of the program is in place already, another attach of the interface won the race.
+ AlreadyAttached,
+ /// The interface is gone, its index names no netdev anymore.
+ IfaceGone,
+ /// The pin was gone by the time it was opened or removed, another attach or unplug took it.
+ PinGone,
+ Other(anyhow::Error),
+}
+
+impl std::fmt::Display for Error {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ match self {
+ Self::AlreadyAttached => f.write_str("link already in place"),
+ Self::IfaceGone => f.write_str("interface is gone"),
+ Self::PinGone => f.write_str("pin is gone"),
+ Self::Other(e) if f.alternate() => write!(f, "{e:#}"),
+ Self::Other(e) => write!(f, "{e}"),
+ }
+ }
+}
+
+impl std::error::Error for Error {
+ fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+ match self {
+ Self::Other(e) => e.chain().nth(1),
+ _ => None,
+ }
+ }
+}
+
+impl From<anyhow::Error> for Error {
+ fn from(e: anyhow::Error) -> Self {
+ Self::Other(e)
+ }
+}
+
+// EEXIST on the link create is the kernel refusing a second link of the program in the chain
+impl From<ProgramError> for Error {
+ fn from(e: ProgramError) -> Self {
+ match e {
+ ProgramError::SyscallError(s) if s.io_error.kind() == ErrorKind::AlreadyExists => {
+ Self::AlreadyAttached
+ }
+ e => Self::Other(e.into()),
+ }
+ }
+}
+
+// EEXIST on the pin is the path taken by another attach, the link is there either way
+impl From<PinError> for Error {
+ fn from(e: PinError) -> Self {
+ match e {
+ PinError::SyscallError(s) if s.io_error.kind() == ErrorKind::AlreadyExists => {
+ Self::AlreadyAttached
+ }
+ e => Self::Other(e.into()),
+ }
+ }
+}
+
+impl From<LinkError> for Error {
+ fn from(e: LinkError) -> Self {
+ match e {
+ LinkError::SyscallError(s) if s.io_error.kind() == ErrorKind::NotFound => Self::PinGone,
+ e => Self::Other(e.into()),
+ }
+ }
+}
+
+impl From<std::io::Error> for Error {
+ fn from(e: std::io::Error) -> Self {
+ if e.kind() == ErrorKind::NotFound {
+ Self::PinGone
+ } else {
+ Self::Other(e.into())
+ }
+ }
+}
+
+#[derive(Copy, Clone, Hash, PartialEq, Eq, Debug)]
+pub enum Direction {
+ Ingress,
+ Egress,
+}
+
+impl Direction {
+ pub fn as_str(self) -> &'static str {
+ match self {
+ Self::Ingress => "ingress",
+ Self::Egress => "egress",
+ }
+ }
+ pub fn aya_type(self) -> TcAttachType {
+ match self {
+ Self::Ingress => TcAttachType::Ingress,
+ Self::Egress => TcAttachType::Egress,
+ }
+ }
+}
+
+impl FromStr for Direction {
+ type Err = ();
+ fn from_str(s: &str) -> Result<Self, Self::Err> {
+ match s {
+ "ingress" => Ok(Self::Ingress),
+ "egress" => Ok(Self::Egress),
+ _ => Err(()),
+ }
+ }
+}
+
+/// One pinned link as its pin file names it. The interface name is part of the file name, so an
+/// unplug reported after the interface is gone still finds the pin and a reused index does not
+/// pass for the old link. The fields are separated by a colon, which the kernel forbids in
+/// interface names. A dot inside the name is written as a colon as well, since bpffs refuses a
+/// dot in a file name.
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+pub struct LinkPin {
+ pub ifindex: u32,
+ pub name: String,
+ pub dir: Direction,
+}
+
+impl LinkPin {
+ pub fn filename(&self) -> String {
+ format!(
+ "{}:{}:{}",
+ self.ifindex,
+ self.dir.as_str(),
+ self.name.replace('.', ":")
+ )
+ }
+
+ /// The pin a file name stands for, none when it is not one of ours.
+ pub fn parse(filename: &str) -> Option<Self> {
+ let mut parts = filename.splitn(3, ':');
+ let ifindex = parts.next()?.parse().ok()?;
+ let dir = parts.next()?.parse().ok()?;
+ let name = parts.next()?.replace(':', ".");
+ let pin = LinkPin { ifindex, name, dir };
+ // a file name that does not round-trip is not one of ours, and neither is one the kernel
+ // would refuse as an interface name
+ let valid_name = !pin.name.is_empty()
+ && pin.name.len() < 16
+ && pin.name != "."
+ && pin.name != ".."
+ && !pin.name.chars().any(|c| c.is_whitespace() || c == '/');
+ (pin.filename() == filename && valid_name).then_some(pin)
+ }
+}
+
+/// Attach `prog` on the interface and direction of `pin` as a tcx link and pin it at
+/// `pin_path`. tcx (kernel 6.6 and newer) runs its programs on the netdev hook ahead of the
+/// filters on the interface's qdiscs, such as the `ingress` qdisc the rate limiter installs.
+/// `TC_ACT_UNSPEC` hands a packet on to the next program and then to the filters. Any other
+/// verdict ends the chain right there. So a subsystem's program returns `TC_ACT_UNSPEC` for
+/// whatever it leaves alone. Programs of several subsystems run in attach order, each new link
+/// goes last.
+pub fn attach_and_pin(
+ prog: &mut SchedClassifier,
+ pin: &LinkPin,
+ pin_path: &Path,
+) -> Result<(), Error> {
+ let link_id = prog
+ .attach_with_options(
+ &pin.name,
+ pin.dir.aya_type(),
+ TcAttachOptions::TcxOrder(LinkOrder::last()),
+ )
+ .map_err(|e| match Error::from(e) {
+ // the interface can go away between the name lookup and the attach
+ Error::Other(_) if matches!(iface_name(pin.ifindex), Ok(None)) => Error::IfaceGone,
+ e => e,
+ })?;
+ let link = prog.take_link(link_id)?;
+ let fd_link: FdLink = link.try_into()?;
+ // a lost race against another attach of the same interface shows up here as well, the
+ // unpinned link detaches again when its fd drops
+ fd_link.pin(pin_path)?;
+ Ok(())
+}
+
+/// Rebind a pinned link to `prog` via LINK_UPDATE. Atomic, traffic sees no detach/reattach gap.
+pub fn swap_pinned_link(prog: &mut SchedClassifier, pin_path: &Path) -> Result<(), Error> {
+ let pinned = PinnedLink::from_pin(pin_path)?;
+ let fd_link: FdLink = pinned.into();
+ let link = fd_link.try_into()?;
+ let new_id = prog.attach_to_link(link)?;
+ // take the handle out of aya's internal tracking, we have the pin
+ let _ = prog.take_link(new_id)?;
+ Ok(())
+}
+
+pub fn detach_pinned_link(pin_path: &Path) -> Result<(), Error> {
+ let pinned = PinnedLink::from_pin(pin_path)?;
+ let _fd_link = pinned.unpin()?;
+ Ok(())
+}
+
+/// The name of an interface by index, none when it is gone. Any other failure says nothing
+/// about the interface and is not taken as its absence, so the lookup reads errno itself.
+pub fn iface_name(ifindex: u32) -> anyhow::Result<Option<String>> {
+ let mut buf = [0u8; nix::libc::IF_NAMESIZE];
+ // SAFETY: the buffer holds IF_NAMESIZE bytes, the most the call writes
+ let name = unsafe { nix::libc::if_indextoname(ifindex, buf.as_mut_ptr().cast()) };
+ if name.is_null() {
+ return match nix::errno::Errno::last() {
+ nix::errno::Errno::ENXIO => Ok(None),
+ e => Err(e).with_context(|| format!("name of interface {ifindex}")),
+ };
+ }
+ let name = std::ffi::CStr::from_bytes_until_nul(&buf)?
+ .to_str()
+ .with_context(|| format!("name of interface {ifindex} is not UTF-8"))?;
+ Ok(Some(name.to_owned()))
+}
+
+/// The pins in `links_dir`, see [`LinkPin`], none for a missing directory. A file that is not a
+/// pin is skipped with a warning.
+pub fn read_pinned_links(links_dir: &Path) -> anyhow::Result<Vec<LinkPin>> {
+ let mut out = Vec::new();
+ let dir = match std::fs::read_dir(links_dir) {
+ Ok(d) => d,
+ Err(e) if matches!(e.kind(), ErrorKind::NotFound | ErrorKind::NotADirectory) => {
+ return Ok(out);
+ }
+ Err(e) => return Err(e.into()),
+ };
+ for entry in dir {
+ let entry = entry?;
+ let name = entry.file_name();
+ match name.to_str().and_then(LinkPin::parse) {
+ Some(pin) => out.push(pin),
+ None => log::warn!(
+ "unrecognized pin file {} in {}",
+ name.to_string_lossy(),
+ links_dir.display()
+ ),
+ }
+ }
+ Ok(out)
+}
+
+#[cfg(test)]
+mod test {
+ use super::*;
+
+ #[test]
+ fn pin_names_round_trip() {
+ for name in ["tap100i0", "veth100i31", "vmbr0.100", "bond0.5.7"] {
+ let pin = LinkPin {
+ ifindex: 42,
+ name: name.into(),
+ dir: Direction::Ingress,
+ };
+ assert!(!pin.filename().contains('.'), "{}", pin.filename());
+ assert_eq!(LinkPin::parse(&pin.filename()), Some(pin));
+ }
+ assert_eq!(
+ LinkPin::parse("7:egress:eno1").map(|p| p.name),
+ Some("eno1".into())
+ );
+ }
+
+ #[test]
+ fn pin_names_are_canonical() {
+ for bad in [
+ "042:ingress:tap0",
+ "+42:ingress:tap0",
+ "42:ingress:",
+ "42:ingress",
+ "42:Ingress:tap0",
+ "42-ingress",
+ "4294967296:ingress:tap0",
+ "42:ingress:tap 0",
+ "42:ingress::",
+ "42:ingress:::",
+ "42:ingress:sixteencharsname",
+ ] {
+ assert_eq!(LinkPin::parse(bad), None, "{bad}");
+ }
+ }
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH proxmox-ebpf v4 2/5] subsystem: add the shared tc program handling
2026-10-19 14:24 [PATCH proxmox-ebpf v4 0/5] add proxmox-ebpf library Hannes Laimer
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 1/5] tc: add the link plumbing Hannes Laimer
@ 2026-10-19 14:24 ` Hannes Laimer
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 3/5] subsystem: place the links of different subsystems in a fixed order Hannes Laimer
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Hannes Laimer @ 2026-10-19 14:24 UTC (permalink / raw)
To: pve-devel
A subsystem owns tc classifier programs and their maps, attached per
interface and pinned in bpffs together with an object fingerprint
and a schema version. So state survives between the one-shot calls
of its consumer, and re-running never interrupts traffic.
Changes run under one lock per subsystem, taken before the change
reads its input, so they reach the kernel in the order they read it.
A subsystem describes its programs with a macro, from its name, the
program names, the maps and the schema version. The object is found
by the subsystem's name, and its fingerprint is derived from its
bytes.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/lib.rs | 1 +
src/subsystem.rs | 829 +++++++++++++++++++++++++++++++++++++++++++++++
src/tc.rs | 12 +-
3 files changed, 836 insertions(+), 6 deletions(-)
create mode 100644 src/subsystem.rs
diff --git a/src/lib.rs b/src/lib.rs
index 8d4420a..572d861 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,4 +1,5 @@
//! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs
//! on its own, consumers pull in only the subsystem they drive.
+pub mod subsystem;
pub mod tc;
diff --git a/src/subsystem.rs b/src/subsystem.rs
new file mode 100644
index 0000000..c740fa8
--- /dev/null
+++ b/src/subsystem.rs
@@ -0,0 +1,829 @@
+//! The shared subsystem code. A subsystem has one tc program per direction it declares, with
+//! their maps and links. They are pinned under `/sys/fs/bpf/proxmox-ebpf/<name>/<schema>/`. The
+//! programs are pinned by the fingerprint of their object, so the pinned state itself says which
+//! build it belongs to.
+//!
+//! The loaded BPF stays in the kernel between invocations, so [`ApplyLock::ensure_loaded`] loads
+//! only when what is pinned is not this build's, or links still sit under another schema. Every
+//! other step works on what is already pinned. Each subsystem owns one [`TcPrograms`].
+//!
+//! Every change runs under the subsystem's apply lock, a full apply exclusively and a
+//! single-interface step shared. The holder reads the input of its change under the lock and
+//! writes before it lets go, so a full apply and every step reach the kernel in the order they
+//! read their input. Two steps of one interface do not order each other.
+use std::{
+ collections::HashSet,
+ fs::File,
+ io::ErrorKind,
+ marker::PhantomData,
+ path::{Path, PathBuf},
+ time::{Duration, Instant},
+};
+
+use anyhow::Context;
+use aya::{EbpfLoader, programs::SchedClassifier};
+use nix::{
+ errno::Errno,
+ fcntl::{Flock, FlockArg},
+};
+
+use crate::tc::{self, Direction};
+
+const PIN_ROOT: &str = "/sys/fs/bpf/proxmox-ebpf";
+const RUN_ROOT: &str = "/run/proxmox-ebpf";
+
+const fn obj_fingerprint(obj: &[u8]) -> u64 {
+ let mut hash = 0xcbf29ce484222325u64;
+ let mut i = 0;
+ while i < obj.len() {
+ hash ^= obj[i] as u64;
+ hash = hash.wrapping_mul(0x100000001b3);
+ i += 1;
+ }
+ hash
+}
+
+/// A subsystem's [`TcPrograms`] from its name. The object is the build's `<name>.bpf.o`, the
+/// fingerprint follows from the bytes.
+#[macro_export]
+macro_rules! tc_programs {
+ (
+ name: $name:literal,
+ programs: $programs:expr,
+ maps: $maps:expr,
+ schema_version: $schema:expr $(,)?
+ ) => {
+ // built at compile time wherever the macro is used, a let binding included
+ const {
+ const OBJ: &[u8] =
+ ::aya::include_bytes_aligned!(concat!(env!("OUT_DIR"), "/", $name, ".bpf.o"));
+ $crate::subsystem::TcPrograms::new($name, OBJ, $programs, $maps, $schema)
+ }
+ };
+}
+
+const LOCK_WAIT: Duration = Duration::from_secs(10);
+const LOCK_POLL: Duration = Duration::from_millis(10);
+
+// flock has no timeout, so the wait retries a nonblocking flock with a sleep in between.
+//
+// not proxmox-sys lock because: blocks and gives up on a timer signal, and under pve-rs any signal
+// perl handles cuts the blocked wait short the same way, since perl installs its handlers without
+// restart
+fn flock_within(
+ path: &Path,
+ exclusive: bool,
+ what: &str,
+ subsystem: &str,
+) -> anyhow::Result<Flock<File>> {
+ let mut file = File::options()
+ .create(true)
+ .truncate(false)
+ .write(true)
+ .open(path)?;
+ let arg = if exclusive {
+ FlockArg::LockExclusiveNonblock
+ } else {
+ FlockArg::LockSharedNonblock
+ };
+ let deadline = Instant::now() + LOCK_WAIT;
+ loop {
+ match Flock::lock(file, arg) {
+ Ok(lock) => return Ok(lock),
+ Err((returned, Errno::EWOULDBLOCK)) if Instant::now() < deadline => {
+ file = returned;
+ std::thread::sleep(LOCK_POLL);
+ }
+ Err((_, Errno::EWOULDBLOCK)) => anyhow::bail!(
+ "{subsystem}: {what} still held after {}s",
+ LOCK_WAIT.as_secs()
+ ),
+ Err((_, e)) => return Err(e.into()),
+ }
+ }
+}
+
+/// A step that needs the programs pinned found none. Only [`ApplyLock::ensure_loaded`] under the
+/// exclusive lock installs them.
+#[derive(Debug)]
+pub struct NotLoaded;
+
+impl std::fmt::Display for NotLoaded {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ f.write_str("programs not loaded")
+ }
+}
+
+impl std::error::Error for NotLoaded {}
+
+/// A subsystem's description of its tc programs, written with the `tc_programs!` macro. The
+/// loaded programs live pinned in the kernel and are reached through those pins.
+pub struct TcPrograms {
+ /// The subsystem's name, its directory under the pin root.
+ name: &'static str,
+ /// The embedded BPF object.
+ obj: &'static [u8],
+ /// FNV-1a of `obj`, the programs are pinned under it.
+ fingerprint: u64,
+ /// The directions the object carries a program for, each with that program's name in the
+ /// object. One entry per direction.
+ programs: &'static [(Direction, &'static str)],
+ /// The object's pinned maps. A missing one means the state is not current.
+ maps: &'static [&'static str],
+ /// Bumped with any change to a map's key or value layout. A pinned map is reused by name
+ /// without any check, only the Rust accessor checks its key and value sizes.
+ schema_version: u32,
+}
+
+impl TcPrograms {
+ pub const fn new(
+ name: &'static str,
+ obj: &'static [u8],
+ programs: &'static [(Direction, &'static str)],
+ maps: &'static [&'static str],
+ schema_version: u32,
+ ) -> Self {
+ Self {
+ name,
+ obj,
+ fingerprint: obj_fingerprint(obj),
+ programs,
+ maps,
+ schema_version,
+ }
+ }
+
+ fn pin_root(&self) -> PathBuf {
+ PathBuf::from(PIN_ROOT).join(self.name)
+ }
+ fn schema_root(&self) -> PathBuf {
+ self.pin_root().join(self.schema_version.to_string())
+ }
+ fn links_dir(&self) -> PathBuf {
+ schema_links_dir(&self.schema_root())
+ }
+ fn link_pin_path(&self, pin: &tc::LinkPin) -> PathBuf {
+ self.links_dir().join(pin.filename())
+ }
+ fn progs_dir(&self) -> PathBuf {
+ schema_progs_dir(&self.schema_root())
+ }
+ fn fingerprint_dir_name(&self) -> String {
+ format!("{:016x}", self.fingerprint)
+ }
+ fn prog_dir(&self) -> PathBuf {
+ self.progs_dir().join(self.fingerprint_dir_name())
+ }
+ fn prog_pin_path(&self, dir: Direction) -> PathBuf {
+ self.prog_dir().join(dir.as_str())
+ }
+ fn program_name(&self, dir: Direction) -> Option<&'static str> {
+ self.programs
+ .iter()
+ .find(|&&(d, _)| d == dir)
+ .map(|&(_, name)| name)
+ }
+ fn run_dir(&self) -> PathBuf {
+ PathBuf::from(RUN_ROOT).join(self.name)
+ }
+ fn hash_map<K: aya::Pod, V: aya::Pod>(
+ &self,
+ name: &str,
+ ) -> anyhow::Result<aya::maps::HashMap<aya::maps::MapData, K, V>> {
+ let map = aya::maps::MapData::from_pin(self.schema_root().join(name))
+ .with_context(|| format!("{}: open pinned map {name}", self.name))?;
+ Ok(aya::maps::HashMap::try_from(aya::maps::Map::HashMap(map))?)
+ }
+
+ /// Open one direction's pinned program, [`NotLoaded`] when it is not pinned.
+ fn program(&self, dir: Direction) -> anyhow::Result<SchedClassifier> {
+ let path = self.prog_pin_path(dir);
+ if !path.exists() {
+ return Err(NotLoaded.into());
+ }
+ SchedClassifier::from_pin(path)
+ .with_context(|| format!("{}: open pinned program {}", self.name, dir.as_str()))
+ }
+
+ /// The apply lock under `/run`. A full apply, an install and a teardown take it exclusively,
+ /// a single-interface step shared. A wait times out instead of hanging. Shared holders do not
+ /// yield to a waiting exclusive one, so a node attaching interfaces without pause can keep a
+ /// full apply waiting until it times out. Every write to the pinned state is
+ /// a method of the returned guard, so a step cannot run outside the lock.
+ fn lock<M>(&self, exclusive: bool) -> anyhow::Result<ApplyLock<'_, M>> {
+ let dir = self.run_dir();
+ std::fs::create_dir_all(&dir)?;
+ let lock = flock_within(&dir.join("lock"), exclusive, "apply lock", self.name)?;
+ Ok(ApplyLock {
+ programs: self,
+ _lock: lock,
+ _mode: PhantomData,
+ })
+ }
+
+ /// Whether a link of the interface name is pinned, read without the lock. Most interfaces
+ /// never had a link and their unplug must not wait behind a full apply. A miss is a snapshot,
+ /// a pin arriving after it is another attach's. A failed listing counts as a hit. So do links
+ /// still pinned under another schema, an install may be moving them over.
+ pub fn link_pinned(&self, name: &str) -> bool {
+ let elsewhere = self
+ .other_schemas()
+ .map_or(true, |others| links_pinned_under(&others));
+ elsewhere
+ || self
+ .live_links()
+ .map_or(true, |links| links.iter().any(|pin| pin.name == name))
+ }
+
+ /// Take the apply lock shared, for a single-interface step.
+ pub fn lock_shared(&self) -> anyhow::Result<ApplyLock<'_, Shared>> {
+ self.lock(false)
+ }
+
+ /// Take the apply lock exclusively, for a full apply, an install or a teardown.
+ pub fn lock_exclusive(&self) -> anyhow::Result<ApplyLock<'_, Exclusive>> {
+ self.lock(true)
+ }
+
+ fn ensure_loaded(&self) -> anyhow::Result<()> {
+ let others = self.other_schemas()?;
+ // a link pinned under another schema is a live attachment to a program of that schema.
+ // A run that died between moving the links and pinning its programs leaves some of the
+ // links there. This build is not current until the install below has taken them over
+ let links_elsewhere = links_pinned_under(&others);
+ // programs pinned under a newer schema mean a newer build runs here and this process is
+ // the stale one, so it leaves the state alone. A newer schema directory without programs
+ // is what a rejected install of that build left behind
+ if let Some(newer) = self.newer_schema_installed(&others) {
+ anyhow::bail!(
+ "{}: pinned state is of schema {newer}, newer than this build's {}, clear it to \
+ downgrade",
+ self.name,
+ self.schema_version
+ );
+ }
+ if self.is_current() && !links_elsewhere {
+ // an older schema here is left over from a run that died between its install and the
+ // removal, its links moved over already
+ for (dir, schema) in others {
+ if (schema.is_some_and(|s| s < self.schema_version)
+ || !any_program_pinned(&schema_progs_dir(&dir)))
+ && let Err(e) = std::fs::remove_dir_all(&dir)
+ {
+ log::warn!(
+ "{}: remove old pinned state {}: {e:#}",
+ self.name,
+ dir.display()
+ );
+ }
+ }
+ return Ok(());
+ }
+
+ // a lost map pin comes back at load time, ahead of the programs. If the install then
+ // failed, the previous run's program pins would make that half-done state pass for
+ // current. So they go first, the links keep the old programs running meanwhile
+ if !self.maps_pinned() {
+ match std::fs::remove_dir_all(self.prog_dir()) {
+ Ok(()) => {}
+ Err(e) if e.kind() == ErrorKind::NotFound => {}
+ Err(e) => return Err(e).context("remove program pins ahead of the reinstall"),
+ }
+ }
+
+ // any other schema pinned here is old state whose map layout this build cannot bind to.
+ // It stays in place through the install and goes afterward
+ let schema_changed = !others.is_empty();
+ if schema_changed {
+ log::warn!("{}: pinned state of another schema, rebuilding", self.name);
+ }
+
+ // a failed install leaves its directory as it is, a link moved into it already is a live
+ // attachment. Without program pins the directory does not pass for state of that build
+ let old_links: Vec<PathBuf> = others
+ .iter()
+ .map(|(dir, _)| schema_links_dir(dir))
+ .collect();
+ self.install(&old_links).context("install")?;
+ for (dir, _) in &others {
+ // a link whose pin could not be moved is still live there
+ if any_links(&schema_links_dir(dir)) {
+ continue;
+ }
+ if let Err(e) = std::fs::remove_dir_all(dir) {
+ log::warn!(
+ "{}: remove old pinned state {}: {e:#}",
+ self.name,
+ dir.display()
+ );
+ }
+ }
+ Ok(())
+ }
+
+ fn is_current(&self) -> bool {
+ self.programs
+ .iter()
+ .all(|&(dir, _)| self.prog_pin_path(dir).exists())
+ && self.maps_pinned()
+ }
+
+ fn maps_pinned(&self) -> bool {
+ self.maps
+ .iter()
+ .all(|map| self.schema_root().join(map).exists())
+ }
+
+ // the directories under the pin root other than this build's schema. One whose name is not
+ // a version counts as well, so its links are moved over too
+ fn other_schemas(&self) -> anyhow::Result<Vec<(PathBuf, Option<u32>)>> {
+ let entries = match std::fs::read_dir(self.pin_root()) {
+ Ok(entries) => entries,
+ Err(e) if e.kind() == ErrorKind::NotFound => return Ok(Vec::new()),
+ Err(e) => return Err(e.into()),
+ };
+ let mut others = Vec::new();
+ for entry in entries {
+ let entry = entry?;
+ if !entry.file_type()?.is_dir() {
+ continue;
+ }
+ let schema = entry
+ .file_name()
+ .to_str()
+ .and_then(|s| s.parse::<u32>().ok());
+ if schema != Some(self.schema_version) {
+ others.push((entry.path(), schema));
+ }
+ }
+ Ok(others)
+ }
+
+ fn newer_schema_installed(&self, others: &[(PathBuf, Option<u32>)]) -> Option<u32> {
+ others
+ .iter()
+ .filter(|(dir, _)| any_program_pinned(&schema_progs_dir(dir)))
+ .filter_map(|(_, schema)| *schema)
+ .find(|&s| s > self.schema_version)
+ }
+
+ fn tear_down(&self) -> anyhow::Result<()> {
+ match std::fs::remove_dir_all(self.pin_root()) {
+ Ok(()) => Ok(()),
+ Err(e) if e.kind() == ErrorKind::NotFound => Ok(()),
+ Err(e) => Err(e.into()),
+ }
+ }
+
+ /// Load this build's object and move every link onto its programs, this schema's and those
+ /// under `old_links`, then pin the programs and drop those of other builds. The verifier runs
+ /// here and nowhere else, so a rejection leaves what served before untouched. The program
+ /// pins mark this build as current and go last, so after a failure the previous build stays
+ /// current and the next run installs again. A link that cannot be swapped is unpinned, and
+ /// the next full apply attaches it again if it is still wanted.
+ fn install(&self, old_links: &[PathBuf]) -> anyhow::Result<()> {
+ std::fs::create_dir_all(self.links_dir())?;
+ let mut bpf = EbpfLoader::new()
+ .map_pin_path(self.schema_root())
+ .load(self.obj)?;
+ // load everything before touching any link, so a verifier rejection of one direction
+ // leaves the old state whole
+ for &(_, name) in self.programs {
+ let prog: &mut SchedClassifier = bpf
+ .program_mut(name)
+ .with_context(|| format!("{}: program {name} not found in object", self.name))?
+ .try_into()?;
+ prog.load()?;
+ }
+ self.remove_foreign_pins()?;
+
+ let live = self.live_links()?;
+ for &(dir, _) in self.programs {
+ let prog = self.loaded(&mut bpf, dir)?;
+ for pin in live.iter().filter(|pin| pin.dir == dir) {
+ let path = self.link_pin_path(pin);
+ if let Err(e) = tc::swap_pinned_link(prog, &path) {
+ log::warn!(
+ "{}: {} swap failed ({e:#}), reclaiming",
+ self.name,
+ pin.filename()
+ );
+ self.reclaim_link(pin);
+ }
+ }
+ }
+
+ // the links of another schema are rebound the same way and their pins moved over, so
+ // their interfaces never run without a program and keep their place in the tcx order.
+ // One this build has no program for, or that already has a link here, is detached
+ // instead. A rebound link whose pin cannot be moved stays where it is
+ for links_dir in old_links {
+ for pin in tc::read_pinned_links(links_dir)? {
+ let old = links_dir.join(pin.filename());
+ let new = self.link_pin_path(&pin);
+ let migrated = self.program_name(pin.dir).is_some()
+ && !new.exists()
+ && match tc::swap_pinned_link(self.loaded(&mut bpf, pin.dir)?, &old) {
+ Ok(()) => true,
+ Err(e) => {
+ log::warn!(
+ "{}: {} swap failed ({e:#}), reclaiming",
+ self.name,
+ pin.filename()
+ );
+ false
+ }
+ };
+ if migrated {
+ if let Err(e) = std::fs::rename(&old, &new) {
+ log::warn!(
+ "{}: move link pin {} to {}: {e:#}",
+ self.name,
+ old.display(),
+ new.display()
+ );
+ }
+ } else if let Err(e) = tc::detach_pinned_link(&old) {
+ log::warn!("{}: detach old link {}: {e:#}", self.name, old.display());
+ let _ = std::fs::remove_file(&old);
+ }
+ }
+ }
+
+ // a run that died after pinning some of the directions left a partial set behind
+ match std::fs::remove_dir_all(self.prog_dir()) {
+ Ok(()) => {}
+ Err(e) if e.kind() == ErrorKind::NotFound => {}
+ Err(e) => return Err(e).context("remove partial program pins"),
+ }
+ std::fs::create_dir_all(self.prog_dir())?;
+ for &(dir, _) in self.programs {
+ self.loaded(&mut bpf, dir)?.pin(self.prog_pin_path(dir))?;
+ }
+
+ // the links are on this build now, the other builds' programs go with their pins
+ let mine = self.fingerprint_dir_name();
+ for entry in std::fs::read_dir(self.progs_dir())? {
+ let entry = entry?;
+ if entry.file_name() != mine.as_str()
+ && let Err(e) = std::fs::remove_dir_all(entry.path())
+ {
+ log::warn!(
+ "{}: remove stale program pins {}: {e:#}",
+ self.name,
+ entry.path().display()
+ );
+ }
+ }
+ Ok(())
+ }
+
+ // one direction's program out of the loaded object, resolved once by the load loop
+ fn loaded<'b>(
+ &self,
+ bpf: &'b mut aya::Ebpf,
+ dir: Direction,
+ ) -> anyhow::Result<&'b mut SchedClassifier> {
+ let name = self
+ .program_name(dir)
+ .expect("only directions with a program are loaded");
+ let prog = bpf
+ .program_mut(name)
+ .expect("every program was resolved by the load loop")
+ .try_into()?;
+ Ok(prog)
+ }
+
+ fn live_links(&self) -> anyhow::Result<Vec<tc::LinkPin>> {
+ tc::read_pinned_links(&self.links_dir())
+ }
+
+ // a pin this build cannot read or has no program for is not its state. It would keep a link
+ // running with nothing to update it, so it goes at install time
+ fn remove_foreign_pins(&self) -> anyhow::Result<()> {
+ for entry in std::fs::read_dir(self.links_dir())? {
+ let entry = entry?;
+ let name = entry.file_name();
+ let ours = name
+ .to_str()
+ .and_then(tc::LinkPin::parse)
+ .is_some_and(|pin| self.program_name(pin.dir).is_some());
+ if ours {
+ continue;
+ }
+ let name = name.to_string_lossy();
+ log::warn!("{}: removing pin {name} this build cannot use", self.name);
+ let path = entry.path();
+ if let Err(e) = tc::detach_pinned_link(&path) {
+ log::warn!("{}: detach {name}: {e:#}", self.name);
+ let _ = std::fs::remove_file(&path);
+ }
+ }
+ Ok(())
+ }
+
+ fn detach_named(
+ &self,
+ name: &str,
+ keep: impl Fn(&tc::LinkPin) -> bool,
+ ) -> anyhow::Result<Vec<u32>> {
+ let mut ifindexes = Vec::new();
+ for pin in self.live_links()? {
+ if pin.name != name || keep(&pin) {
+ continue;
+ }
+ self.reclaim_link(&pin);
+ if !ifindexes.contains(&pin.ifindex) {
+ ifindexes.push(pin.ifindex);
+ }
+ }
+ Ok(ifindexes)
+ }
+
+ /// Unpin a link. The pin file goes even when the kernel object behind it cannot be opened
+ /// anymore.
+ fn reclaim_link(&self, pin: &tc::LinkPin) {
+ let path = self.link_pin_path(pin);
+ match tc::detach_pinned_link(&path) {
+ Ok(()) => {}
+ // the other side of a lost race, another attach or unplug took the pin first
+ Err(tc::Error::PinGone) => {
+ log::debug!(
+ "{}: reclaim {}: pin gone already",
+ self.name,
+ pin.filename()
+ );
+ }
+ Err(e) => {
+ log::warn!(
+ "{}: reclaim {}: unpin stale link: {e:#}",
+ self.name,
+ pin.filename()
+ );
+ let _ = std::fs::remove_file(&path);
+ }
+ }
+ }
+
+ fn reconcile(&self, desired: &HashSet<u32>) -> anyhow::Result<()> {
+ // opened before anything is detached, so a missing program fails the full apply before it
+ // changes anything
+ let mut progs = Vec::new();
+ for &(dir, _) in self.programs {
+ progs.push((dir, self.program(dir)?));
+ }
+ let live = self.live_links()?;
+ let mut attached: HashSet<(u32, Direction)> = HashSet::new();
+
+ for pin in &live {
+ // a pin whose index now belongs to another interface links nothing wanted
+ let alive = tc::iface_name(pin.ifindex)?.is_some_and(|name| name == pin.name);
+ if alive && desired.contains(&pin.ifindex) {
+ attached.insert((pin.ifindex, pin.dir));
+ continue;
+ }
+ log::debug!("{}: detach {}", self.name, pin.filename());
+ let path = self.link_pin_path(pin);
+ if let Err(e) = tc::detach_pinned_link(&path) {
+ log::error!("{}: detach {}: {e:#}", self.name, pin.filename());
+ let _ = std::fs::remove_file(&path);
+ } else {
+ log::info!("{}: detached {}", self.name, pin.filename());
+ }
+ }
+
+ let mut failed = 0usize;
+ for (dir, prog) in &mut progs {
+ let dir = *dir;
+ for &ifindex in desired {
+ if attached.contains(&(ifindex, dir)) {
+ continue;
+ }
+ let Some(name) = tc::iface_name(ifindex)? else {
+ log::info!("{}: {ifindex} vanished before the attach", self.name);
+ continue;
+ };
+ let pin = tc::LinkPin { ifindex, name, dir };
+ match tc::attach_and_pin(prog, &pin, &self.link_pin_path(&pin)) {
+ Ok(()) => {}
+ Err(tc::Error::IfaceGone) => {
+ log::info!("{}: {ifindex} vanished before the attach", self.name);
+ }
+ Err(e) => {
+ log::error!("{}: attach {}: {e:#}", self.name, pin.filename());
+ failed += 1;
+ }
+ }
+ }
+ }
+ if failed > 0 {
+ anyhow::bail!("{}: {failed} link(s) failed to attach", self.name);
+ }
+ Ok(())
+ }
+
+ fn attach_iface(&self, ifindex: u32) -> anyhow::Result<()> {
+ // opened for every direction ahead of the pin checks, so a missing program fails here,
+ // never counts as a dead link and leaves no direction attached alone
+ let mut progs = Vec::new();
+ for &(dir, _) in self.programs {
+ progs.push((dir, self.program(dir)?));
+ }
+ let Some(name) = tc::iface_name(ifindex)? else {
+ log::info!("{}: {ifindex} vanished before the attach", self.name);
+ return Ok(());
+ };
+ let live = self.live_links()?;
+ for (dir, prog) in &mut progs {
+ let dir = *dir;
+ let pin = tc::LinkPin {
+ ifindex,
+ name: name.clone(),
+ dir,
+ };
+ let path = self.link_pin_path(&pin);
+ for stale in live.iter().filter(|p| p.ifindex == ifindex && p.dir == dir) {
+ if stale.name == name {
+ match tc::swap_pinned_link(prog, &path) {
+ Ok(()) => {}
+ // the pin went away under the swap, another plug of the same interface
+ // is rebuilding it and the attach below finds its link
+ Err(tc::Error::PinGone) => {
+ log::debug!(
+ "{}: {} pin taken by another attach",
+ self.name,
+ path.display()
+ );
+ continue;
+ }
+ Err(e) => {
+ log::warn!(
+ "{}: {} swap failed ({e:#}), rebuilding",
+ self.name,
+ pin.filename()
+ );
+ self.reclaim_link(stale);
+ continue;
+ }
+ }
+ } else {
+ // the index was reused since, the old link is not this interface's
+ self.reclaim_link(stale);
+ }
+ }
+ if path.exists() {
+ continue;
+ }
+ match tc::attach_and_pin(prog, &pin, &path) {
+ Ok(()) => {}
+ Err(tc::Error::IfaceGone) => {
+ log::info!("{}: {ifindex} vanished before the attach", self.name);
+ }
+ Err(tc::Error::AlreadyAttached) => {
+ log::debug!("{}: {} attached concurrently", self.name, pin.filename());
+ }
+ Err(e) => {
+ return Err(e)
+ .with_context(|| format!("{}: attach {}", self.name, pin.filename()));
+ }
+ }
+ }
+ Ok(())
+ }
+}
+
+fn schema_links_dir(schema_root: &Path) -> PathBuf {
+ schema_root.join("links")
+}
+
+fn schema_progs_dir(schema_root: &Path) -> PathBuf {
+ schema_root.join("prog")
+}
+
+fn links_pinned_under(schemas: &[(PathBuf, Option<u32>)]) -> bool {
+ schemas
+ .iter()
+ .any(|(dir, _)| any_links(&schema_links_dir(dir)))
+}
+
+// whether link pins remain in a links directory. Debris of another kind does not count, it goes
+// with the directory. One that cannot be read may hold anything, a missing one is empty
+fn any_links(links_dir: &Path) -> bool {
+ match tc::read_pinned_links(links_dir) {
+ Ok(links) => !links.is_empty(),
+ Err(_) => true,
+ }
+}
+
+// a program pin is a file under a fingerprint directory. A run killed between creating that
+// directory and pinning leaves it empty. Like the links, a directory that cannot be read may
+// hold anything, a missing one is empty
+fn any_program_pinned(progs_dir: &Path) -> bool {
+ let non_empty = |dir: &Path| match std::fs::read_dir(dir) {
+ Ok(mut entries) => entries.next().is_some(),
+ Err(e) => e.kind() != ErrorKind::NotFound,
+ };
+ match std::fs::read_dir(progs_dir) {
+ Ok(fingerprints) => fingerprints.into_iter().any(|fp| match fp {
+ Ok(fp) => non_empty(&fp.path()),
+ Err(_) => true,
+ }),
+ Err(e) => e.kind() != ErrorKind::NotFound,
+ }
+}
+
+/// Marker for a guard holding the apply lock shared.
+pub struct Shared;
+
+/// Marker for a guard holding the apply lock exclusively.
+pub struct Exclusive;
+
+/// The apply lock of one subsystem, held for as long as the guard lives. Loading and matching
+/// the links to the desired set rewrite pinned state and exist only on the [`Exclusive`]
+/// guard, so nothing runs them next to shared holders. A shared holder adds or drops its own
+/// link and writes map entries that stand on their own, each write atomic in the kernel. A
+/// write that depends on what was read before it takes the exclusive lock, or the kernel's
+/// conditional write, insert only if absent and update only if present. A shared holder cannot
+/// upgrade, one that finds nothing current leaves the install to a full apply under the
+/// exclusive lock.
+pub struct ApplyLock<'a, M> {
+ programs: &'a TcPrograms,
+ _lock: Flock<File>,
+ _mode: PhantomData<M>,
+}
+
+impl<M> ApplyLock<'_, M> {
+ /// Whether the programs and maps of this build are pinned. What a shared holder checks before
+ /// it writes, since it cannot install. Under the lock the program and map pins do not change,
+ /// only links come and go, so the answer holds for the holder's steps.
+ pub fn is_current(&self) -> bool {
+ self.programs.is_current()
+ }
+
+ /// Open a pinned BPF hash map by name. Valid while the programs are pinned, an exclusive
+ /// holder runs [`ensure_loaded`](ApplyLock::ensure_loaded) first.
+ pub fn hash_map<K: aya::Pod, V: aya::Pod>(
+ &self,
+ name: &str,
+ ) -> anyhow::Result<aya::maps::HashMap<aya::maps::MapData, K, V>> {
+ self.programs.hash_map(name)
+ }
+
+ /// Attach the programs to a single interface, leaving the others alone.
+ ///
+ /// If a pin already exists, swap the program in place with no traffic gap. A pin that cannot
+ /// be swapped is reclaimed and the interface attached fresh. Two concurrent attaches of the
+ /// same interface run under the shared lock, the one that loses the race finds the other's
+ /// link in place and is done. A failed attach returns the error, [`NotLoaded`] when nothing
+ /// is pinned yet, and the caller decides what an interface without the program means for it.
+ pub fn attach_iface(&self, ifindex: u32) -> anyhow::Result<()> {
+ self.programs.attach_iface(ifindex)
+ }
+
+ /// Detach the programs from the interface of that name, even when the interface is gone
+ /// already. A pin for which `keep` returns true stays, such as one of a newer interface under
+ /// the same name. Returns the indexes the detached links had.
+ pub fn detach_named(
+ &self,
+ name: &str,
+ keep: impl Fn(&tc::LinkPin) -> bool,
+ ) -> anyhow::Result<Vec<u32>> {
+ self.programs.detach_named(name, keep)
+ }
+
+ /// The links of this subsystem as their pins name them. A shared holder sees a snapshot,
+ /// another attach may add to it meanwhile. The name is not unique. A leftover pin of an
+ /// interface that went away and the pin of its successor under the name share it, only the
+ /// index tells them apart.
+ pub fn attached(&self) -> anyhow::Result<Vec<tc::LinkPin>> {
+ self.programs.live_links()
+ }
+}
+
+impl ApplyLock<'_, Exclusive> {
+ /// Make sure this build's programs and maps are pinned, loading only when they are not. Maps
+ /// keep their entries across a load unless the schema changed, then they start empty, and a
+ /// lost map pin comes back empty. The caller syncs the maps in the same apply.
+ pub fn ensure_loaded(&self) -> anyhow::Result<()> {
+ self.programs.ensure_loaded()
+ }
+
+ /// Make the attached set match `desired`. Refreshing links onto new code is
+ /// [`ensure_loaded`](Self::ensure_loaded)'s job. The live set cannot change under it, every
+ /// other writer waits on the exclusive lock. What may go is decided from the kernel's state.
+ ///
+ /// An interface that failed to attach fails the apply once every other was tried, since the
+ /// subsystem does not work on an interface without its program. One that vanished since it
+ /// was named is no failure, and a failed detach only leaves a program running longer and is
+ /// logged.
+ pub fn reconcile(&self, desired: &HashSet<u32>) -> anyhow::Result<()> {
+ self.programs.reconcile(desired)
+ }
+
+ /// Drop the pinned state of this subsystem, links included. Leaving links attached would keep
+ /// the programs running with nothing left to update them. The lock under `/run` stays,
+ /// unlinking it under a waiter would let that waiter run unserialized.
+ pub fn clear(&self) -> anyhow::Result<()> {
+ self.programs.tear_down()
+ }
+}
diff --git a/src/tc.rs b/src/tc.rs
index 05d823e..63d3fbf 100644
--- a/src/tc.rs
+++ b/src/tc.rs
@@ -17,7 +17,7 @@ use aya::{
/// Why a link step did not go through, with the kernel's own answer where it gives one, so a
/// caller can tell a lost race or a vanished interface from a real failure.
#[derive(Debug)]
-pub enum Error {
+pub(crate) enum Error {
/// A link of the program is in place already, another attach of the interface won the race.
AlreadyAttached,
/// The interface is gone, its index names no netdev anymore.
@@ -176,7 +176,7 @@ impl LinkPin {
/// verdict ends the chain right there. So a subsystem's program returns `TC_ACT_UNSPEC` for
/// whatever it leaves alone. Programs of several subsystems run in attach order, each new link
/// goes last.
-pub fn attach_and_pin(
+pub(crate) fn attach_and_pin(
prog: &mut SchedClassifier,
pin: &LinkPin,
pin_path: &Path,
@@ -201,7 +201,7 @@ pub fn attach_and_pin(
}
/// Rebind a pinned link to `prog` via LINK_UPDATE. Atomic, traffic sees no detach/reattach gap.
-pub fn swap_pinned_link(prog: &mut SchedClassifier, pin_path: &Path) -> Result<(), Error> {
+pub(crate) fn swap_pinned_link(prog: &mut SchedClassifier, pin_path: &Path) -> Result<(), Error> {
let pinned = PinnedLink::from_pin(pin_path)?;
let fd_link: FdLink = pinned.into();
let link = fd_link.try_into()?;
@@ -211,7 +211,7 @@ pub fn swap_pinned_link(prog: &mut SchedClassifier, pin_path: &Path) -> Result<(
Ok(())
}
-pub fn detach_pinned_link(pin_path: &Path) -> Result<(), Error> {
+pub(crate) fn detach_pinned_link(pin_path: &Path) -> Result<(), Error> {
let pinned = PinnedLink::from_pin(pin_path)?;
let _fd_link = pinned.unpin()?;
Ok(())
@@ -219,7 +219,7 @@ pub fn detach_pinned_link(pin_path: &Path) -> Result<(), Error> {
/// The name of an interface by index, none when it is gone. Any other failure says nothing
/// about the interface and is not taken as its absence, so the lookup reads errno itself.
-pub fn iface_name(ifindex: u32) -> anyhow::Result<Option<String>> {
+pub(crate) fn iface_name(ifindex: u32) -> anyhow::Result<Option<String>> {
let mut buf = [0u8; nix::libc::IF_NAMESIZE];
// SAFETY: the buffer holds IF_NAMESIZE bytes, the most the call writes
let name = unsafe { nix::libc::if_indextoname(ifindex, buf.as_mut_ptr().cast()) };
@@ -237,7 +237,7 @@ pub fn iface_name(ifindex: u32) -> anyhow::Result<Option<String>> {
/// The pins in `links_dir`, see [`LinkPin`], none for a missing directory. A file that is not a
/// pin is skipped with a warning.
-pub fn read_pinned_links(links_dir: &Path) -> anyhow::Result<Vec<LinkPin>> {
+pub(crate) fn read_pinned_links(links_dir: &Path) -> anyhow::Result<Vec<LinkPin>> {
let mut out = Vec::new();
let dir = match std::fs::read_dir(links_dir) {
Ok(d) => d,
--
2.47.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH proxmox-ebpf v4 3/5] subsystem: place the links of different subsystems in a fixed order
2026-10-19 14:24 [PATCH proxmox-ebpf v4 0/5] add proxmox-ebpf library Hannes Laimer
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 1/5] tc: add the link plumbing Hannes Laimer
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 2/5] subsystem: add the shared tc program handling Hannes Laimer
@ 2026-10-19 14:24 ` Hannes Laimer
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 4/5] tests: add a native harness for the BPF C programs Hannes Laimer
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 5/5] debian: package the crate as a Rust library Hannes Laimer
4 siblings, 0 replies; 6+ messages in thread
From: Hannes Laimer @ 2026-10-19 14:24 UTC (permalink / raw)
To: pve-devel
tcx runs the programs on an interface in the order their links were
attached, so without a fixed order the subsystem that sees a packet
first is whichever attached there first. The run order is a list in
the lib. An attach places its link relative to the links the other
subsystems hold on that interface, found through their pins, under
one lock for all subsystems so two attaches to one interface see each
other. Swaps keep a link's slot.
A subsystem's place comes from the list at compile time, and a name
missing from the list fails the build instead of landing somewhere in
the chain.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
src/subsystem.rs | 203 ++++++++++++++++++++++++++++++++++++++++++++++-
src/tc.rs | 26 ++++--
2 files changed, 217 insertions(+), 12 deletions(-)
diff --git a/src/subsystem.rs b/src/subsystem.rs
index c740fa8..3bf1634 100644
--- a/src/subsystem.rs
+++ b/src/subsystem.rs
@@ -11,6 +11,11 @@
//! single-interface step shared. The holder reads the input of its change under the lock and
//! writes before it lets go, so a full apply and every step reach the kernel in the order they
//! read their input. Two steps of one interface do not order each other.
+//!
+//! The programs of different subsystems on one interface run in the order of [`RUN_ORDER`]. An
+//! attach places its link among the links the other subsystems hold on that interface, which it
+//! finds through their pins. It does that under a lock all subsystems share, so each attach sees
+//! the links pinned before it.
use std::{
collections::HashSet,
fs::File,
@@ -21,7 +26,10 @@ use std::{
};
use anyhow::Context;
-use aya::{EbpfLoader, programs::SchedClassifier};
+use aya::{
+ EbpfLoader,
+ programs::{SchedClassifier, links::LinkOrder},
+};
use nix::{
errno::Errno,
fcntl::{Flock, FlockArg},
@@ -32,6 +40,36 @@ use crate::tc::{self, Direction};
const PIN_ROOT: &str = "/sys/fs/bpf/proxmox-ebpf";
const RUN_ROOT: &str = "/run/proxmox-ebpf";
+/// The subsystems in the order their programs run on an interface, the first runs first. A name
+/// missing from the list fails the build.
+pub const RUN_ORDER: &[&str] = &[];
+
+const fn rank_of(name: &str) -> usize {
+ let mut i = 0;
+ while i < RUN_ORDER.len() {
+ if str_eq(RUN_ORDER[i], name) {
+ return i;
+ }
+ i += 1;
+ }
+ panic!("subsystem not in RUN_ORDER");
+}
+
+const fn str_eq(a: &str, b: &str) -> bool {
+ let (a, b) = (a.as_bytes(), b.as_bytes());
+ if a.len() != b.len() {
+ return false;
+ }
+ let mut i = 0;
+ while i < a.len() {
+ if a[i] != b[i] {
+ return false;
+ }
+ i += 1;
+ }
+ true
+}
+
const fn obj_fingerprint(obj: &[u8]) -> u64 {
let mut hash = 0xcbf29ce484222325u64;
let mut i = 0;
@@ -44,7 +82,7 @@ const fn obj_fingerprint(obj: &[u8]) -> u64 {
}
/// A subsystem's [`TcPrograms`] from its name. The object is the build's `<name>.bpf.o`, the
-/// fingerprint follows from the bytes.
+/// rank and the fingerprint follow from the name and the bytes.
#[macro_export]
macro_rules! tc_programs {
(
@@ -121,6 +159,8 @@ impl std::error::Error for NotLoaded {}
pub struct TcPrograms {
/// The subsystem's name, its directory under the pin root.
name: &'static str,
+ /// Its place in [`RUN_ORDER`].
+ rank: usize,
/// The embedded BPF object.
obj: &'static [u8],
/// FNV-1a of `obj`, the programs are pinned under it.
@@ -145,6 +185,7 @@ impl TcPrograms {
) -> Self {
Self {
name,
+ rank: rank_of(name),
obj,
fingerprint: obj_fingerprint(obj),
programs,
@@ -221,6 +262,99 @@ impl TcPrograms {
})
}
+ // one lock for every subsystem. An attach holds it from the neighbor lookup to the pin of its
+ // link, so two attaches to one interface see each other. An install holds it while it moves
+ // its pins to a new schema directory, so a lookup never misses a link in the middle of the
+ // move
+ fn attach_lock(&self) -> anyhow::Result<Flock<File>> {
+ std::fs::create_dir_all(RUN_ROOT)?;
+ let path = Path::new(RUN_ROOT).join("attach.lock");
+ flock_within(&path, true, "attach lock", self.name)
+ }
+
+ // the links the other subsystems hold on the interface and direction of `pin`, with their
+ // ranks. A pin has the same file name in every subsystem, so finding one is a stat of that
+ // name in each of their schema directories, their links directories are never listed
+ fn neighbors(&self, pin: &tc::LinkPin) -> anyhow::Result<Vec<(usize, PathBuf)>> {
+ let mut found = Vec::new();
+ for (rank, name) in RUN_ORDER.iter().enumerate() {
+ if *name == self.name {
+ continue;
+ }
+ let schemas = match std::fs::read_dir(Path::new(PIN_ROOT).join(name)) {
+ Ok(schemas) => schemas,
+ Err(e) if e.kind() == ErrorKind::NotFound => continue,
+ Err(e) => return Err(e).with_context(|| format!("list the pins of {name}")),
+ };
+ for schema in schemas {
+ let path = schema_links_dir(&schema?.path()).join(pin.filename());
+ if path.exists() {
+ found.push((rank, path));
+ }
+ }
+ }
+ Ok(found)
+ }
+
+ // attach `prog` for `pin` in its place among the other subsystems' links, see `RUN_ORDER`,
+ // and pin it under `pin_path`
+ fn attach_link(
+ &self,
+ prog: &mut SchedClassifier,
+ pin: &tc::LinkPin,
+ pin_path: &Path,
+ ) -> Result<(), tc::Error> {
+ let _lock = self.attach_lock()?;
+ // under the lock pins only go away, other attaches and the pin moves of an install wait
+ // on it. So a neighbor that cannot be placed against is dropped and the attach tries the
+ // rest without reading the list again
+ let mut neighbors = self.neighbors(pin)?;
+ loop {
+ let (index, before) = match place(self.rank, &neighbors) {
+ Place::Before(i) => (Some(i), true),
+ Place::After(i) => (Some(i), false),
+ Place::Last => (None, false),
+ };
+ // the order holds only the neighbor's fd, the link stays open until the attach is done
+ let neighbor = match index {
+ Some(i) => match tc::open_pinned_link(&neighbors[i].1) {
+ Ok(link) => Some(link),
+ // unpinned since the lookup, its subsystem detached it
+ Err(tc::Error::PinGone) => {
+ neighbors.remove(i);
+ continue;
+ }
+ Err(e) => return Err(e),
+ },
+ None => None,
+ };
+ let order = match (&neighbor, before) {
+ (Some(link), true) => LinkOrder::before_link(link)?,
+ (Some(link), false) => LinkOrder::after_link(link)?,
+ (None, _) => LinkOrder::last(),
+ };
+ let attached = tc::attach_and_pin(prog, pin, pin_path, order);
+ let Some(i) = index else {
+ return attached;
+ };
+ match attached {
+ // a neighbor's pin may name a link whose interface is gone under a reused index,
+ // nothing can be placed against that one until its subsystem reclaims it. The
+ // interface is better served out of order than not at all
+ Err(tc::Error::NotAttached) => {
+ log::warn!(
+ "{}: {} not placed against {}, that link is not attached",
+ self.name,
+ pin.filename(),
+ neighbors[i].1.display()
+ );
+ neighbors.remove(i);
+ }
+ result => return result,
+ }
+ }
+ }
+
/// Whether a link of the interface name is pinned, read without the lock. Most interfaces
/// never had a link and their unplug must not wait behind a full apply. A miss is a snapshot,
/// a pin arriving after it is another attach's. A failed listing counts as a hit. So do links
@@ -417,6 +551,7 @@ impl TcPrograms {
// their interfaces never run without a program and keep their place in the tcx order.
// One this build has no program for, or that already has a link here, is detached
// instead. A rebound link whose pin cannot be moved stays where it is
+ let moving = self.attach_lock()?;
for links_dir in old_links {
for pin in tc::read_pinned_links(links_dir)? {
let old = links_dir.join(pin.filename());
@@ -449,6 +584,7 @@ impl TcPrograms {
}
}
}
+ drop(moving);
// a run that died after pinning some of the directions left a partial set behind
match std::fs::remove_dir_all(self.prog_dir()) {
@@ -604,7 +740,7 @@ impl TcPrograms {
continue;
};
let pin = tc::LinkPin { ifindex, name, dir };
- match tc::attach_and_pin(prog, &pin, &self.link_pin_path(&pin)) {
+ match self.attach_link(prog, &pin, &self.link_pin_path(&pin)) {
Ok(()) => {}
Err(tc::Error::IfaceGone) => {
log::info!("{}: {ifindex} vanished before the attach", self.name);
@@ -674,7 +810,7 @@ impl TcPrograms {
if path.exists() {
continue;
}
- match tc::attach_and_pin(prog, &pin, &path) {
+ match self.attach_link(prog, &pin, &path) {
Ok(()) => {}
Err(tc::Error::IfaceGone) => {
log::info!("{}: {ifindex} vanished before the attach", self.name);
@@ -706,6 +842,31 @@ fn links_pinned_under(schemas: &[(PathBuf, Option<u32>)]) -> bool {
.any(|(dir, _)| any_links(&schema_links_dir(dir)))
}
+// where a new link goes among the neighbors' links, the index names the neighbor
+enum Place {
+ Before(usize),
+ After(usize),
+ Last,
+}
+
+// before the earliest neighbor that runs later, otherwise after the latest one that runs
+// earlier. Each attach inserts into a chain sorted this way, so the chain stays sorted
+fn place(rank: usize, neighbors: &[(usize, PathBuf)]) -> Place {
+ let ranked = neighbors.iter().map(|(r, _)| *r).enumerate();
+ let later = ranked
+ .clone()
+ .filter(|&(_, r)| r > rank)
+ .min_by_key(|&(_, r)| r);
+ if let Some((i, _)) = later {
+ return Place::Before(i);
+ }
+ let earlier = ranked.filter(|&(_, r)| r < rank).max_by_key(|&(_, r)| r);
+ match earlier {
+ Some((i, _)) => Place::After(i),
+ None => Place::Last,
+ }
+}
+
// whether link pins remain in a links directory. Debris of another kind does not count, it goes
// with the directory. One that cannot be read may hold anything, a missing one is empty
fn any_links(links_dir: &Path) -> bool {
@@ -827,3 +988,37 @@ impl ApplyLock<'_, Exclusive> {
self.programs.tear_down()
}
}
+
+#[cfg(test)]
+mod test {
+ use super::*;
+
+ fn placed(rank: usize, ranks: &[usize]) -> String {
+ let neighbors: Vec<(usize, PathBuf)> = ranks
+ .iter()
+ .map(|&r| (r, PathBuf::from(format!("link-{r}"))))
+ .collect();
+ match place(rank, &neighbors) {
+ Place::Before(i) => format!("before {}", neighbors[i].1.display()),
+ Place::After(i) => format!("after {}", neighbors[i].1.display()),
+ Place::Last => "last".into(),
+ }
+ }
+
+ #[test]
+ fn a_link_goes_before_the_earliest_later_neighbor() {
+ assert_eq!(placed(1, &[3, 2]), "before link-2");
+ assert_eq!(placed(1, &[0, 3, 2]), "before link-2");
+ }
+
+ #[test]
+ fn a_link_goes_after_the_latest_earlier_neighbor() {
+ assert_eq!(placed(2, &[0, 1]), "after link-1");
+ assert_eq!(placed(3, &[1, 0]), "after link-1");
+ }
+
+ #[test]
+ fn a_link_without_neighbors_goes_last() {
+ assert_eq!(placed(1, &[]), "last");
+ }
+}
diff --git a/src/tc.rs b/src/tc.rs
index 63d3fbf..15da990 100644
--- a/src/tc.rs
+++ b/src/tc.rs
@@ -10,7 +10,7 @@ use aya::{
programs::{
ProgramError, SchedClassifier, TcAttachType,
links::{FdLink, LinkError, LinkOrder, PinnedLink},
- tc::TcAttachOptions,
+ tc::{SchedClassifierLink, TcAttachOptions},
},
};
@@ -24,6 +24,9 @@ pub(crate) enum Error {
IfaceGone,
/// The pin was gone by the time it was opened or removed, another attach or unplug took it.
PinGone,
+ /// The link the call referred to is not attached to the interface, the neighbor of a
+ /// placement or the link of a swap.
+ NotAttached,
Other(anyhow::Error),
}
@@ -33,6 +36,7 @@ impl std::fmt::Display for Error {
Self::AlreadyAttached => f.write_str("link already in place"),
Self::IfaceGone => f.write_str("interface is gone"),
Self::PinGone => f.write_str("pin is gone"),
+ Self::NotAttached => f.write_str("link not attached"),
Self::Other(e) if f.alternate() => write!(f, "{e:#}"),
Self::Other(e) => write!(f, "{e}"),
}
@@ -61,6 +65,9 @@ impl From<ProgramError> for Error {
ProgramError::SyscallError(s) if s.io_error.kind() == ErrorKind::AlreadyExists => {
Self::AlreadyAttached
}
+ ProgramError::SyscallError(s) if s.io_error.kind() == ErrorKind::NotFound => {
+ Self::NotAttached
+ }
e => Self::Other(e.into()),
}
}
@@ -174,18 +181,19 @@ impl LinkPin {
/// filters on the interface's qdiscs, such as the `ingress` qdisc the rate limiter installs.
/// `TC_ACT_UNSPEC` hands a packet on to the next program and then to the filters. Any other
/// verdict ends the chain right there. So a subsystem's program returns `TC_ACT_UNSPEC` for
-/// whatever it leaves alone. Programs of several subsystems run in attach order, each new link
-/// goes last.
+/// whatever it leaves alone. `order` is the link's place among the links already there, see
+/// [`RUN_ORDER`](crate::subsystem::RUN_ORDER).
pub(crate) fn attach_and_pin(
prog: &mut SchedClassifier,
pin: &LinkPin,
pin_path: &Path,
+ order: LinkOrder,
) -> Result<(), Error> {
let link_id = prog
.attach_with_options(
&pin.name,
pin.dir.aya_type(),
- TcAttachOptions::TcxOrder(LinkOrder::last()),
+ TcAttachOptions::TcxOrder(order),
)
.map_err(|e| match Error::from(e) {
// the interface can go away between the name lookup and the attach
@@ -200,12 +208,14 @@ pub(crate) fn attach_and_pin(
Ok(())
}
+pub(crate) fn open_pinned_link(pin_path: &Path) -> Result<SchedClassifierLink, Error> {
+ let fd_link: FdLink = PinnedLink::from_pin(pin_path)?.into();
+ Ok(fd_link.try_into()?)
+}
+
/// Rebind a pinned link to `prog` via LINK_UPDATE. Atomic, traffic sees no detach/reattach gap.
pub(crate) fn swap_pinned_link(prog: &mut SchedClassifier, pin_path: &Path) -> Result<(), Error> {
- let pinned = PinnedLink::from_pin(pin_path)?;
- let fd_link: FdLink = pinned.into();
- let link = fd_link.try_into()?;
- let new_id = prog.attach_to_link(link)?;
+ let new_id = prog.attach_to_link(open_pinned_link(pin_path)?)?;
// take the handle out of aya's internal tracking, we have the pin
let _ = prog.take_link(new_id)?;
Ok(())
--
2.47.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH proxmox-ebpf v4 4/5] tests: add a native harness for the BPF C programs
2026-10-19 14:24 [PATCH proxmox-ebpf v4 0/5] add proxmox-ebpf library Hannes Laimer
` (2 preceding siblings ...)
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 3/5] subsystem: place the links of different subsystems in a fixed order Hannes Laimer
@ 2026-10-19 14:24 ` Hannes Laimer
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 5/5] debian: package the crate as a Rust library Hannes Laimer
4 siblings, 0 replies; 6+ messages in thread
From: Hannes Laimer @ 2026-10-19 14:24 UTC (permalink / raw)
To: pve-devel
The programs' packet parsing and rewriting is plain C whose behavior
does not depend on the compilation target. So the build compiles each
one a second time natively, against shim headers that turn the
helpers into plain extern functions. A test hands a program a crafted
frame and asserts on the rewritten bytes.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
build.rs | 62 ++++++++++
src/bpf-shim/bpf/bpf_endian.h | 12 ++
src/bpf-shim/bpf/bpf_helpers.h | 34 ++++++
src/bpf-shim/bpf_debug.h | 10 ++
src/bpf-shim/vmlinux.h | 70 +++++++++++
tests/common/mod.rs | 207 +++++++++++++++++++++++++++++++++
tests/harness.rs | 65 +++++++++++
7 files changed, 460 insertions(+)
create mode 100644 src/bpf-shim/bpf/bpf_endian.h
create mode 100644 src/bpf-shim/bpf/bpf_helpers.h
create mode 100644 src/bpf-shim/bpf_debug.h
create mode 100644 src/bpf-shim/vmlinux.h
create mode 100644 tests/common/mod.rs
create mode 100644 tests/harness.rs
diff --git a/build.rs b/build.rs
index 270a97f..81c2ce4 100644
--- a/build.rs
+++ b/build.rs
@@ -15,6 +15,9 @@ fn main() {
println!("cargo:rerun-if-changed=build.rs");
println!("cargo:rerun-if-env-changed=CARGO_FEATURE_BPF_DEBUG");
+ let shim_include = src_dir.join("bpf-shim");
+ let mut native_objs = Vec::new();
+
for sub_entry in std::fs::read_dir(&src_dir).expect("read src/") {
let sub_path = sub_entry.expect("read src/ entry").path();
let bpf_dir = sub_path.join("bpf");
@@ -43,8 +46,67 @@ fn main() {
continue;
}
compile_bpf(&path, &bpf_dir, &global_include, &out_dir, bpf_debug);
+ native_objs.push(compile_native(
+ &path,
+ &bpf_dir,
+ &global_include,
+ &shim_include,
+ &out_dir,
+ ));
}
}
+
+ // archived so a member is only linked when a test references its symbols, the regular
+ // build neither grows nor needs the mocked helpers
+ let archive = out_dir.join("libbpf_native.a");
+ let _ = std::fs::remove_file(&archive);
+ let status = Command::new("ar")
+ .arg("rcs")
+ .arg(&archive)
+ .args(&native_objs)
+ .status()
+ .expect("failed to invoke ar");
+ assert!(
+ status.success(),
+ "ar failed to create {}",
+ archive.display()
+ );
+ println!("cargo:rustc-link-search=native={}", out_dir.display());
+ println!("cargo:rustc-link-lib=static=bpf_native");
+}
+
+// native build against the shim headers, so tests call the program as a plain function
+fn compile_native(
+ src: &Path,
+ local_include: &Path,
+ global_include: &Path,
+ shim_include: &Path,
+ out_dir: &Path,
+) -> PathBuf {
+ let stem = src.file_stem().unwrap().to_str().unwrap();
+ let obj_path = out_dir.join(format!("{stem}.native.o"));
+
+ let status = Command::new("clang")
+ .args(["-O2", "-g", "-Wall", "-fPIC"])
+ // shim first, it shadows vmlinux.h and the bpf headers
+ .arg("-I")
+ .arg(shim_include)
+ .arg("-I")
+ .arg(global_include)
+ .arg("-I")
+ .arg(local_include)
+ .arg("-c")
+ .arg(src)
+ .arg("-o")
+ .arg(&obj_path)
+ .status()
+ .expect("failed to invoke clang, is it installed?");
+ assert!(
+ status.success(),
+ "clang failed to compile {} natively",
+ src.display()
+ );
+ obj_path
}
fn compile_bpf(
diff --git a/src/bpf-shim/bpf/bpf_endian.h b/src/bpf-shim/bpf/bpf_endian.h
new file mode 100644
index 0000000..ebcbceb
--- /dev/null
+++ b/src/bpf-shim/bpf/bpf_endian.h
@@ -0,0 +1,12 @@
+#ifndef PROXMOX_EBPF_SHIM_BPF_ENDIAN_H
+#define PROXMOX_EBPF_SHIM_BPF_ENDIAN_H
+
+// Native stand-in for <bpf/bpf_endian.h>, little-endian hosts only (the
+// shim vmlinux.h enforces that).
+
+#define bpf_htons(x) __builtin_bswap16(x)
+#define bpf_ntohs(x) __builtin_bswap16(x)
+#define bpf_htonl(x) __builtin_bswap32(x)
+#define bpf_ntohl(x) __builtin_bswap32(x)
+
+#endif
diff --git a/src/bpf-shim/bpf/bpf_helpers.h b/src/bpf-shim/bpf/bpf_helpers.h
new file mode 100644
index 0000000..de3bd84
--- /dev/null
+++ b/src/bpf-shim/bpf/bpf_helpers.h
@@ -0,0 +1,34 @@
+#ifndef PROXMOX_EBPF_SHIM_BPF_HELPERS_H
+#define PROXMOX_EBPF_SHIM_BPF_HELPERS_H
+
+// Native stand-in for <bpf/bpf_helpers.h>. The helpers become plain extern
+// functions the test harness provides. Sections reduce to weak linkage so the
+// program objects can be linked into one test binary. Weak symbols of one
+// name link silently, so program and map names must stay distinct across
+// subsystems. The map definition macros keep their libbpf shapes, which are
+// plain C already.
+
+#define SEC(name) __attribute__((weak))
+#define __always_inline inline __attribute__((always_inline))
+#define __uint(name, val) int(*name)[val]
+#define __type(name, val) typeof(val) *name
+
+#ifndef barrier_var
+#define barrier_var(var) asm volatile("" : "+r"(var))
+#endif
+
+// map-definition constants, the values are irrelevant natively
+enum {
+ BPF_MAP_TYPE_HASH = 1,
+};
+#define BPF_F_NO_PREALLOC 1
+#define LIBBPF_PIN_BY_NAME 1
+
+extern void *bpf_map_lookup_elem(void *map, const void *key);
+extern long bpf_skb_load_bytes(const void *skb, __u32 offset, void *to, __u32 len);
+extern long bpf_skb_store_bytes(void *skb, __u32 offset, const void *from, __u32 len, __u64 flags);
+extern long bpf_skb_pull_data(void *skb, __u32 len);
+extern long bpf_skb_change_tail(void *skb, __u32 new_len, __u64 flags);
+extern long bpf_redirect(__u32 ifindex, __u64 flags);
+
+#endif
diff --git a/src/bpf-shim/bpf_debug.h b/src/bpf-shim/bpf_debug.h
new file mode 100644
index 0000000..a7e5eac
--- /dev/null
+++ b/src/bpf-shim/bpf_debug.h
@@ -0,0 +1,10 @@
+#ifndef PROXMOX_EBPF_SHIM_BPF_DEBUG_H
+#define PROXMOX_EBPF_SHIM_BPF_DEBUG_H
+
+// Native stand-in for bpf_debug.h, debug printing is a no-op in tests.
+
+#define DBG(...) \
+ do { \
+ } while (0)
+
+#endif
diff --git a/src/bpf-shim/vmlinux.h b/src/bpf-shim/vmlinux.h
new file mode 100644
index 0000000..99a7aaf
--- /dev/null
+++ b/src/bpf-shim/vmlinux.h
@@ -0,0 +1,70 @@
+#ifndef PROXMOX_EBPF_SHIM_VMLINUX_H
+#define PROXMOX_EBPF_SHIM_VMLINUX_H
+
+// Native stand-in for vmlinux.h, only the types the programs touch. The
+// header structs match the kernel's on a little-endian host.
+
+#if __BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__
+#error "the native BPF test build assumes a little-endian host"
+#endif
+
+typedef unsigned char __u8;
+typedef unsigned short __u16;
+typedef unsigned int __u32;
+typedef unsigned long long __u64;
+typedef __u16 __be16;
+typedef __u32 __be32;
+
+// data and data_end are wide enough for native pointers, keep in sync with SkBuff in
+// tests/common/mod.rs
+struct __sk_buff {
+ unsigned long data;
+ unsigned long data_end;
+ __u32 len;
+ __u32 ifindex;
+ __u32 mark;
+};
+
+struct ethhdr {
+ __u8 h_dest[6];
+ __u8 h_source[6];
+ __be16 h_proto;
+} __attribute__((packed));
+
+struct iphdr {
+ __u8 ihl : 4;
+ __u8 version : 4;
+ __u8 tos;
+ __be16 tot_len;
+ __be16 id;
+ __be16 frag_off;
+ __u8 ttl;
+ __u8 protocol;
+ __u16 check;
+ __be32 saddr;
+ __be32 daddr;
+};
+
+struct udphdr {
+ __be16 source;
+ __be16 dest;
+ __be16 len;
+ __u16 check;
+};
+
+struct in6_addr {
+ __u8 s6_addr[16];
+};
+
+struct ipv6hdr {
+ __u8 priority : 4;
+ __u8 version : 4;
+ __u8 flow_lbl[3];
+ __be16 payload_len;
+ __u8 nexthdr;
+ __u8 hop_limit;
+ struct in6_addr saddr;
+ struct in6_addr daddr;
+};
+
+#endif
diff --git a/tests/common/mod.rs b/tests/common/mod.rs
new file mode 100644
index 0000000..67fac33
--- /dev/null
+++ b/tests/common/mod.rs
@@ -0,0 +1,207 @@
+//! Native harness for the subsystems' BPF C programs. A test declares a program's symbol and
+//! calls it like a plain function on a fake skb, with the helpers as bounds-checked functions
+//! over its buffer and a per-thread map registry.
+
+// every test crate compiles the harness and uses its own part of it
+#![allow(dead_code)]
+
+// the archive is linked through the lib, a test crate that never uses the lib would drop it from
+// the link together with the program symbols
+use proxmox_ebpf as _;
+
+use std::cell::RefCell;
+use std::collections::HashMap;
+use std::ffi::{c_int, c_long, c_void};
+
+pub const TC_ACT_UNSPEC: c_int = -1;
+pub const TC_ACT_OK: c_int = 0;
+pub const TC_ACT_REDIRECT: c_int = 7;
+
+pub const PKT_CAP: usize = 2048;
+
+/// Keep in sync with struct __sk_buff in src/bpf-shim/vmlinux.h.
+#[repr(C)]
+pub struct SkBuff {
+ pub data: usize,
+ pub data_end: usize,
+ pub len: u32,
+ pub ifindex: u32,
+ pub mark: u32,
+}
+
+/// The skb handed to a program plus the buffer behind it. The helpers get the skb pointer and
+/// cast back to this, so the skb must stay the first field.
+#[repr(C)]
+pub struct TestSkb {
+ skb: SkBuff,
+ buf: Box<[u8; PKT_CAP]>,
+}
+
+impl TestSkb {
+ pub fn new(packet: &[u8], ifindex: u32) -> Self {
+ assert!(packet.len() <= PKT_CAP);
+ let mut buf = Box::new([0u8; PKT_CAP]);
+ buf[..packet.len()].copy_from_slice(packet);
+ let mut t = TestSkb {
+ skb: SkBuff {
+ data: 0,
+ data_end: 0,
+ len: packet.len() as u32,
+ ifindex,
+ mark: 0,
+ },
+ buf,
+ };
+ t.sync();
+ t
+ }
+
+ fn sync(&mut self) {
+ let base = self.buf.as_ptr() as usize;
+ self.skb.data = base;
+ self.skb.data_end = base + self.skb.len as usize;
+ }
+
+ pub fn packet(&self) -> &[u8] {
+ &self.buf[..self.skb.len as usize]
+ }
+
+ pub fn run(&mut self, prog: unsafe extern "C" fn(*mut SkBuff) -> c_int) -> c_int {
+ REDIRECTED.with(|r| *r.borrow_mut() = None);
+ unsafe { prog((self as *mut TestSkb).cast()) }
+ }
+}
+
+struct MockMap {
+ key_size: usize,
+ entries: HashMap<Vec<u8>, Box<[u8]>>,
+}
+
+thread_local! {
+ static MAPS: RefCell<HashMap<usize, MockMap>> = RefCell::new(HashMap::new());
+ static REDIRECTED: RefCell<Option<(u32, u64)>> = const { RefCell::new(None) };
+}
+
+pub fn register_map(map: *const c_void, key_size: usize) {
+ MAPS.with(|maps| {
+ maps.borrow_mut().insert(
+ map as usize,
+ MockMap {
+ key_size,
+ entries: HashMap::new(),
+ },
+ )
+ });
+}
+
+pub fn map_insert(map: *const c_void, key: &[u8], value: &[u8]) {
+ MAPS.with(|maps| {
+ let mut maps = maps.borrow_mut();
+ let m = maps.get_mut(&(map as usize)).expect("map not registered");
+ assert_eq!(key.len(), m.key_size);
+ m.entries.insert(key.to_vec(), value.into());
+ });
+}
+
+/// The ifindex of the redirect the last run issued, if any.
+pub fn redirected() -> Option<u32> {
+ REDIRECTED.with(|r| r.borrow().map(|(ifindex, _)| ifindex))
+}
+
+/// The flags of the redirect the last run issued, zero sends out of the interface.
+pub fn redirect_flags() -> Option<u64> {
+ REDIRECTED.with(|r| r.borrow().map(|(_, flags)| flags))
+}
+
+unsafe fn testskb<'a>(skb: *mut c_void) -> &'a mut TestSkb {
+ unsafe { &mut *(skb as *mut TestSkb) }
+}
+
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_map_lookup_elem(map: *mut c_void, key: *const c_void) -> *mut c_void {
+ MAPS.with(|maps| {
+ let mut maps = maps.borrow_mut();
+ let Some(m) = maps.get_mut(&(map as usize)) else {
+ panic!("lookup on unregistered map");
+ };
+ let key = unsafe { std::slice::from_raw_parts(key as *const u8, m.key_size) };
+ match m.entries.get_mut(key) {
+ // the box's heap allocation stays put while the registry holds it
+ Some(v) => v.as_mut_ptr() as *mut c_void,
+ None => std::ptr::null_mut(),
+ }
+ })
+}
+
+const EFAULT: c_long = nix::libc::EFAULT as c_long;
+const ENOMEM: c_long = nix::libc::ENOMEM as c_long;
+const EINVAL: c_long = nix::libc::EINVAL as c_long;
+
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_skb_load_bytes(
+ skb: *mut c_void,
+ offset: u32,
+ to: *mut c_void,
+ len: u32,
+) -> c_long {
+ let t = unsafe { testskb(skb) };
+ let (offset, len) = (offset as usize, len as usize);
+ if offset + len > t.skb.len as usize {
+ // the kernel clears the destination of a failed read
+ unsafe { std::ptr::write_bytes(to as *mut u8, 0, len) };
+ return -EFAULT;
+ }
+ unsafe { std::ptr::copy_nonoverlapping(t.buf.as_ptr().add(offset), to as *mut u8, len) };
+ 0
+}
+
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_skb_store_bytes(
+ skb: *mut c_void,
+ offset: u32,
+ from: *const c_void,
+ len: u32,
+ _flags: u64,
+) -> c_long {
+ let t = unsafe { testskb(skb) };
+ let (offset, len) = (offset as usize, len as usize);
+ if offset + len > t.skb.len as usize {
+ return -EFAULT;
+ }
+ unsafe {
+ std::ptr::copy_nonoverlapping(from as *const u8, t.buf.as_mut_ptr().add(offset), len)
+ };
+ 0
+}
+
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_skb_pull_data(skb: *mut c_void, len: u32) -> c_long {
+ // the buffer is always linear, pulling within it is a no-op
+ let t = unsafe { testskb(skb) };
+ if len > t.skb.len { -ENOMEM } else { 0 }
+}
+
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_skb_change_tail(skb: *mut c_void, new_len: u32, _flags: u64) -> c_long {
+ let t = unsafe { testskb(skb) };
+ // the kernel refuses to cut into the headers it knows, which is the link header here.
+ // Growing stops at the harness buffer
+ let min_len = 14;
+ if new_len < min_len || new_len as usize > PKT_CAP {
+ return -EINVAL;
+ }
+ // like the kernel, grown room reads as zeros
+ let old = t.skb.len as usize;
+ if new_len as usize > old {
+ t.buf[old..new_len as usize].fill(0);
+ }
+ t.skb.len = new_len;
+ t.sync();
+ 0
+}
+
+#[unsafe(no_mangle)]
+pub extern "C" fn bpf_redirect(ifindex: u32, flags: u64) -> c_long {
+ REDIRECTED.with(|r| *r.borrow_mut() = Some((ifindex, flags)));
+ TC_ACT_REDIRECT as c_long
+}
diff --git a/tests/harness.rs b/tests/harness.rs
new file mode 100644
index 0000000..22fdf2d
--- /dev/null
+++ b/tests/harness.rs
@@ -0,0 +1,65 @@
+//! A test target of the crate's own, so a change to the harness breaks here and not in a
+//! subsystem's tests.
+mod common;
+
+use std::ffi::{c_int, c_void};
+
+use common::*;
+
+unsafe extern "C" fn pass(_skb: *mut SkBuff) -> c_int {
+ TC_ACT_UNSPEC
+}
+
+unsafe extern "C" fn accept(_skb: *mut SkBuff) -> c_int {
+ TC_ACT_OK
+}
+
+#[test]
+fn skb_round_trips_the_frame() {
+ let frame = [1u8, 2, 3, 4];
+ let mut skb = TestSkb::new(&frame, 7);
+ assert_eq!(skb.run(pass), TC_ACT_UNSPEC);
+ assert_eq!(skb.run(accept), TC_ACT_OK);
+ assert_eq!(skb.packet(), &frame);
+ assert_eq!(redirected(), None);
+ assert_eq!(redirect_flags(), None);
+}
+
+#[test]
+fn helpers_read_and_write_the_frame() {
+ let mut skb = TestSkb::new(&[1, 2, 3, 4, 5, 6], 7);
+ let raw = (&mut skb as *mut TestSkb).cast::<c_void>();
+ let mut out = [0u8; 2];
+ assert_eq!(bpf_skb_load_bytes(raw, 2, out.as_mut_ptr().cast(), 2), 0);
+ assert_eq!(out, [3, 4]);
+ assert_eq!(
+ bpf_skb_store_bytes(raw, 4, [9u8, 9].as_ptr().cast(), 2, 0),
+ 0
+ );
+ assert_eq!(skb.packet(), &[1, 2, 3, 4, 9, 9]);
+ // past the frame, the kernel clears the destination on a failed load
+ assert_ne!(bpf_skb_load_bytes(raw, 5, out.as_mut_ptr().cast(), 2), 0);
+ assert_eq!(out, [0, 0]);
+ assert_ne!(
+ bpf_skb_store_bytes(raw, 5, [9u8, 9].as_ptr().cast(), 2, 0),
+ 0
+ );
+ assert_eq!(skb.packet(), &[1, 2, 3, 4, 9, 9]);
+}
+
+#[test]
+fn maps_answer_registered_entries() {
+ // any address serves as a map's identity, the program hands the map's address to the helper
+ let map = 0x1000 as *const c_void;
+ register_map(map, 4);
+ map_insert(map, &7u32.to_ne_bytes(), &[9, 8, 7]);
+ let key = 7u32.to_ne_bytes();
+ let hit = bpf_map_lookup_elem(map.cast_mut(), key.as_ptr().cast());
+ assert!(!hit.is_null());
+ assert_eq!(
+ unsafe { std::slice::from_raw_parts(hit as *const u8, 3) },
+ &[9, 8, 7]
+ );
+ let key = 8u32.to_ne_bytes();
+ assert!(bpf_map_lookup_elem(map.cast_mut(), key.as_ptr().cast()).is_null());
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH proxmox-ebpf v4 5/5] debian: package the crate as a Rust library
2026-10-19 14:24 [PATCH proxmox-ebpf v4 0/5] add proxmox-ebpf library Hannes Laimer
` (3 preceding siblings ...)
2026-10-19 14:24 ` [PATCH proxmox-ebpf v4 4/5] tests: add a native harness for the BPF C programs Hannes Laimer
@ 2026-10-19 14:24 ` Hannes Laimer
4 siblings, 0 replies; 6+ messages in thread
From: Hannes Laimer @ 2026-10-19 14:24 UTC (permalink / raw)
To: pve-devel
The overlay in debian/ builds it through debcargo, like the other Rust
crates. The BPF programs are compiled by the build script, so building
against the crate pulls in clang and the libbpf headers as well.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
.gitignore | 1 +
Cargo.toml | 7 ++++++
Makefile | 54 ++++++++++++++++++++++++++++++++++++++++++++
build.rs | 6 +++++
debian/changelog | 5 ++++
debian/control | 50 ++++++++++++++++++++++++++++++++++++++++
debian/copyright | 26 +++++++++++++++++++++
debian/debcargo.toml | 13 +++++++++++
debian/source/format | 1 +
9 files changed, 163 insertions(+)
create mode 100644 Makefile
create mode 100644 debian/changelog
create mode 100644 debian/control
create mode 100644 debian/copyright
create mode 100644 debian/debcargo.toml
create mode 100644 debian/source/format
diff --git a/.gitignore b/.gitignore
index 3cb8cae..574dcbd 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,3 +8,4 @@ Cargo.lock
*.changes
*.tar.?z
/proxmox-ebpf-[0-9]*/
+/build
diff --git a/Cargo.toml b/Cargo.toml
index 27010c4..1e8703d 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,7 +1,14 @@
[package]
name = "proxmox-ebpf"
version = "0.1.0"
+description = "Proxmox VE eBPF"
+authors = ["Proxmox Support Team <support@proxmox.com>"]
edition = "2024"
+rust-version = "1.88"
+license = "AGPL-3"
+homepage = "https://proxmox.com"
+
+exclude = [".cargo", ".gitignore", "Makefile", "build", "debian"]
[features]
# When enabled, BPF programs compile with -DBPF_DEBUG so their DBG() macros
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..03f0a17
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,54 @@
+include /usr/share/dpkg/pkg-info.mk
+
+CRATE := proxmox-ebpf
+BUILDDIR ?= build
+CARGO ?= cargo
+
+DSC := $(BUILDDIR)/rust-$(CRATE)_$(DEB_VERSION).dsc
+
+all: cargo-build
+
+.PHONY: cargo-build
+cargo-build:
+ $(CARGO) build --all-features
+
+.PHONY: check test
+check test:
+ $(CARGO) test --all-features
+
+# the BPF programs compile against a checked-in dump of a kernel's BTF, refreshed by hand when a
+# subsystem needs a type the dump lacks. The package build never runs this
+.PHONY: vmlinux.h
+vmlinux.h:
+ bpftool btf dump file /sys/kernel/btf/vmlinux format c > include/vmlinux.h
+
+# the generated control is copied back so the checked-in one stays current
+$(BUILDDIR)/$(CRATE): Cargo.toml build.rs src include tests debian
+ rm -rf $@
+ mkdir -p $(BUILDDIR)
+ echo system > $(BUILDDIR)/rust-toolchain
+ rm -f debian/control
+ debcargo package \
+ --config $(CURDIR)/debian/debcargo.toml \
+ --changelog-ready \
+ --no-overlay-write-back \
+ --directory $(CURDIR)/$@ \
+ $(CRATE) \
+ $(DEB_VERSION_UPSTREAM)
+ rm -f $@/debian/source/format.debcargo.hint
+ cp $@/debian/control debian/control
+
+.PHONY: deb
+deb: $(BUILDDIR)/$(CRATE)
+ cd $(BUILDDIR)/$(CRATE); dpkg-buildpackage -b -uc -us
+ lintian $(BUILDDIR)/*.deb
+
+.PHONY: dsc
+dsc: $(BUILDDIR)/$(CRATE)
+ cd $(BUILDDIR)/$(CRATE); dpkg-buildpackage -S -us -uc -d
+ lintian $(DSC)
+
+.PHONY: clean
+clean:
+ $(CARGO) clean
+ rm -rf $(BUILDDIR)
diff --git a/build.rs b/build.rs
index 81c2ce4..34652bf 100644
--- a/build.rs
+++ b/build.rs
@@ -73,6 +73,12 @@ fn main() {
);
println!("cargo:rustc-link-search=native={}", out_dir.display());
println!("cargo:rustc-link-lib=static=bpf_native");
+ // the archive holds code compiled from the crate's C sources, dh-cargo lists that in the
+ // Built-Using of a package linking it
+ println!(
+ "dh-cargo:deb-built-using=bpf_native=1={}",
+ manifest_dir.display()
+ );
}
// native build against the shim headers, so tests call the program as a plain function
diff --git a/debian/changelog b/debian/changelog
new file mode 100644
index 0000000..71e3394
--- /dev/null
+++ b/debian/changelog
@@ -0,0 +1,5 @@
+rust-proxmox-ebpf (0.1.0) trixie; urgency=medium
+
+ * Initial release.
+
+ -- Proxmox Support Team <support@proxmox.com> Wed, 02 Sep 2026 10:00:00 +0200
diff --git a/debian/control b/debian/control
new file mode 100644
index 0000000..d3a9cc8
--- /dev/null
+++ b/debian/control
@@ -0,0 +1,50 @@
+Source: rust-proxmox-ebpf
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.88) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-anyhow-1+default-dev <!nocheck>,
+ librust-aya-0.13+default-dev <!nocheck>,
+ librust-log-0.4+default-dev <!nocheck>,
+ librust-nix-0.29+default-dev <!nocheck>,
+ librust-nix-0.29+fs-dev <!nocheck>,
+ librust-nix-0.29+net-dev <!nocheck>,
+ clang <!nocheck>,
+ libbpf-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.2
+Vcs-Git: git://git.proxmox.com/git/proxmox-ebpf.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox-ebpf.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: proxmox-ebpf
+
+Package: librust-proxmox-ebpf-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-anyhow-1+default-dev,
+ librust-aya-0.13+default-dev,
+ librust-log-0.4+default-dev,
+ librust-nix-0.29+default-dev,
+ librust-nix-0.29+fs-dev,
+ librust-nix-0.29+net-dev,
+ clang,
+ libbpf-dev
+Provides:
+ librust-proxmox-ebpf+bpf-debug-dev (= ${binary:Version}),
+ librust-proxmox-ebpf+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0+bpf-debug-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1+bpf-debug-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1.0-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1.0+bpf-debug-dev (= ${binary:Version}),
+ librust-proxmox-ebpf-0.1.0+default-dev (= ${binary:Version})
+Description: Proxmox VE eBPF - Rust source code
+ Source code for Debianized Rust crate "proxmox-ebpf"
diff --git a/debian/copyright b/debian/copyright
new file mode 100644
index 0000000..42d83a3
--- /dev/null
+++ b/debian/copyright
@@ -0,0 +1,26 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+
+Files:
+ *
+Copyright: 2026 Proxmox Server Solutions GmbH <support@proxmox.com>
+License: AGPL-3.0-or-later
+ This program is free software: you can redistribute it and/or modify it under
+ the terms of the GNU Affero General Public License as published by the Free
+ Software Foundation, either version 3 of the License, or (at your option) any
+ later version.
+ .
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+ details.
+ .
+ You should have received a copy of the GNU Affero General Public License along
+ with this program. If not, see <https://www.gnu.org/licenses/>.
+
+Files:
+ include/vmlinux.h
+Copyright: The Linux kernel authors
+License: GPL-2.0-only
+ The kernel's type definitions as dumped from its BTF, see the Makefile.
+ On Debian systems the full text of the GNU General Public License version 2
+ can be found in /usr/share/common-licenses/GPL-2.
diff --git a/debian/debcargo.toml b/debian/debcargo.toml
new file mode 100644
index 0000000..a27eb72
--- /dev/null
+++ b/debian/debcargo.toml
@@ -0,0 +1,13 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+# the BPF programs are C sources of this crate, not vendored code
+whitelist = ["src/*/bpf/*.c"]
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox-ebpf.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox-ebpf.git"
+
+[packages.lib]
+depends = ["clang", "libbpf-dev"]
diff --git a/debian/source/format b/debian/source/format
new file mode 100644
index 0000000..89ae9db
--- /dev/null
+++ b/debian/source/format
@@ -0,0 +1 @@
+3.0 (native)
--
2.47.3
^ permalink raw reply related [flat|nested] 6+ messages in thread