From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id B98931FF0B3 for ; Wed, 09 Sep 2026 12:43:04 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 77B69216BA; Wed, 09 Sep 2026 12:42:12 +0200 (CEST) From: Hannes Laimer To: pve-devel@lists.proxmox.com Subject: [PATCH proxmox-ebpf v2 02/16] dhcp: add responder subsystem Date: Wed, 9 Sep 2026 12:41:30 +0200 Message-ID: <20260909104144.1110031-3-h.laimer@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260909104144.1110031-1-h.laimer@proxmox.com> References: <20260909104144.1110031-1-h.laimer@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1788950503188 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.950 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLACK 3 Contains an URL listed in the URIBL blacklist [types.rs] Message-ID-Hash: KKJQ6UV6LFA2Y2SNKSKIGSWWEPT7SFUE X-Message-ID-Hash: KKJQ6UV6LFA2Y2SNKSKIGSWWEPT7SFUE X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: The record map is fed entirely from outside, the subsystem keeps no state source of its own. A full pass makes the programs current and gives every interface named to it and found here an entry and the link it is due. The entry is a record where the interface is served with one and a marker otherwise. Bad input leaves its interface as it is. So the empty state a rebuild leaves behind is refilled by the pass for every interface the configs name. A single record or tap plug is written on its own under the shared lock and touches nothing else. It reports when nothing is loaded to write into, so the caller runs a full pass first. An unplug drops its own link and leaves a later incarnation of the name alone. The link is found by the name the pin carries, since the interface may be gone by then. Every record write carries a generation. A record change draws it after its input was written, a plug before its cache read. A record keeps the generation of the change that wrote it. So a full pass leaves alone what changed since it read its input, and an older change never overwrites a newer one. A record change writes the entry of every interface that exists here, whether it has a link yet or not. The pass writes a marker where nothing is served, so a plug still reading its input finds an entry to lose against. The configs trail the kernel around a hotplug. A tap is plugged before the guest config names its new bridge, and a generation cannot order a pass against that. So every entry also names the vnet its writer saw the interface on. A pass or a record change leaves an entry alone that names another vnet than the configs do, and its link follows the entry. A plug replaces such an entry, it is the latest word on where its interface sits. On its own vnet a plug compares by generation like every other change. A plug finding no entry writes regardless, a pass that stamped meanwhile never saw its interface. An unplug drops the entry, the interface is gone or goes away with it. An interface plugged onto a bridge this subsystem does not serve trades its entry for a marker naming that place. So a pass still reading the old vnet leaves it alone, and one reading the new place keeps it as it is. A linked interface the pass does not know at all is left alone as well. One that is gone is dropped with its entry, which a crashed guest leaves behind. Signed-off-by: Hannes Laimer --- src/dhcp/mod.rs | 1113 +++++++++++++++++++++++++++++++++++++++++++++ src/dhcp/types.rs | 80 ++++ src/lib.rs | 3 + 3 files changed, 1196 insertions(+) create mode 100644 src/dhcp/mod.rs create mode 100644 src/dhcp/types.rs diff --git a/src/dhcp/mod.rs b/src/dhcp/mod.rs new file mode 100644 index 0000000..a7e483b --- /dev/null +++ b/src/dhcp/mod.rs @@ -0,0 +1,1113 @@ +//! The dhcp subsystem, a per-tap DHCPv4 responder answering from a pinned map of one entry per +//! interface. +//! +//! The map holds nothing of its own. The SDN side hands in every guest interface with the answer +//! it gets, if any. A full pass makes the kernel match that for the interfaces it names. A linked +//! interface it does not name keeps its link and its entry. The single steps touch their own +//! interface only, so they run under the shared lock and serialize on their map writes alone. +//! +//! Two things order the writers. The first is a generation. A record change draws it after its input +//! is written, a pass and a plug draw before they read theirs. Older input never overwrites newer, +//! see [`ApplyLock::with_generation`](crate::subsystem::ApplyLock::with_generation) and +//! [`ApplyLock::change`](crate::subsystem::ApplyLock::change). The second is the vnet every entry +//! names. A hotplug plugs the tap before the guest config names the new bridge, and no generation can +//! order that. So a pass or a record change leaves an entry alone that names another vnet than the +//! configs do, and the link follows such an entry. A plug replaces it. Plugs of one interface are +//! serialized by the guest lock, so the plug is the latest word on where its interface sits. A plug +//! finding no entry writes regardless of the stamp, a pass that stamped since never saw its +//! interface. An unplugged interface takes its entry with it, one gone without a link leaves it to +//! the next pass. One plugged onto a bridge this subsystem does not serve trades its entry for a +//! marker naming that place. +mod types; + +use std::collections::{HashMap, HashSet}; +use std::io::ErrorKind; +use std::net::Ipv4Addr; + +use anyhow::{Context, bail}; +use aya::include_bytes_aligned; +use aya::maps::MapError; + +use self::types::*; +use crate::subsystem::{NotLoaded, TcPrograms}; +use crate::tc::{Direction, LinkPin}; + +/// One guest interface as the configs know it. `serve` says whether a pass links it, a plug +/// links its interface regardless. `record` is what it answers there, none answers nothing. +/// `vnet` is what its bridge is, none for a plain bridge. +pub struct Iface { + pub name: String, + pub vnet: Option, + pub serve: bool, + pub record: Option, +} + +/// One interface's answer, the address and every option the reply carries. +pub struct Record { + pub ip: Ipv4Addr, + pub prefixlen: u8, + pub server_id: Ipv4Addr, + pub lease: u32, + pub router: Option, + pub dns: Option, + pub mtu: Option, + pub mac: [u8; 6], +} + +/// What became of an unplugged interface. +pub enum Unplugged { + /// Gone, or going with the unplug. + Gone, + /// Plugged onto a bridge this subsystem does not serve, the vnet that bridge is, if any. + MovedTo(Option), +} + +/// What a single-entry step did. +#[derive(Debug, PartialEq, Eq)] +pub enum Outcome { + /// Nothing is left to do. The entry is in place, or the change was older than what is there, + /// or its interface is not here. + Done, + /// Nothing is loaded to write into, only a full pass loads, the caller runs one and retries. + NeedsFullPass, +} + +const NAME: &str = "dhcp"; +const RECORDS_MAP: &str = "dhcp_iface_records"; +const MAPS: [&str; 1] = [RECORDS_MAP]; + +const DHCP_OBJ: &[u8] = include_bytes_aligned!(concat!(env!("OUT_DIR"), "/dhcp.bpf.o")); +const DHCP_FINGERPRINT: u64 = TcPrograms::obj_fingerprint(DHCP_OBJ); + +// BUMP THIS when a semantically-incompatible change to a map definition in dhcp.bpf.c is made +const SCHEMA_VERSION: u32 = 1; + +// requests only arrive from the guest side and replies leave through a redirect, so only ingress +// carries a program +const DIRECTIONS: [Direction; 1] = [Direction::Ingress]; + +fn program_name(dir: Direction) -> &'static str { + match dir { + Direction::Ingress => "tc_dhcp_ingress", + Direction::Egress => unreachable!("dhcp is ingress-only"), + } +} + +type RecordsMap = aya::maps::HashMap; + +const PROGRAMS: TcPrograms = TcPrograms { + name: NAME, + obj: DHCP_OBJ, + fingerprint: DHCP_FINGERPRINT, + prog_name: program_name, + directions: &DIRECTIONS, + maps: &MAPS, + schema_version: SCHEMA_VERSION, +}; + +pub struct DhcpSubsystem { + programs: TcPrograms, +} + +impl Default for DhcpSubsystem { + fn default() -> Self { + Self::new() + } +} + +impl DhcpSubsystem { + pub fn new() -> Self { + Self { programs: PROGRAMS } + } + + pub fn next_generation(&self) -> anyhow::Result { + self.programs.next_generation() + } + + /// The full pass. Every interface named and found here gets its entry, a record where served + /// with one and a marker otherwise, and a link where served or where the entry names another + /// vnet. Entries of gone interfaces are dropped, and of live ones it does not name that have + /// no link. Bad input leaves its interface as it is, with a warning. The rest is left alone + /// as the module doc says. Returns whether it applied, one older than a pass already in place + /// is dropped. + pub fn apply(&self, generation: u64, ifaces: &[Iface]) -> anyhow::Result { + let lock = self.programs.lock_exclusive()?; + // loading belongs to the ordered step, a pass dropped as stale must not rebuild the + // programs with an empty map it then never fills + let applied = lock.with_generation(generation, || { + lock.ensure_loaded()?; + let present = present_ifindexes(ifaces)?; + // a link the configs do not name stays while its interface lives under the pin's + // name, a reused index is not that interface + let mut unknown_alive = HashSet::new(); + for pin in lock.attached()? { + if !present.contains_key(&pin.ifindex) && pin_alive(&pin)? { + unknown_alive.insert(pin.ifindex); + } + } + let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?; + let (elsewhere, failed) = sync_records(&mut map, &present, &unknown_alive, generation)?; + // a failed write or removal leaves its entry as it is, the links are still put right + // and every failure reported once the rest is done + let links = lock.reconcile(&wanted_links(&present, &elsewhere, &unknown_alive)); + match (failed, links) { + (0, links) => links, + (failed, Ok(())) => bail!("{failed} entries could not be changed"), + (failed, Err(e)) => { + Err(e).with_context(|| format!("{failed} entries could not be changed")) + } + } + })?; + Ok(applied.is_some()) + } + + /// A record change. Interfaces not found here are skipped. The rest are written whether they + /// have a link yet or not, so a plug still reading its input loses to it. A record the reply + /// cannot carry leaves its interface as it is. With nothing loaded the caller runs a full pass + /// first, like a plug. + pub fn update(&self, generation: u64, ifaces: &[Iface]) -> anyhow::Result { + let lock = self.programs.lock_shared()?; + if !lock.is_current() { + return Ok(Outcome::NeedsFullPass); + } + let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?; + // resolved and built before the generation lock, it covers the map writes alone + let mut failed = 0usize; + let mut entries = Vec::new(); + for iface in ifaces { + let ifindex = match ifindex_of(iface.name.as_str()) { + Ok(Some(ifindex)) => ifindex, + Ok(None) => continue, + Err(e) => { + log::warn!("dhcp: {e:#}"); + failed += 1; + continue; + } + }; + match iface.entry(generation) { + Ok(entry) => entries.push((ifindex, iface, entry)), + Err(e) => log::warn!("dhcp: {} is left as it is: {e:#}", iface.name), + } + } + lock.change(|applied| { + for (ifindex, iface, entry) in &entries { + if let Err(e) = write_entry(&mut map, applied, *ifindex, entry, Elsewhere::Skip) { + log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name); + failed += 1; + } + } + Ok(()) + })?; + if failed > 0 { + bail!("{failed} interfaces could not be updated"); + } + + Ok(Outcome::Done) + } + + /// A tap plug, the entry goes in and the program on. Nothing is loaded here, with nothing + /// pinned the caller runs a full pass first. + pub fn attach(&self, generation: u64, iface: &Iface) -> anyhow::Result { + let Some(ifindex) = ifindex_of(iface.name.as_str())? else { + log::info!("dhcp attach: {} is gone, nothing to do", iface.name); + return Ok(Outcome::Done); + }; + let lock = self.programs.lock_shared()?; + if !lock.is_current() { + return Ok(Outcome::NeedsFullPass); + } + // resolved again after the wait, a tap gone or recreated meanwhile is not this plug's + if ifindex_of(iface.name.as_str())? != Some(ifindex) { + log::info!("dhcp attach: {} went away, nothing to do", iface.name); + return Ok(Outcome::Done); + } + // the entry goes in before the program runs, a marker when there is no record or one + // the program cannot carry. A guest start must not fail on it + let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?; + let entry = match iface.entry(generation) { + Ok(entry) => entry, + Err(e) => { + log::warn!( + "dhcp: {} attaches with nothing to answer: {e:#}", + iface.name + ); + DhcpRecord::removed(iface.place()?, generation) + } + }; + lock.change(|applied| { + write_entry(&mut map, applied, ifindex, &entry, Elsewhere::Override) + })?; + match lock.attach_iface(ifindex) { + Ok(()) => Ok(Outcome::Done), + Err(e) if e.downcast_ref::().is_some() => Ok(Outcome::NeedsFullPass), + Err(e) => Err(e).with_context(|| format!("dhcp responder for {}", iface.name)), + } + } + + /// A tap unplug. The interface may be gone by then, so its link is found by the name the pin + /// carries. [`Unplugged`] decides what becomes of the entry. + pub fn detach(&self, iface: &str, unplugged: Unplugged) -> anyhow::Result<()> { + // resolved before the lock wait, an interface recreated under the name meanwhile is not + // this unplug's and keeps its link + let now = ifindex_of(iface)?; + // every guest unplug on the host lands here. Most never had a link and must not wait + // behind a pass, one that is gone as well leaves its entry to the next pass + if !self.programs.link_pinned(iface) && now.is_none() { + return Ok(()); + } + let lock = self.programs.lock_shared()?; + + if !lock.is_current() { + log::debug!("dhcp detach: nothing current, {iface} is left to the next full pass"); + return Ok(()); + } + // the map is opened and the marker built before the links go, a failure there leaves + // them attached + let mut map: RecordsMap = lock.hash_map(RECORDS_MAP)?; + let marker = match &unplugged { + Unplugged::Gone => None, + Unplugged::MovedTo(vnet) => Some(DhcpRecord::removed(place(vnet.as_deref())?, 0)), + }; + // an interface without a link has an entry as well, a marker where it is not served + let mut ifindexes = lock.detach_named(iface, |pin| { + Some(pin.ifindex) != now && pin_alive(pin).unwrap_or(false) + })?; + if let Some(now) = now.filter(|now| !ifindexes.contains(now)) { + ifindexes.push(now); + } + // the marker is the moved interface's, older incarnations under the name are gone + lock.change(|_| unplug_entries(&mut map, &ifindexes, now.zip(marker))) + } + + /// Detach everywhere and drop the pinned state, for the last zone leaving the backend. + /// Ordered like a full pass, returns whether it applied. + pub fn clear(&self, generation: u64) -> anyhow::Result { + let lock = self.programs.lock_exclusive()?; + Ok(lock.with_generation(generation, || lock.clear())?.is_some()) + } +} + +const NO_VNET: [u8; 8] = [0; 8]; + +fn unplug_entries( + map: &mut impl RecordStore, + ifindexes: &[u32], + moved: Option<(u32, DhcpRecord)>, +) -> anyhow::Result<()> { + for &ifindex in ifindexes { + match moved { + // the marker replaces an entry, an interface without one was served by nothing a + // pass could see and gets none + Some((current, marker)) if current == ifindex => { + if map.get(ifindex)?.is_some() { + map.insert(ifindex, marker)?; + } + } + _ => map.remove(ifindex)?, + } + } + Ok(()) +} + +// the vnet id padded with zeros, all zero for none +fn place(vnet: Option<&str>) -> anyhow::Result<[u8; 8]> { + let Some(vnet) = vnet else { + return Ok(NO_VNET); + }; + let bytes = vnet.as_bytes(); + if bytes.is_empty() || bytes.len() > NO_VNET.len() { + bail!("vnet id {vnet:?} does not fit a record"); + } + let mut place = NO_VNET; + place[..bytes.len()].copy_from_slice(bytes); + Ok(place) +} + +// the record map as the writers see it, so their decisions run against a plain map in tests +trait RecordStore { + fn get(&self, ifindex: u32) -> anyhow::Result>; + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()>; + // a missing entry is removed already + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()>; + fn entries(&self) -> anyhow::Result>; +} + +impl RecordStore for RecordsMap { + fn get(&self, ifindex: u32) -> anyhow::Result> { + match aya::maps::HashMap::get(self, &ifindex, 0) { + Ok(live) => Ok(Some(live)), + Err(MapError::KeyNotFound) => Ok(None), + Err(e) => Err(e.into()), + } + } + + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> { + aya::maps::HashMap::insert(self, ifindex, rec, 0)?; + Ok(()) + } + + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> { + match aya::maps::HashMap::remove(self, &ifindex) { + Ok(()) => Ok(()), + // the kernel answers a missing key with ENOENT, aya passes that through as is + Err(MapError::SyscallError(e)) if e.io_error.kind() == ErrorKind::NotFound => Ok(()), + Err(e) => Err(e.into()), + } + } + + // a record the listing cannot read is one the pass cannot compare against, so the pass + // fails rather than writes past it + fn entries(&self) -> anyhow::Result> { + self.iter() + .collect::, _>>() + .context("listing the records") + } +} + +// what a write does with an entry naming another vnet than its own +enum Elsewhere { + // a record change, its configs are stale for the interface + Skip, + // a plug, the latest word on where the interface sits + Override, +} + +// a plug finding nothing writes regardless of the stamp, a pass that stamped since never saw its +// interface +fn write_entry( + map: &mut impl RecordStore, + applied: Option, + ifindex: u32, + rec: &DhcpRecord, + elsewhere: Elsewhere, +) -> anyhow::Result<()> { + let live = map.get(ifindex)?; + match (&elsewhere, &live) { + (Elsewhere::Skip, Some(live)) if live.vnet != rec.vnet => { + log::debug!("dhcp: ifindex {ifindex} sits elsewhere than the configs say, leaving it"); + return Ok(()); + } + (Elsewhere::Override, None) => return map.insert(ifindex, *rec), + (Elsewhere::Override, Some(live)) if live.vnet != rec.vnet => { + return map.insert(ifindex, *rec); + } + _ => {} + } + if applied.is_some_and(|applied| applied >= rec.generation) + || live.is_some_and(|live| live.generation >= rec.generation) + { + log::debug!( + "dhcp: change of generation {} for ifindex {ifindex} is older than what is in place, \ + dropping it", + rec.generation + ); + return Ok(()); + } + map.insert(ifindex, *rec) +} + +// entries of live links the pass does not know stay, their plug wrote them. The other unknown +// ones go with their interface. Returns the interfaces whose entry names another vnet than the +// configs, and how many writes failed +fn sync_records( + map: &mut impl RecordStore, + present: &HashMap, + unknown_alive: &HashSet, + generation: u64, +) -> anyhow::Result<(HashSet, usize)> { + let live: HashMap = map.entries()?.into_iter().collect(); + let dead: Vec = live + .keys() + .filter(|ifindex| !present.contains_key(ifindex) && !unknown_alive.contains(ifindex)) + .copied() + .collect(); + let mut failed = 0usize; + for &ifindex in &dead { + if let Err(e) = map.remove(ifindex) { + log::warn!("dhcp: removing the record of ifindex {ifindex}: {e:#}"); + failed += 1; + } + } + let (mut written, mut newer) = (0usize, 0usize); + + let mut elsewhere = HashSet::new(); + + for (ifindex, iface) in present { + // bad input leaves its interface as it is and the pass goes on. The marker for an + // unserved interface is what a plug still reading its input loses against + let desired = match iface.entry(generation) { + Ok(desired) => desired, + Err(e) => { + log::warn!("dhcp: {} is left as it is: {e:#}", iface.name); + continue; + } + }; + match live.get(ifindex) { + Some(entry) if entry.vnet != desired.vnet => { + elsewhere.insert(*ifindex); + } + + Some(entry) if entry.generation > generation => newer += 1, + Some(entry) if entry.same_answer(&desired) => {} + _ => match map.insert(*ifindex, desired) { + Ok(()) => written += 1, + Err(e) => { + log::warn!("dhcp: writing the entry of {}: {e:#}", iface.name); + failed += 1; + } + }, + } + } + + if written + dead.len() + newer + elsewhere.len() > 0 { + log::info!( + "dhcp: {written} entries written, {} removed, {newer} left to newer changes, {} \ + left where their plug put them", + dead.len(), + elsewhere.len() + ); + } else { + log::debug!("dhcp: {} records, no changes", live.len()); + } + Ok((elsewhere, failed)) +} + +// the interface's index here, none when it does not exist. Any other failure says nothing about +// the interface, a step taking it for gone would detach what it could not resolve +fn ifindex_of(name: &str) -> anyhow::Result> { + match nix::net::if_::if_nametoindex(name) { + Ok(ifindex) => Ok(Some(ifindex)), + Err(nix::errno::Errno::ENODEV) => Ok(None), + Err(e) => Err(e).with_context(|| format!("resolve interface {name}")), + } +} + +// the named interfaces that exist here by index, the names come from configs cluster-wide and +// most belong to other nodes +fn present_ifindexes(ifaces: &[Iface]) -> anyhow::Result> { + let mut present = HashMap::new(); + for iface in ifaces { + if let Some(ifindex) = ifindex_of(iface.name.as_str())? { + present.insert(ifindex, iface); + } + } + Ok(present) +} + +// the link follows the configs where they agree with the entry. An entry naming another vnet +// keeps its link, and a served interface without a mapping gets one. So a record change filling +// a marker attaches nothing itself. A live link the configs do not know stays + +fn wanted_links( + present: &HashMap, + elsewhere: &HashSet, + unknown_alive: &HashSet, +) -> HashSet { + let mut wanted: HashSet = present + .iter() + .filter(|(ifindex, iface)| elsewhere.contains(ifindex) || iface.serve) + .map(|(ifindex, _)| *ifindex) + .collect(); + + wanted.extend(unknown_alive); + wanted +} + +// whether the link's interface still exists under the name and index its pin carries +fn pin_alive(pin: &LinkPin) -> anyhow::Result { + Ok(ifindex_of(pin.name.as_str())? == Some(pin.ifindex)) +} + +impl Iface { + // a marker where nothing is served, so an older change arriving late still loses + fn entry(&self, generation: u64) -> anyhow::Result { + let vnet = self.place()?; + match &self.record { + Some(record) if self.serve => record + .entry(vnet, generation) + .with_context(|| format!("record for {}", self.name)), + _ => Ok(DhcpRecord::removed(vnet, generation)), + } + } + + fn place(&self) -> anyhow::Result<[u8; 8]> { + place(self.vnet.as_deref()).with_context(|| format!("interface {}", self.name)) + } +} + +impl Record { + // refused here is what the reply cannot carry. Zero stands for absent in the entry, so an + // address or an MTU of zero cannot be told from none + fn entry(&self, vnet: [u8; 8], generation: u64) -> anyhow::Result { + if self.prefixlen == 0 || self.prefixlen > 32 { + bail!("prefixlen {} out of range", self.prefixlen); + } + if self.ip.is_unspecified() { + bail!("no address"); + } + if self.server_id.is_unspecified() { + bail!("no server address"); + } + if self.lease == 0 { + bail!("lease of zero seconds"); + } + if self.mtu.is_some_and(|mtu| mtu < 68) { + bail!("mtu below the IPv4 minimum of 68"); + } + if self.mac == [0; 6] { + bail!("no MAC"); + } + let netmask = u32::MAX << (32 - self.prefixlen); + Ok(DhcpRecord { + ip: u32::from(self.ip).to_be(), + netmask: netmask.to_be(), + router: u32::from(self.router.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(), + dns: u32::from(self.dns.unwrap_or(Ipv4Addr::UNSPECIFIED)).to_be(), + server_id: u32::from(self.server_id).to_be(), + lease: self.lease.to_be(), + mtu: self.mtu.unwrap_or(0).to_be(), + vnet, + mac: self.mac, + generation, + }) + } +} + +#[cfg(test)] +mod test { + use std::collections::BTreeMap; + + use super::*; + + fn record() -> Record { + Record { + ip: Ipv4Addr::new(10, 0, 0, 100), + prefixlen: 24, + server_id: Ipv4Addr::new(10, 0, 0, 1), + lease: 300, + router: Some(Ipv4Addr::new(10, 0, 0, 1)), + dns: Some(Ipv4Addr::new(1, 1, 1, 1)), + mtu: Some(1500), + mac: MAC, + } + } + + const MAC: [u8; 6] = [0xda, 0x65, 0x8f, 0x18, 0x9b, 0x6f]; + + const VNET: [u8; 8] = *b"vnet1\0\0\0"; + + #[test] + fn converts_full_record() { + let rec = record().entry(VNET, 7).unwrap(); + assert_eq!( + u32::from_be(rec.ip), + u32::from(Ipv4Addr::new(10, 0, 0, 100)) + ); + assert_eq!(u32::from_be(rec.netmask), 0xffffff00); + assert_eq!( + u32::from_be(rec.router), + u32::from(Ipv4Addr::new(10, 0, 0, 1)) + ); + assert_eq!(u32::from_be(rec.dns), u32::from(Ipv4Addr::new(1, 1, 1, 1))); + assert_eq!( + u32::from_be(rec.server_id), + u32::from(Ipv4Addr::new(10, 0, 0, 1)) + ); + assert_eq!(u32::from_be(rec.lease), 300); + assert_eq!(u16::from_be(rec.mtu), 1500); + assert_eq!(rec.vnet, VNET); + assert_eq!(rec.mac, MAC); + assert_eq!(rec.generation, 7); + + let rec = Record { + router: None, + dns: None, + mtu: None, + ..record() + } + .entry(VNET, 7) + .unwrap(); + assert_eq!(rec.router, 0); + assert_eq!(rec.dns, 0); + assert_eq!(rec.mtu, 0); + } + + #[test] + fn answers_compare_without_the_generation() { + let a = record().entry(VNET, 7).unwrap(); + let b = record().entry(VNET, 9).unwrap(); + assert!(a.same_answer(&b)); + assert!(!a.same_answer(&DhcpRecord::removed(VNET, 9))); + assert_eq!(DhcpRecord::removed(VNET, 9).generation, 9); + assert_eq!(DhcpRecord::removed(VNET, 9).vnet, VNET); + } + + #[test] + fn entries_name_the_vnet_of_the_interface() { + let iface = Iface { + name: "tap100i0".into(), + vnet: Some("testvnet".into()), + serve: true, + record: Some(record()), + }; + assert_eq!(&iface.entry(7).unwrap().vnet, b"testvnet"); + let iface = Iface { + name: "tap100i0".into(), + vnet: None, + serve: false, + record: None, + }; + assert_eq!(iface.entry(7).unwrap().vnet, NO_VNET); + let iface = Iface { + name: "tap100i0".into(), + vnet: Some("ninechars".into()), + serve: true, + record: Some(record()), + }; + assert!(iface.entry(7).is_err()); + let iface = Iface { + name: "tap100i0".into(), + vnet: Some(String::new()), + serve: true, + record: Some(record()), + }; + assert!(iface.entry(7).is_err()); + } + + #[test] + fn unserved_interfaces_get_a_marker() { + let iface = Iface { + name: "tap100i0".into(), + vnet: Some("vnet1".into()), + serve: false, + record: Some(record()), + }; + assert_eq!(iface.entry(7).unwrap().ip, 0); + assert_eq!(iface.entry(7).unwrap().vnet, VNET); + let iface = Iface { + name: "tap100i0".into(), + vnet: Some("vnet1".into()), + serve: true, + record: None, + }; + assert_eq!(iface.entry(7).unwrap().ip, 0); + let iface = Iface { + name: "tap100i0".into(), + vnet: Some("vnet1".into()), + serve: true, + record: Some(record()), + }; + assert_ne!(iface.entry(7).unwrap().ip, 0); + } + + #[test] + fn refuses_what_the_reply_cannot_carry() { + for rec in [ + Record { + prefixlen: 33, + ..record() + }, + Record { + prefixlen: 0, + ..record() + }, + Record { + server_id: Ipv4Addr::UNSPECIFIED, + ..record() + }, + Record { + lease: 0, + ..record() + }, + Record { + ip: Ipv4Addr::UNSPECIFIED, + ..record() + }, + Record { + mtu: Some(67), + ..record() + }, + Record { + mac: [0; 6], + ..record() + }, + ] { + assert!(rec.entry(VNET, 7).is_err()); + } + let rec = Record { + mtu: Some(68), + ..record() + }; + assert_eq!(u16::from_be(rec.entry(VNET, 7).unwrap().mtu), 68); + + let rec = Record { + prefixlen: 32, + ..record() + }; + assert_eq!(u32::from_be(rec.entry(VNET, 7).unwrap().netmask), u32::MAX); + let rec = Record { + prefixlen: 1, + ..rec + }; + assert_eq!(u32::from_be(rec.entry(VNET, 7).unwrap().netmask), 1 << 31); + } + + impl RecordStore for BTreeMap { + fn get(&self, ifindex: u32) -> anyhow::Result> { + Ok(BTreeMap::get(self, &ifindex).copied()) + } + + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> { + BTreeMap::insert(self, ifindex, rec); + Ok(()) + } + + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> { + BTreeMap::remove(self, &ifindex); + Ok(()) + } + + fn entries(&self) -> anyhow::Result> { + Ok(self.iter().map(|(k, v)| (*k, *v)).collect()) + } + } + + const OTHER: [u8; 8] = *b"vnet2\0\0\0"; + + fn entry(vnet: [u8; 8], generation: u64) -> DhcpRecord { + record().entry(vnet, generation).unwrap() + } + + fn other_answer(vnet: [u8; 8], generation: u64) -> DhcpRecord { + Record { + ip: Ipv4Addr::new(10, 0, 0, 200), + ..record() + } + .entry(vnet, generation) + .unwrap() + } + + #[test] + fn a_record_change_is_ordered_by_generation() { + let mut map = BTreeMap::new(); + write_entry(&mut map, None, 1, &entry(VNET, 5), Elsewhere::Skip).unwrap(); + assert_eq!(map[&1].generation, 5); + // older than the live entry + write_entry(&mut map, None, 1, &other_answer(VNET, 4), Elsewhere::Skip).unwrap(); + assert!(map[&1].same_answer(&entry(VNET, 0))); + // older than the last full apply + write_entry( + &mut map, + Some(6), + 1, + &other_answer(VNET, 6), + Elsewhere::Skip, + ) + .unwrap(); + assert!(map[&1].same_answer(&entry(VNET, 0))); + // newer than both + write_entry( + &mut map, + Some(6), + 1, + &other_answer(VNET, 7), + Elsewhere::Skip, + ) + .unwrap(); + assert!(map[&1].same_answer(&other_answer(VNET, 0))); + assert_eq!(map[&1].generation, 7); + } + + #[test] + fn a_record_change_leaves_an_entry_of_another_vnet() { + let mut map = BTreeMap::new(); + map.insert(1, entry(OTHER, 1)); + write_entry(&mut map, None, 1, &other_answer(VNET, 9), Elsewhere::Skip).unwrap(); + assert_eq!(map[&1].vnet, OTHER); + assert_eq!(map[&1].generation, 1); + } + + #[test] + fn a_plug_writes_where_nothing_is() { + let mut map = BTreeMap::new(); + // a full apply stamped since the plug drew, it never saw the interface + write_entry(&mut map, Some(51), 1, &entry(VNET, 50), Elsewhere::Override).unwrap(); + assert_eq!(map[&1].generation, 50); + // a record change finding nothing is still ordered against the stamp + let mut map = BTreeMap::new(); + write_entry(&mut map, Some(51), 1, &entry(VNET, 50), Elsewhere::Skip).unwrap(); + assert!(map.is_empty()); + } + + #[test] + fn a_plug_replaces_an_entry_of_another_vnet_and_compares_on_its_own() { + let mut map = BTreeMap::new(); + map.insert(1, entry(OTHER, 9)); + // whatever the generations say, the tap sits on the plug's vnet now + write_entry(&mut map, Some(9), 1, &entry(VNET, 2), Elsewhere::Override).unwrap(); + assert_eq!(map[&1].vnet, VNET); + assert_eq!(map[&1].generation, 2); + // on its own vnet an older plug loses like any other change + write_entry( + &mut map, + None, + 1, + &other_answer(VNET, 1), + Elsewhere::Override, + ) + .unwrap(); + assert!(map[&1].same_answer(&entry(VNET, 0))); + write_entry( + &mut map, + None, + 1, + &other_answer(VNET, 3), + Elsewhere::Override, + ) + .unwrap(); + assert!(map[&1].same_answer(&other_answer(VNET, 0))); + // and one older than the last full apply loses against an existing entry too + write_entry(&mut map, Some(9), 1, &entry(VNET, 5), Elsewhere::Override).unwrap(); + assert!(map[&1].same_answer(&other_answer(VNET, 0))); + assert_eq!(map[&1].generation, 3); + } + + #[test] + fn an_unplug_removes_or_marks_the_entries() { + let mut map = BTreeMap::new(); + map.insert(1, entry(VNET, 3)); + map.insert(2, entry(VNET, 3)); + map.insert(3, entry(VNET, 3)); + unplug_entries(&mut map, &[1, 2], None).unwrap(); + assert_eq!(map.keys().copied().collect::>(), [3]); + // a missing entry is removed already + unplug_entries(&mut map, &[1], None).unwrap(); + // a move marks the interface that moved, an older incarnation under its name is gone + map.insert(4, entry(VNET, 3)); + let marker = DhcpRecord::removed(NO_VNET, 0); + unplug_entries(&mut map, &[3, 4], Some((4, marker))).unwrap(); + assert_eq!(map.keys().copied().collect::>(), [4]); + assert_eq!(map[&4].ip, 0); + assert_eq!(map[&4].vnet, NO_VNET); + assert_eq!(map[&4].generation, 0); + // an interface without an entry gets no marker + unplug_entries(&mut map, &[5], Some((5, marker))).unwrap(); + assert!(!map.contains_key(&5)); + } + + fn iface(name: &str, vnet: Option<&str>, serve: bool, record: Option) -> Iface { + Iface { + name: name.into(), + vnet: vnet.map(String::from), + serve, + record, + } + } + + #[test] + fn the_pass_syncs_the_records_it_knows() { + let mut map = BTreeMap::new(); + map.insert(1, entry(VNET, 3)); // same answer as the config, older + map.insert(2, other_answer(VNET, 3)); // a changed answer + map.insert(3, other_answer(VNET, 12)); // a change newer than the pass + map.insert(4, entry(VNET, 3)); // not served anymore + map.insert(6, entry(VNET, 3)); // its interface is gone + map.insert(7, entry(VNET, 3)); // a live link the configs do not name + map.insert(8, entry(VNET, 3)); // its mapping is gone + let ifaces = [ + iface("a", Some("vnet1"), true, Some(record())), + iface("b", Some("vnet1"), true, Some(record())), + iface("c", Some("vnet1"), true, Some(record())), + iface("d", Some("vnet1"), false, None), + iface("e", Some("vnet1"), true, Some(record())), // no entry yet + iface("f", Some("vnet1"), true, None), // no entry, served, no mapping + iface("g", Some("vnet1"), true, None), + ]; + let present: HashMap = [1, 2, 3, 4, 5, 9, 8] + .into_iter() + .zip(ifaces.iter()) + .collect(); + let unknown_alive = HashSet::from([7]); + let (elsewhere, _) = sync_records(&mut map, &present, &unknown_alive, 10).unwrap(); + + assert!(elsewhere.is_empty()); + assert_eq!( + map[&1].generation, 3, + "an unchanged answer keeps its generation" + ); + assert!(map[&2].same_answer(&entry(VNET, 0))); + assert_eq!(map[&2].generation, 10); + assert!( + map[&3].same_answer(&other_answer(VNET, 0)), + "a newer change stays" + ); + assert_eq!( + map[&4].ip, 0, + "an interface not served anymore keeps a marker" + ); + assert_eq!(map[&4].generation, 10); + assert_eq!(map[&5].generation, 10); + + assert!(!map.contains_key(&6)); + assert_eq!( + map[&9].ip, 0, + "a served interface without a mapping gets a marker" + ); + assert_eq!(map[&9].generation, 10); + assert_eq!(map[&8].ip, 0, "a mapping that is gone leaves a marker"); + assert_eq!(map[&8].generation, 10); + // a plug that read the cache before the pass now loses to the marker + write_entry(&mut map, Some(10), 8, &entry(VNET, 5), Elsewhere::Override).unwrap(); + assert_eq!(map[&8].ip, 0); + assert!(map.contains_key(&7)); + } + + #[test] + fn the_links_follow_the_configs_or_the_entry() { + let ifaces = [ + iface("a", Some("vnet1"), true, Some(record())), // served, entry agrees + iface("b", Some("vnet1"), false, None), // not served + iface("c", Some("vnet1"), false, None), // config stale, entry has a record + iface("d", Some("vnet1"), true, Some(record())), // config stale, entry is a marker + ]; + let present: HashMap = (1..=4).zip(ifaces.iter()).collect(); + let elsewhere = HashSet::from([3, 4]); + let unknown_alive = HashSet::from([9]); + assert_eq!( + wanted_links(&present, &elsewhere, &unknown_alive), + HashSet::from([1, 3, 4, 9]) + ); + } + + // a store that refuses one interface's writes + struct Flaky(BTreeMap, u32); + + impl RecordStore for Flaky { + fn get(&self, ifindex: u32) -> anyhow::Result> { + RecordStore::get(&self.0, ifindex) + } + + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> { + if ifindex == self.1 { + bail!("map full"); + } + RecordStore::insert(&mut self.0, ifindex, rec) + } + + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> { + if ifindex == self.1 { + bail!("map busy"); + } + RecordStore::remove(&mut self.0, ifindex) + } + + fn entries(&self) -> anyhow::Result> { + RecordStore::entries(&self.0) + } + } + + #[test] + fn bad_input_leaves_its_interface_alone() { + let mut map = BTreeMap::new(); + map.insert(1, entry(VNET, 3)); + let bad = Record { + lease: 0, + ..record() + }; + let ifaces = [ + iface("a", Some("vnet1"), true, Some(bad)), + iface("b", Some("vnet1"), true, Some(record())), + ]; + let present: HashMap = (1..=2).zip(ifaces.iter()).collect(); + let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new(), 10).unwrap(); + assert!(elsewhere.is_empty()); + assert_eq!(failed, 0); + assert_eq!(map[&1].generation, 3, "the old entry stays"); + assert_eq!(map[&2].generation, 10, "the rest is written"); + } + + #[test] + fn a_failed_write_does_not_stop_the_pass() { + let mut map = Flaky(BTreeMap::new(), 2); + let ifaces = [ + iface("a", Some("vnet1"), true, Some(record())), + iface("b", Some("vnet1"), true, Some(record())), + iface("c", Some("vnet1"), true, Some(record())), + ]; + let present: HashMap = (1..=3).zip(ifaces.iter()).collect(); + let (elsewhere, failed) = sync_records(&mut map, &present, &HashSet::new(), 10).unwrap(); + assert!(elsewhere.is_empty()); + assert_eq!(failed, 1); + assert!(map.0.contains_key(&1)); + assert!(!map.0.contains_key(&2)); + assert!(map.0.contains_key(&3)); + } + + // a store whose listing fails + struct Unlistable(BTreeMap); + + impl RecordStore for Unlistable { + fn get(&self, ifindex: u32) -> anyhow::Result> { + RecordStore::get(&self.0, ifindex) + } + + fn insert(&mut self, ifindex: u32, rec: DhcpRecord) -> anyhow::Result<()> { + RecordStore::insert(&mut self.0, ifindex, rec) + } + + fn remove(&mut self, ifindex: u32) -> anyhow::Result<()> { + RecordStore::remove(&mut self.0, ifindex) + } + + fn entries(&self) -> anyhow::Result> { + bail!("map gone") + } + } + + #[test] + fn a_failed_listing_fails_the_pass_before_it_writes() { + let mut map = Unlistable(BTreeMap::new()); + map.0.insert(1, entry(OTHER, 1)); + let ifaces = [iface("a", Some("vnet1"), true, Some(record()))]; + let present: HashMap = [(1, &ifaces[0])].into_iter().collect(); + assert!(sync_records(&mut map, &present, &HashSet::new(), 10).is_err()); + assert_eq!(map.0[&1].vnet, OTHER); + } + + #[test] + fn a_failed_removal_counts_like_a_failed_write() { + let mut map = Flaky(BTreeMap::new(), 2); + map.0.insert(2, entry(VNET, 3)); + let ifaces = [iface("a", Some("vnet1"), true, Some(record()))]; + let present: HashMap = [(1, &ifaces[0])].into_iter().collect(); + let (_, failed) = sync_records(&mut map, &present, &HashSet::new(), 10).unwrap(); + assert_eq!(failed, 1); + assert!(map.0.contains_key(&1)); + assert!(map.0.contains_key(&2)); + } + + #[test] + fn the_pass_leaves_entries_of_another_vnet_to_their_link() { + let mut map = BTreeMap::new(); + map.insert(1, entry(OTHER, 1)); // a plug the configs trail, with a record + map.insert(2, DhcpRecord::removed(NO_VNET, 0)); // moved to a plain bridge + map.insert(3, DhcpRecord::removed(NO_VNET, 0)); // the configs caught up with that + let ifaces = [ + iface("a", Some("vnet1"), true, Some(record())), + iface("b", Some("vnet1"), true, Some(record())), + iface("c", None, false, None), + ]; + let present: HashMap = (1..=3).zip(ifaces.iter()).collect(); + let (elsewhere, _) = sync_records(&mut map, &present, &HashSet::new(), 10).unwrap(); + + assert_eq!(elsewhere, HashSet::from([1, 2])); + + assert_eq!(map[&1].vnet, OTHER); + assert_eq!(map[&2].vnet, NO_VNET); + assert_eq!( + map[&3].ip, 0, + "a marker the configs agree with stays as it is" + ); + assert_eq!(map[&3].generation, 0); + } +} diff --git a/src/dhcp/types.rs b/src/dhcp/types.rs new file mode 100644 index 0000000..e421f96 --- /dev/null +++ b/src/dhcp/types.rs @@ -0,0 +1,80 @@ +//! Keep in sync with 'bpf/types.h'. + +/// Addresses, lease and mtu are stored in network byte order, so a reply copies them into the +/// packet as-is. +#[repr(C)] +#[derive(Copy, Clone)] +pub struct DhcpRecord { + /// 0 = removed, the entry only keeps its generation + pub ip: u32, + pub netmask: u32, + /// 0 = not served + pub router: u32, + /// 0 = not served + pub dns: u32, + pub server_id: u32, + pub lease: u32, + /// 0 = not served + pub mtu: u16, + /// The vnet the writer saw the interface on, the id padded with zeros, all zero for none. + /// Never read by the program. + pub vnet: [u8; 8], + /// The NIC's MAC, a request carrying another is passed on. Zero in a marker. + pub mac: [u8; 6], + /// The generation of the change that wrote the entry, host order, never read by the program. + pub generation: u64, +} + +impl DhcpRecord { + /// Whether two entries answer alike. The generation and the vnet are not part of the answer. + pub fn same_answer(&self, other: &Self) -> bool { + self.ip == other.ip + && self.netmask == other.netmask + && self.router == other.router + && self.dns == other.dns + && self.server_id == other.server_id + && self.lease == other.lease + && self.mtu == other.mtu + && self.mac == other.mac + } + + /// The marker a removal leaves behind, on the vnet the writer saw the interface on. + pub fn removed(vnet: [u8; 8], generation: u64) -> Self { + Self { + ip: 0, + netmask: 0, + router: 0, + dns: 0, + server_id: 0, + lease: 0, + mtu: 0, + vnet, + mac: [0; 6], + generation, + } + } +} + +unsafe impl aya::Pod for DhcpRecord {} + +#[cfg(test)] +mod layout { + use core::mem::{offset_of, size_of}; + + use super::*; + + #[test] + fn matches_bpf_abi() { + assert_eq!(size_of::(), 48); + assert_eq!(offset_of!(DhcpRecord, ip), 0); + assert_eq!(offset_of!(DhcpRecord, netmask), 4); + assert_eq!(offset_of!(DhcpRecord, router), 8); + assert_eq!(offset_of!(DhcpRecord, dns), 12); + assert_eq!(offset_of!(DhcpRecord, server_id), 16); + assert_eq!(offset_of!(DhcpRecord, lease), 20); + assert_eq!(offset_of!(DhcpRecord, mtu), 24); + assert_eq!(offset_of!(DhcpRecord, vnet), 26); + assert_eq!(offset_of!(DhcpRecord, mac), 34); + assert_eq!(offset_of!(DhcpRecord, generation), 40); + } +} diff --git a/src/lib.rs b/src/lib.rs index 572d861..a8c94ba 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -1,5 +1,8 @@ //! eBPF subsystems, one per concern, each behind a cargo feature of its name. Nothing here runs //! on its own, consumers pull in only the subsystem they drive. +#[cfg(feature = "dhcp")] +pub mod dhcp; + pub mod subsystem; pub mod tc; -- 2.47.3