From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 97EF77365B; Fri, 18 Jun 2021 10:15:04 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8249D249A4; Fri, 18 Jun 2021 10:15:04 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 0BAA824997; Fri, 18 Jun 2021 10:15:01 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id CBDD043E6B; Fri, 18 Jun 2021 10:15:00 +0200 (CEST) Date: Fri, 18 Jun 2021 10:14:46 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: pbs-devel@lists.proxmox.com, Proxmox VE development discussion References: <20210611114418.28772-1-f.ebner@proxmox.com> <20210611114418.28772-2-f.ebner@proxmox.com> In-Reply-To: <20210611114418.28772-2-f.ebner@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid) Message-Id: <1624003551.byacxlscjd.astroid@nora.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.777 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [repositories.rs, lib.rs, falcot.com, file.rs, types.rs, check.rs, writer.rs, mod.rs, strutl.cc, proxmox.com] Subject: Re: [pve-devel] [PATCH v6 proxmox-apt 01/11] initial commit X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jun 2021 08:15:04 -0000 On June 11, 2021 1:43 pm, Fabian Ebner wrote: > Signed-off-by: Fabian Ebner > --- >=20 > Changes from v5: > * tests: add a URI with username and port >=20 > .cargo/config | 5 + > .gitignore | 4 + > Cargo.toml | 23 ++ > rustfmt.toml | 1 + > src/lib.rs | 3 + > src/repositories/check.rs | 47 ++++ > src/repositories/file.rs | 96 +++++++ > src/repositories/list_parser.rs | 171 ++++++++++++ > src/repositories/mod.rs | 224 ++++++++++++++++ > src/repositories/sources_parser.rs | 204 +++++++++++++++ > src/repositories/writer.rs | 92 +++++++ > src/types.rs | 246 ++++++++++++++++++ > tests/repositories.rs | 129 +++++++++ > .../absolute_suite.list | 5 + > .../absolute_suite.sources | 5 + > tests/sources.list.d.expected/case.sources | 16 ++ > .../sources.list.d.expected/multiline.sources | 10 + > .../options_comment.list | 6 + > .../pbs-enterprise.list | 2 + > tests/sources.list.d.expected/pve.list | 13 + > tests/sources.list.d.expected/standard.list | 7 + > .../sources.list.d.expected/standard.sources | 11 + > tests/sources.list.d/absolute_suite.list | 4 + > tests/sources.list.d/absolute_suite.sources | 5 + > tests/sources.list.d/case.sources | 17 ++ > tests/sources.list.d/multiline.sources | 11 + > tests/sources.list.d/options_comment.list | 3 + > tests/sources.list.d/pbs-enterprise.list | 1 + > tests/sources.list.d/pve.list | 10 + > tests/sources.list.d/standard.list | 6 + > tests/sources.list.d/standard.sources | 10 + > 31 files changed, 1387 insertions(+) > create mode 100644 .cargo/config > create mode 100644 .gitignore > create mode 100644 Cargo.toml > create mode 100644 rustfmt.toml > create mode 100644 src/lib.rs > create mode 100644 src/repositories/check.rs > create mode 100644 src/repositories/file.rs > create mode 100644 src/repositories/list_parser.rs > create mode 100644 src/repositories/mod.rs > create mode 100644 src/repositories/sources_parser.rs > create mode 100644 src/repositories/writer.rs > create mode 100644 src/types.rs > create mode 100644 tests/repositories.rs > create mode 100644 tests/sources.list.d.expected/absolute_suite.list > create mode 100644 tests/sources.list.d.expected/absolute_suite.sources > create mode 100644 tests/sources.list.d.expected/case.sources > create mode 100644 tests/sources.list.d.expected/multiline.sources > create mode 100644 tests/sources.list.d.expected/options_comment.list > create mode 100644 tests/sources.list.d.expected/pbs-enterprise.list > create mode 100644 tests/sources.list.d.expected/pve.list > create mode 100644 tests/sources.list.d.expected/standard.list > create mode 100644 tests/sources.list.d.expected/standard.sources > create mode 100644 tests/sources.list.d/absolute_suite.list > create mode 100644 tests/sources.list.d/absolute_suite.sources > create mode 100644 tests/sources.list.d/case.sources > create mode 100644 tests/sources.list.d/multiline.sources > create mode 100644 tests/sources.list.d/options_comment.list > create mode 100644 tests/sources.list.d/pbs-enterprise.list > create mode 100644 tests/sources.list.d/pve.list > create mode 100644 tests/sources.list.d/standard.list > create mode 100644 tests/sources.list.d/standard.sources >=20 > diff --git a/.cargo/config b/.cargo/config > new file mode 100644 > index 0000000..3b5b6e4 > --- /dev/null > +++ b/.cargo/config > @@ -0,0 +1,5 @@ > +[source] > +[source.debian-packages] > +directory =3D "/usr/share/cargo/registry" > +[source.crates-io] > +replace-with =3D "debian-packages" > diff --git a/.gitignore b/.gitignore > new file mode 100644 > index 0000000..24917d4 > --- /dev/null > +++ b/.gitignore > @@ -0,0 +1,4 @@ > +Cargo.lock > +target/ > +tests/sources.list.d.actual > +tests/sources.list.d.digest > diff --git a/Cargo.toml b/Cargo.toml > new file mode 100644 > index 0000000..24f734b > --- /dev/null > +++ b/Cargo.toml > @@ -0,0 +1,23 @@ > +[package] > +name =3D "proxmox-apt" > +version =3D "0.1.0" > +authors =3D [ > + "Fabian Ebner ", > + "Proxmox Support Team ", > +] > +edition =3D "2018" > +license =3D "AGPL-3" > +description =3D "Proxmox library for APT" > +homepage =3D "https://www.proxmox.com" > + > +exclude =3D [ "debian" ] > + > +[lib] > +name =3D "proxmox_apt" > +path =3D "src/lib.rs" > + > +[dependencies] > +anyhow =3D "1.0" > +openssl =3D "0.10" > +proxmox =3D { version =3D "0.11.5", features =3D [ "api-macro" ] } > +serde =3D { version =3D "1.0", features =3D ["derive"] } > diff --git a/rustfmt.toml b/rustfmt.toml > new file mode 100644 > index 0000000..32a9786 > --- /dev/null > +++ b/rustfmt.toml > @@ -0,0 +1 @@ > +edition =3D "2018" > diff --git a/src/lib.rs b/src/lib.rs > new file mode 100644 > index 0000000..b065c0f > --- /dev/null > +++ b/src/lib.rs > @@ -0,0 +1,3 @@ > +pub mod types; > + > +pub mod repositories; > diff --git a/src/repositories/check.rs b/src/repositories/check.rs > new file mode 100644 > index 0000000..87fbbac > --- /dev/null > +++ b/src/repositories/check.rs > @@ -0,0 +1,47 @@ > +use anyhow::{bail, Error}; > + > +use crate::types::{APTRepository, APTRepositoryFileType}; > + > +impl APTRepository { > + /// Makes sure that all basic properties of a repository are present= and > + /// not obviously invalid. > + pub fn basic_check(&self) -> Result<(), Error> { > + if self.types.is_empty() { > + bail!("missing package type(s)"); > + } > + if self.uris.is_empty() { > + bail!("missing URI(s)"); > + } > + if self.suites.is_empty() { > + bail!("missing suite(s)"); > + } > + > + for uri in self.uris.iter() { > + if !uri.contains(':') || uri.len() < 3 { > + bail!("invalid URI: '{}'", uri); > + } > + } > + > + for suite in self.suites.iter() { > + if !suite.ends_with('/') && self.components.is_empty() { > + bail!("missing component(s)"); > + } else if suite.ends_with('/') && !self.components.is_empty(= ) { > + bail!("absolute suite '{}' does not allow component(s)",= suite); > + } > + } > + > + if self.file_type =3D=3D APTRepositoryFileType::List { > + if self.types.len() > 1 { > + bail!("more than one package type"); > + } > + if self.uris.len() > 1 { > + bail!("more than one URI"); > + } > + if self.suites.len() > 1 { > + bail!("more than one suite"); > + } > + } > + > + Ok(()) > + } > +} > diff --git a/src/repositories/file.rs b/src/repositories/file.rs > new file mode 100644 > index 0000000..e264ec6 > --- /dev/null > +++ b/src/repositories/file.rs > @@ -0,0 +1,96 @@ > +use std::convert::TryFrom; > +use std::path::{Path, PathBuf}; > + > +use anyhow::{format_err, Error}; > + > +use crate::types::{APTRepositoryFile, APTRepositoryFileError, APTReposit= oryFileType}; > + > +impl APTRepositoryFile { > + /// Creates a new `APTRepositoryFile` without parsing. > + /// > + /// If the file is hidden or the path points to a directory, `Ok(Non= e)` is > + /// returned, while invalid file names yield an error. > + pub fn new>(path: P) -> Result, APTRepos= itoryFileError> { > + let path: PathBuf =3D path.as_ref().to_path_buf(); > + > + let new_err =3D |path_string: String, err: &str| APTRepositoryFi= leError { > + path: path_string, > + error: err.to_string(), > + }; > + > + let path_string =3D path > + .clone() > + .into_os_string() > + .into_string() > + .map_err(|os_string| { > + new_err( > + os_string.to_string_lossy().to_string(), > + "path is not valid unicode", > + ) > + })?; > + > + let new_err =3D |err| new_err(path_string.clone(), err); > + > + if path.is_dir() { > + return Ok(None); > + } > + > + let file_name =3D match path.file_name() { > + Some(file_name) =3D> file_name > + .to_os_string() > + .into_string() > + .map_err(|_| new_err("invalid path"))?, > + None =3D> return Err(new_err("invalid path")), > + }; > + > + if file_name.starts_with('.') { > + return Ok(None); > + } > + > + let extension =3D match path.extension() { > + Some(extension) =3D> extension > + .to_os_string() > + .into_string() > + .map_err(|_| new_err("invalid path"))?, > + None =3D> return Err(new_err("invalid extension")), > + }; > + > + let file_type =3D APTRepositoryFileType::try_from(&extension[..]= ) > + .map_err(|_| new_err("invalid extension"))?; > + > + if !file_name > + .chars() > + .all(|x| x.is_ascii_alphanumeric() || x =3D=3D '_' || x =3D= =3D '-' || x =3D=3D '.') > + { > + return Err(new_err("invalid characters in file name")); > + } > + > + Ok(Some(Self { > + path: path_string, > + file_type, > + repositories: vec![], > + digest: None, > + })) > + } > + > + /// Check if the file exists. > + pub fn exists(&self) -> bool { > + PathBuf::from(&self.path).exists() > + } > + > + pub fn read_with_digest(&self) -> Result<(Vec, [u8; 32]), APTRep= ositoryFileError> { > + let content =3D std::fs::read(&self.path).map_err(|err| self.err= (format_err!("{}", err)))?; > + > + let digest =3D openssl::sha::sha256(&content); > + > + Ok((content, digest)) > + } > + > + /// Create an `APTRepositoryFileError`. > + pub fn err(&self, error: Error) -> APTRepositoryFileError { > + APTRepositoryFileError { > + path: self.path.clone(), > + error: error.to_string(), > + } > + } > +} > diff --git a/src/repositories/list_parser.rs b/src/repositories/list_pars= er.rs > new file mode 100644 > index 0000000..6c9f898 > --- /dev/null > +++ b/src/repositories/list_parser.rs > @@ -0,0 +1,171 @@ > +use std::convert::TryInto; > +use std::io::BufRead; > +use std::iter::{Iterator, Peekable}; > +use std::str::SplitAsciiWhitespace; > + > +use anyhow::{bail, format_err, Error}; > + > +use super::APTRepositoryParser; > +use crate::types::{APTRepository, APTRepositoryFileType, APTRepositoryOp= tion}; > + > +pub struct APTListFileParser { > + input: R, > + line_nr: usize, > + comment: String, > +} > + > +impl APTListFileParser { > + pub fn new(reader: R) -> Self { > + Self { > + input: reader, > + line_nr: 0, > + comment: String::new(), > + } > + } > + > + /// Helper to parse options from the existing token stream. > + /// > + /// Also returns `Ok(())` if there are no options. > + /// > + /// Errors when options are invalid or not closed by `']'`. > + fn parse_options( > + options: &mut Vec, > + tokens: &mut Peekable, > + ) -> Result<(), Error> { > + let mut option =3D match tokens.peek() { > + Some(token) =3D> { > + match token.strip_prefix('[') { > + Some(option) =3D> option, > + None =3D> return Ok(()), // doesn't look like option= s > + } > + } > + None =3D> return Ok(()), > + }; > + > + tokens.next(); // avoid reading the beginning twice > + > + let mut finished =3D false; > + loop { > + if let Some(stripped) =3D option.strip_suffix(']') { > + option =3D stripped; > + if option.is_empty() { > + break; > + } > + finished =3D true; // but still need to handle the last = one > + }; > + > + if let Some(mid) =3D option.find('=3D') { > + let (key, mut value_str) =3D option.split_at(mid); > + value_str =3D &value_str[1..]; > + > + if key.is_empty() { > + bail!("option has no key: '{}'", option); > + } > + > + if value_str.is_empty() { > + bail!("option has no value: '{}'", option); > + } > + > + let values: Vec =3D value_str > + .split(',') > + .map(|value| value.to_string()) > + .collect(); > + > + options.push(APTRepositoryOption { > + key: key.to_string(), > + values, > + }); > + } else if !option.is_empty() { > + bail!("got invalid option - '{}'", option); > + } > + > + if finished { > + break; > + } > + > + option =3D match tokens.next() { > + Some(option) =3D> option, > + None =3D> bail!("options not closed by ']'"), > + } > + } > + > + Ok(()) > + } > + > + /// Parse a repository or comment in one-line format. > + /// > + /// Commented out repositories are also detected and returned with t= he > + /// `enabled` property set to `false`. > + /// > + /// If the line contains a repository, `self.comment` is added to th= e > + /// `comment` property. > + /// > + /// If the line contains a comment, it is added to `self.comment`. > + fn parse_one_line(&mut self, mut line: &str) -> Result, Error> { > + line =3D line.trim_matches(|c| char::is_ascii_whitespace(&c)); > + > + // check for commented out repository first > + if let Some(commented_out) =3D line.strip_prefix('#') { > + if let Ok(Some(mut repo)) =3D self.parse_one_line(commented_= out) { > + repo.set_enabled(false); > + return Ok(Some(repo)); > + } > + } > + > + let mut repo =3D APTRepository::new(APTRepositoryFileType::List)= ; > + > + // now handle "real" comment > + if let Some(comment_start) =3D line.find('#') { > + let (line_start, comment) =3D line.split_at(comment_start); > + self.comment =3D format!("{}{}\n", self.comment, &comment[1.= .]); > + line =3D line_start; > + } > + > + let mut tokens =3D line.split_ascii_whitespace().peekable(); > + > + match tokens.next() { > + Some(package_type) =3D> { > + repo.types.push(package_type.try_into()?); > + } > + None =3D> return Ok(None), // empty line > + } > + > + Self::parse_options(&mut repo.options, &mut tokens)?; > + > + // the rest of the line is just ' [...]= ' > + let mut tokens =3D tokens.map(str::to_string); > + repo.uris > + .push(tokens.next().ok_or_else(|| format_err!("missing URI")= )?); > + repo.suites > + .push(tokens.next().ok_or_else(|| format_err!("missing suite= "))?); > + repo.components.extend(tokens); > + > + repo.comment =3D std::mem::take(&mut self.comment); > + > + Ok(Some(repo)) > + } > +} > + > +impl APTRepositoryParser for APTListFileParser { > + fn parse_repositories(&mut self) -> Result, Error= > { > + let mut repos =3D vec![]; > + let mut line =3D String::new(); > + > + loop { > + self.line_nr +=3D 1; > + line.clear(); > + > + match self.input.read_line(&mut line) { > + Err(err) =3D> bail!("input error - {}", err), > + Ok(0) =3D> break, > + Ok(_) =3D> match self.parse_one_line(&line) { > + Ok(Some(repo)) =3D> repos.push(repo), > + Ok(None) =3D> continue, > + Err(err) =3D> bail!("malformed entry on line {} - {}= ", self.line_nr, err), > + }, > + } > + } > + > + Ok(repos) > + } > +} > diff --git a/src/repositories/mod.rs b/src/repositories/mod.rs > new file mode 100644 > index 0000000..187ead3 > --- /dev/null > +++ b/src/repositories/mod.rs > @@ -0,0 +1,224 @@ > +use std::path::PathBuf; > + > +use anyhow::{bail, format_err, Error}; > + > +use crate::types::{ > + APTRepository, APTRepositoryFile, APTRepositoryFileError, APTReposit= oryFileType, > + APTRepositoryOption, > +}; > + > +mod list_parser; > +use list_parser::APTListFileParser; > + > +mod sources_parser; > +use sources_parser::APTSourcesFileParser; > + > +mod check; > +mod file; > +mod writer; > + > +const APT_SOURCES_LIST_FILENAME: &str =3D "/etc/apt/sources.list"; > +const APT_SOURCES_LIST_DIRECTORY: &str =3D "/etc/apt/sources.list.d/"; > + > +impl APTRepository { > + /// Crates an empty repository. > + fn new(file_type: APTRepositoryFileType) -> Self { > + Self { > + types: vec![], > + uris: vec![], > + suites: vec![], > + components: vec![], > + options: vec![], > + comment: String::new(), > + file_type, > + enabled: true, > + } > + } > + > + /// Changes the `enabled` flag and makes sure the `Enabled` option f= or > + /// `APTRepositoryPackageType::Sources` repositories is updated too. > + fn set_enabled(&mut self, enabled: bool) { > + self.enabled =3D enabled; > + > + if self.file_type =3D=3D APTRepositoryFileType::Sources { > + let enabled_string =3D match enabled { > + true =3D> "true".to_string(), > + false =3D> "false".to_string(), > + }; > + for option in self.options.iter_mut() { > + if option.key =3D=3D "Enabled" { > + option.values =3D vec![enabled_string]; > + return; > + } > + } > + self.options.push(APTRepositoryOption { > + key: "Enabled".to_string(), > + values: vec![enabled_string], > + }); > + } > + } > +} > + > +trait APTRepositoryParser { > + /// Parse all repositories including the disabled ones and push them= onto > + /// the provided vector. > + fn parse_repositories(&mut self) -> Result, Error= >; > +} > + > +impl APTRepositoryFile { > + /// Parses the APT repositories configured in the file on disk, incl= uding > + /// disabled ones. > + /// > + /// Resets the current repositories and digest, even on failure. > + pub fn parse(&mut self) -> Result<(), APTRepositoryFileError> { > + self.repositories.clear(); > + self.digest =3D None; > + > + let (content, digest) =3D self.read_with_digest()?; > + > + let mut parser: Box =3D match self.file= _type { > + APTRepositoryFileType::List =3D> Box::new(APTListFileParser:= :new(&content[..])), > + APTRepositoryFileType::Sources =3D> Box::new(APTSourcesFileP= arser::new(&content[..])), > + }; > + > + let repos =3D parser.parse_repositories().map_err(|err| self.err= (err))?; > + > + for (n, repo) in repos.iter().enumerate() { > + repo.basic_check() > + .map_err(|err| self.err(format_err!("check for repositor= y {} - {}", n + 1, err)))?; > + } > + > + self.repositories =3D repos; > + self.digest =3D Some(digest); > + > + Ok(()) > + } > + > + /// Writes the repositories to the file on disk. > + /// > + /// If a digest is provided, checks that the current content of the = file still > + /// produces the same one. > + pub fn write(&self) -> Result<(), APTRepositoryFileError> { > + if let Some(digest) =3D self.digest { > + if !self.exists() { > + return Err(self.err(format_err!("digest specified, but f= ile does not exist"))); > + } > + > + let (_, current_digest) =3D self.read_with_digest()?; > + if digest !=3D current_digest { > + return Err(self.err(format_err!("digest mismatch"))); > + } > + } > + > + let mut content =3D vec![]; > + > + for (n, repo) in self.repositories.iter().enumerate() { > + repo.basic_check() > + .map_err(|err| self.err(format_err!("check for repositor= y {} - {}", n + 1, err)))?; > + > + repo.write(&mut content) > + .map_err(|err| self.err(format_err!("writing repository = {} - {}", n + 1, err)))?; > + } > + > + let path =3D PathBuf::from(&self.path); > + let dir =3D match path.parent() { > + Some(dir) =3D> dir, > + None =3D> return Err(self.err(format_err!("invalid path"))), > + }; > + > + std::fs::create_dir_all(dir) > + .map_err(|err| self.err(format_err!("unable to create parent= dir - {}", err)))?; > + > + let pid =3D std::process::id(); > + let mut tmp_path =3D path.clone(); > + tmp_path.set_extension("tmp"); > + tmp_path.set_extension(format!("{}", pid)); > + > + if let Err(err) =3D std::fs::write(&tmp_path, content) { > + let _ =3D std::fs::remove_file(&tmp_path); > + return Err(self.err(format_err!("writing {:?} failed - {}", = path, err))); > + } > + > + if let Err(err) =3D std::fs::rename(&tmp_path, &path) { > + let _ =3D std::fs::remove_file(&tmp_path); > + return Err(self.err(format_err!("rename failed for {:?} - {}= ", path, err))); > + } > + > + Ok(()) > + } > +} > + > +/// Returns all APT repositories configured in `/etc/apt/sources.list` a= nd > +/// in `/etc/apt/sources.list.d` including disabled repositories. > +/// > +/// Returns the parsable files with their repositories and a list of err= ors for > +/// files that could not be read or parsed. > +/// > +/// The digest is guaranteed to be set for each successfully parsed file= . since all(?) the callers for this then calculate the common digest (at=20 least optionally), it might make sense to just return it here and not=20 have the whole common digest thing as separate, public interface? also possible making this an APTRepositoryFOOBAR struct with repos,=20 errors, digest as fields looks like a nicer interface to me (although=20 tuples with 3 values are kind of the grey area between okay-as-tuple and=20 definitely-too-big-should-be-a-struct ;)) > +pub fn repositories() -> Result<(Vec, Vec), Error> { > + let mut files =3D vec![]; > + let mut errors =3D vec![]; > + > + let sources_list_path =3D PathBuf::from(APT_SOURCES_LIST_FILENAME); > + > + let sources_list_d_path =3D PathBuf::from(APT_SOURCES_LIST_DIRECTORY= ); > + > + match APTRepositoryFile::new(sources_list_path) { > + Ok(Some(mut file)) =3D> match file.parse() { > + Ok(()) =3D> files.push(file), > + Err(err) =3D> errors.push(err), > + }, > + _ =3D> bail!("internal error with '{}'", APT_SOURCES_LIST_FILENA= ME), > + } > + > + if !sources_list_d_path.exists() { > + return Ok((files, errors)); > + } > + > + if !sources_list_d_path.is_dir() { > + errors.push(APTRepositoryFileError { > + path: APT_SOURCES_LIST_DIRECTORY.to_string(), > + error: "not a directory!".to_string(), > + }); > + return Ok((files, errors)); > + } > + > + for entry in std::fs::read_dir(sources_list_d_path)? { > + let path =3D entry?.path(); > + > + match APTRepositoryFile::new(path) { > + Ok(Some(mut file)) =3D> match file.parse() { > + Ok(()) =3D> { > + if file.digest.is_none() { > + bail!("internal error - digest not set"); > + } > + files.push(file); > + } > + Err(err) =3D> errors.push(err), > + }, > + Ok(None) =3D> (), > + Err(err) =3D> errors.push(err), > + } > + } > + > + Ok((files, errors)) > +} > + > +/// Write the repositories for each file. > +/// > +/// Returns an error for each file that could not be written successfull= y. > +pub fn write_repositories(files: &[APTRepositoryFile]) -> Result<(), Vec= > { > + let mut errors =3D vec![]; > + > + for file in files { > + if let Err(err) =3D file.write() { > + errors.push(err); > + } > + } > + > + if !errors.is_empty() { > + return Err(errors); > + } > + > + Ok(()) > +} > diff --git a/src/repositories/sources_parser.rs b/src/repositories/source= s_parser.rs > new file mode 100644 > index 0000000..a056b8f > --- /dev/null > +++ b/src/repositories/sources_parser.rs > @@ -0,0 +1,204 @@ > +use std::convert::TryInto; > +use std::io::BufRead; > +use std::iter::Iterator; > + > +use anyhow::{bail, Error}; > + > +use crate::types::{ > + APTRepository, APTRepositoryFileType, APTRepositoryOption, APTReposi= toryPackageType, > +}; > + > +use super::APTRepositoryParser; > + > +pub struct APTSourcesFileParser { > + input: R, > + stanza_nr: usize, > + comment: String, > +} > + > +/// See `man sources.list` and `man deb822` for the format specification= . > +impl APTSourcesFileParser { > + pub fn new(reader: R) -> Self { > + Self { > + input: reader, > + stanza_nr: 1, > + comment: String::new(), > + } > + } > + > + /// Based on APT's `StringToBool` in `strutl.cc` > + fn string_to_bool(string: &str, default: bool) -> bool { > + let string =3D string.trim_matches(|c| char::is_ascii_whitespace= (&c)); > + let string =3D string.to_lowercase(); > + > + match &string[..] { > + "1" | "yes" | "true" | "with" | "on" | "enable" =3D> true, > + "0" | "no" | "false" | "without" | "off" | "disable" =3D> fa= lse, > + _ =3D> default, > + } > + } > + > + /// Checks if `key` is valid according to deb822 > + fn valid_key(key: &str) -> bool { > + if key.starts_with('-') { > + return false; > + }; > + return key.chars().all(|c| matches!(c, '!'..=3D'9' | ';'..=3D'~'= )); > + } > + > + /// Try parsing a repository in stanza format from `lines`. > + /// > + /// Returns `Ok(None)` when no stanza can be found. > + /// > + /// Comments are added to `self.comments`. If a stanza can be found, > + /// `self.comment` is added to the repository's `comment` property. > + /// > + /// Fully commented out stanzas are treated as comments. > + fn parse_stanza(&mut self, lines: &str) -> Result, Error> { > + let mut repo =3D APTRepository::new(APTRepositoryFileType::Sourc= es); > + > + // Values may be folded into multiple lines. > + // Those lines have to start with a space or a tab. > + let lines =3D lines.replace("\n ", " "); > + let lines =3D lines.replace("\n\t", " "); > + > + let mut got_something =3D false; > + > + for line in lines.lines() { > + let line =3D line.trim_matches(|c| char::is_ascii_whitespace= (&c)); > + if line.is_empty() { > + continue; > + } > + > + if let Some(commented_out) =3D line.strip_prefix('#') { > + self.comment =3D format!("{}{}\n", self.comment, comment= ed_out); > + continue; > + } > + > + if let Some(mid) =3D line.find(':') { > + let (key, value_str) =3D line.split_at(mid); > + let value_str =3D &value_str[1..]; > + let key =3D key.trim_matches(|c| char::is_ascii_whitespa= ce(&c)); > + > + if key.is_empty() { > + bail!("option has no key: '{}'", line); > + } > + > + if value_str.is_empty() { > + // ignored by APT > + eprintln!("option has no value: '{}'", line); > + continue; > + } > + > + if !Self::valid_key(key) { > + // ignored by APT > + eprintln!("option with invalid key '{}'", key); > + continue; > + } > + > + let values: Vec =3D value_str > + .split_ascii_whitespace() > + .map(|value| value.to_string()) > + .collect(); > + > + match &key.to_lowercase()[..] { > + "types" =3D> { > + if !repo.types.is_empty() { > + eprintln!("key 'Types' was defined twice"); > + } > + let mut types =3D Vec::::new(); > + for package_type in values { > + types.push((&package_type[..]).try_into()?); > + } > + repo.types =3D types; > + } > + "uris" =3D> { > + if !repo.uris.is_empty() { > + eprintln!("key 'URIs' was defined twice"); > + } > + repo.uris =3D values; > + } > + "suites" =3D> { > + if !repo.suites.is_empty() { > + eprintln!("key 'Suites' was defined twice"); > + } > + repo.suites =3D values; > + } > + "components" =3D> { > + if !repo.components.is_empty() { > + eprintln!("key 'Components' was defined twic= e"); > + } > + repo.components =3D values; > + } > + "enabled" =3D> { > + repo.set_enabled(Self::string_to_bool(value_str,= true)); > + } > + _ =3D> repo.options.push(APTRepositoryOption { > + key: key.to_string(), > + values, > + }), > + } > + } else { > + bail!("got invalid line - '{:?}'", line); > + } > + > + got_something =3D true; > + } > + > + if !got_something { > + return Ok(None); > + } > + > + repo.comment =3D std::mem::take(&mut self.comment); > + > + Ok(Some(repo)) > + } > + > + /// Helper function for `parse_repositories`. > + fn try_parse_stanza( > + &mut self, > + lines: &str, > + repos: &mut Vec, > + ) -> Result<(), Error> { > + match self.parse_stanza(lines) { > + Ok(Some(repo)) =3D> { > + repos.push(repo); > + self.stanza_nr +=3D 1; > + } > + Ok(None) =3D> (), > + Err(err) =3D> bail!("malformed entry in stanza {} - {}", sel= f.stanza_nr, err), > + } > + > + Ok(()) > + } > +} > + > +impl APTRepositoryParser for APTSourcesFileParser { > + fn parse_repositories(&mut self) -> Result, Error= > { > + let mut repos =3D vec![]; > + let mut lines =3D String::new(); > + > + loop { > + let old_length =3D lines.len(); > + match self.input.read_line(&mut lines) { > + Err(err) =3D> bail!("input error - {}", err), > + Ok(0) =3D> { > + self.try_parse_stanza(&lines[..], &mut repos)?; > + break; > + } > + Ok(_) =3D> { > + if (&lines[old_length..]) > + .trim_matches(|c| char::is_ascii_whitespace(&c)) > + .is_empty() > + { > + // detected end of stanza > + self.try_parse_stanza(&lines[..], &mut repos)?; > + lines.clear(); > + } > + } > + } > + } > + > + Ok(repos) > + } > +} > diff --git a/src/repositories/writer.rs b/src/repositories/writer.rs > new file mode 100644 > index 0000000..d9e937c > --- /dev/null > +++ b/src/repositories/writer.rs > @@ -0,0 +1,92 @@ > +use std::io::Write; > + > +use anyhow::{bail, Error}; > + > +use crate::types::{APTRepository, APTRepositoryFileType}; > + > +impl APTRepository { > + /// Writes a repository in the corresponding format followed by a bl= ank. > + /// > + /// Expects that `basic_check()` for the repository was successful. > + pub fn write(&self, w: &mut dyn Write) -> Result<(), Error> { > + match self.file_type { > + APTRepositoryFileType::List =3D> write_one_line(self, w), > + APTRepositoryFileType::Sources =3D> write_stanza(self, w), > + } > + } > +} > + > +/// Writes a repository in one-line format followed by a blank line. > +/// > +/// Expects that `repo.file_type =3D=3D APTRepositoryFileType::List`. > +/// > +/// Expects that `basic_check()` for the repository was successful. > +fn write_one_line(repo: &APTRepository, w: &mut dyn Write) -> Result<(),= Error> { > + if repo.file_type !=3D APTRepositoryFileType::List { > + bail!("not a .list repository"); > + } > + > + if !repo.comment.is_empty() { > + for line in repo.comment.lines() { > + writeln!(w, "#{}", line)?; > + } > + } > + > + if !repo.enabled { > + write!(w, "# ")?; > + } > + > + write!(w, "{} ", repo.types[0])?; > + > + if !repo.options.is_empty() { > + write!(w, "[ ")?; > + repo.options > + .iter() > + .try_for_each(|option| write!(w, "{}=3D{} ", option.key, opt= ion.values.join(",")))?; > + write!(w, "] ")?; > + }; > + > + write!(w, "{} ", repo.uris[0])?; > + write!(w, "{} ", repo.suites[0])?; > + writeln!(w, "{}", repo.components.join(" "))?; > + > + writeln!(w)?; > + > + Ok(()) > +} > + > +/// Writes a single stanza followed by a blank line. > +/// > +/// Expects that `repo.file_type =3D=3D APTRepositoryFileType::Sources`. > +fn write_stanza(repo: &APTRepository, w: &mut dyn Write) -> Result<(), E= rror> { > + if repo.file_type !=3D APTRepositoryFileType::Sources { > + bail!("not a .sources repository"); > + } > + > + if !repo.comment.is_empty() { > + for line in repo.comment.lines() { > + writeln!(w, "#{}", line)?; > + } > + } > + > + write!(w, "Types:")?; > + repo.types > + .iter() > + .try_for_each(|package_type| write!(w, " {}", package_type))?; > + writeln!(w)?; > + > + writeln!(w, "URIs: {}", repo.uris.join(" "))?; > + writeln!(w, "Suites: {}", repo.suites.join(" "))?; > + > + if !repo.components.is_empty() { > + writeln!(w, "Components: {}", repo.components.join(" "))?; > + } > + > + for option in repo.options.iter() { > + writeln!(w, "{}: {}", option.key, option.values.join(" "))?; > + } > + > + writeln!(w)?; > + > + Ok(()) > +} > diff --git a/src/types.rs b/src/types.rs > new file mode 100644 > index 0000000..bbd8e7e > --- /dev/null > +++ b/src/types.rs > @@ -0,0 +1,246 @@ > +use std::convert::TryFrom; > +use std::fmt::Display; > + > +use anyhow::{bail, Error}; > +use serde::{Deserialize, Serialize}; > + > +use proxmox::api::api; > + > +#[api] > +#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq)] > +#[serde(rename_all =3D "lowercase")] > +pub enum APTRepositoryFileType { > + /// One-line-style format > + List, > + /// DEB822-style format > + Sources, > +} > + > +impl TryFrom<&str> for APTRepositoryFileType { > + type Error =3D Error; > + > + fn try_from(string: &str) -> Result { > + match string { > + "list" =3D> Ok(APTRepositoryFileType::List), > + "sources" =3D> Ok(APTRepositoryFileType::Sources), > + _ =3D> bail!("invalid file type '{}'", string), > + } > + } > +} > + > +impl Display for APTRepositoryFileType { > + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { > + match self { > + APTRepositoryFileType::List =3D> write!(f, "list"), > + APTRepositoryFileType::Sources =3D> write!(f, "sources"), > + } > + } > +} > + > +#[api] > +#[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq)] > +#[serde(rename_all =3D "kebab-case")] > +pub enum APTRepositoryPackageType { > + /// Debian package > + Deb, > + /// Debian source package > + DebSrc, > +} > + > +impl TryFrom<&str> for APTRepositoryPackageType { > + type Error =3D Error; > + > + fn try_from(string: &str) -> Result { > + match string { > + "deb" =3D> Ok(APTRepositoryPackageType::Deb), > + "deb-src" =3D> Ok(APTRepositoryPackageType::DebSrc), > + _ =3D> bail!("invalid package type '{}'", string), > + } > + } > +} > + > +impl Display for APTRepositoryPackageType { > + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { > + match self { > + APTRepositoryPackageType::Deb =3D> write!(f, "deb"), > + APTRepositoryPackageType::DebSrc =3D> write!(f, "deb-src"), > + } > + } > +} > + > +#[api( > + properties: { > + Key: { > + description: "Option key.", > + type: String, > + }, > + Values: { > + description: "Option values.", > + type: Array, > + items: { > + description: "Value.", > + type: String, > + }, > + }, > + }, > +)] > +#[derive(Debug, Clone, Serialize, Deserialize)] > +#[serde(rename_all =3D "PascalCase")] // for consistency > +/// Additional options for an APT repository. > +/// Used for both single- and mutli-value options. > +pub struct APTRepositoryOption { > + /// Option key. > + pub key: String, > + /// Option value(s). > + pub values: Vec, > +} > + > +#[api( > + properties: { > + Types: { > + description: "List of package types.", > + type: Array, > + items: { > + type: APTRepositoryPackageType, > + }, > + }, > + URIs: { > + description: "List of repository URIs.", > + type: Array, > + items: { > + description: "Repository URI.", > + type: String, > + }, > + }, > + Suites: { > + description: "List of distributions.", > + type: Array, > + items: { > + description: "Package distribution.", > + type: String, > + }, > + }, > + Components: { > + description: "List of repository components.", > + type: Array, > + items: { > + description: "Repository component.", > + type: String, > + }, > + }, > + Options: { > + type: Array, > + optional: true, > + items: { > + type: APTRepositoryOption, > + }, > + }, > + Comment: { > + description: "Associated comment.", > + type: String, > + optional: true, > + }, > + FileType: { > + type: APTRepositoryFileType, > + }, > + Enabled: { > + description: "Whether the repository is enabled or not.", > + type: Boolean, > + }, > + }, > +)] > +#[derive(Debug, Clone, Serialize, Deserialize)] > +#[serde(rename_all =3D "PascalCase")] > +/// Describes an APT repository. > +pub struct APTRepository { > + /// List of package types. > + #[serde(skip_serializing_if =3D "Vec::is_empty")] > + pub types: Vec, > + > + /// List of repository URIs. > + #[serde(skip_serializing_if =3D "Vec::is_empty")] > + #[serde(rename =3D "URIs")] > + pub uris: Vec, > + > + /// List of package distributions. > + #[serde(skip_serializing_if =3D "Vec::is_empty")] > + pub suites: Vec, > + > + /// List of repository components. > + #[serde(skip_serializing_if =3D "Vec::is_empty")] > + pub components: Vec, > + > + /// Additional options. > + #[serde(skip_serializing_if =3D "Vec::is_empty")] > + pub options: Vec, > + > + /// Associated comment. > + #[serde(skip_serializing_if =3D "String::is_empty")] > + pub comment: String, > + > + /// Format of the defining file. > + pub file_type: APTRepositoryFileType, > + > + /// Whether the repository is enabled or not. > + pub enabled: bool, > +} > + > +#[api( > + properties: { > + file_type: { > + type: APTRepositoryFileType, > + }, > + repositories: { > + description: "List of APT repositories.", > + type: Array, > + items: { > + type: APTRepository, > + }, > + }, > + digest: { > + description: "Digest for the content of the file.", > + optional: true, > + type: Array, > + items: { > + description: "Digest byte.", > + type: Integer, > + }, > + }, > + }, > +)] > +#[derive(Debug, Clone, Serialize, Deserialize)] > +#[serde(rename_all =3D "lowercase")] > +/// Represents an abstract APT repository file. > +pub struct APTRepositoryFile { > + /// The path to the file. > + pub path: String, > + /// The type of the file. > + pub file_type: APTRepositoryFileType, > + /// List of repositories in the file. > + pub repositories: Vec, > + /// Digest of the original contents. > + pub digest: Option<[u8; 32]>, > +} > + > +#[api] > +#[derive(Debug, Clone, Serialize, Deserialize)] > +#[serde(rename_all =3D "lowercase")] > +/// Error type for problems with APT repository files. > +pub struct APTRepositoryFileError { > + /// The path to the problematic file. > + pub path: String, > + /// The error message. > + pub error: String, > +} > + > +impl Display for APTRepositoryFileError { > + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { > + write!(f, "proxmox-apt error for '{}' - {}", self.path, self.err= or) > + } > +} > + > +impl std::error::Error for APTRepositoryFileError { > + fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { > + None > + } > +} > diff --git a/tests/repositories.rs b/tests/repositories.rs > new file mode 100644 > index 0000000..aca05ef > --- /dev/null > +++ b/tests/repositories.rs > @@ -0,0 +1,129 @@ > +use std::path::PathBuf; > + > +use anyhow::{bail, format_err, Error}; > + > +use proxmox_apt::repositories::write_repositories; > +use proxmox_apt::types::APTRepositoryFile; > + > +#[test] > +fn test_parse_write() -> Result<(), Error> { > + let test_dir =3D std::env::current_dir()?.join("tests"); > + let read_dir =3D test_dir.join("sources.list.d"); > + let write_dir =3D test_dir.join("sources.list.d.actual"); > + let expected_dir =3D test_dir.join("sources.list.d.expected"); > + > + if write_dir.is_dir() { > + std::fs::remove_dir_all(&write_dir) > + .map_err(|err| format_err!("unable to remove dir {:?} - {}",= write_dir, err))?; > + } > + > + std::fs::create_dir_all(&write_dir) > + .map_err(|err| format_err!("unable to create dir {:?} - {}", wri= te_dir, err))?; > + > + let mut files =3D vec![]; > + let mut errors =3D vec![]; > + > + for entry in std::fs::read_dir(read_dir)? { > + let path =3D entry?.path(); > + > + match APTRepositoryFile::new(&path)? { > + Some(mut file) =3D> match file.parse() { > + Ok(()) =3D> files.push(file), > + Err(err) =3D> errors.push(err), > + }, > + None =3D> bail!("unexpected None for '{:?}'", path), > + } > + } > + > + assert!(errors.is_empty()); > + > + for file in files.iter_mut() { > + let path =3D PathBuf::from(&file.path); > + let new_path =3D write_dir.join(path.file_name().unwrap()); > + file.path =3D new_path.into_os_string().into_string().unwrap(); > + file.digest =3D None; > + } > + > + write_repositories(&files).map_err(|err| format_err!("{:?}", err))?; > + > + let mut expected_count =3D 0; > + > + for entry in std::fs::read_dir(expected_dir)? { > + expected_count +=3D 1; > + > + let expected_path =3D entry?.path(); > + let actual_path =3D write_dir.join(expected_path.file_name().unw= rap()); > + > + let expected_contents =3D std::fs::read(&expected_path) > + .map_err(|err| format_err!("unable to read {:?} - {}", expec= ted_path, err))?; > + > + let actual_contents =3D std::fs::read(&actual_path) > + .map_err(|err| format_err!("unable to read {:?} - {}", actua= l_path, err))?; > + > + assert_eq!( > + expected_contents, actual_contents, > + "Use\n\ndiff {:?} {:?}\n\nif you're not fluent in byte decim= als", > + expected_path, actual_path > + ); > + } > + > + let actual_count =3D std::fs::read_dir(write_dir)?.count(); > + > + assert_eq!(expected_count, actual_count); > + > + Ok(()) > +} > + > +#[test] > +fn test_digest() -> Result<(), Error> { > + let test_dir =3D std::env::current_dir()?.join("tests"); > + let read_dir =3D test_dir.join("sources.list.d"); > + let write_dir =3D test_dir.join("sources.list.d.digest"); > + > + if write_dir.is_dir() { > + std::fs::remove_dir_all(&write_dir) > + .map_err(|err| format_err!("unable to remove dir {:?} - {}",= write_dir, err))?; > + } > + > + std::fs::create_dir_all(&write_dir) > + .map_err(|err| format_err!("unable to create dir {:?} - {}", wri= te_dir, err))?; > + > + let path =3D read_dir.join("standard.list"); > + > + let mut file =3D APTRepositoryFile::new(&path)?.unwrap(); > + file.parse()?; > + > + let new_path =3D write_dir.join(path.file_name().unwrap()); > + file.path =3D new_path.clone().into_os_string().into_string().unwrap= (); > + > + let old_digest =3D file.digest.unwrap(); > + let mut files =3D vec![file]; > + > + // file does not exist yet... > + assert!(files.first().unwrap().read_with_digest().is_err()); > + assert!(write_repositories(&files).is_err()); > + > + // ...but it should work if there's no digest > + files[0].digest =3D None; > + write_repositories(&files).map_err(|err| format_err!("{:?}", err))?; > + > + // overwrite with old contents... > + std::fs::copy(path, new_path)?; > + > + // modify the repo > + let mut file =3D files.first_mut().unwrap(); > + let mut repo =3D file.repositories.first_mut().unwrap(); > + repo.enabled =3D !repo.enabled; > + > + // ...then it should work > + file.digest =3D Some(old_digest); > + write_repositories(&files).map_err(|err| format_err!("{:?}", err))?; > + > + // expect a different digest, because the repo was modified > + let (_, new_digest) =3D files.first().unwrap().read_with_digest()?; > + assert_ne!(old_digest, new_digest); > + > + assert!(write_repositories(&files).is_err()); > + > + Ok(()) > +} > diff --git a/tests/sources.list.d.expected/absolute_suite.list b/tests/so= urces.list.d.expected/absolute_suite.list > new file mode 100644 > index 0000000..af6b966 > --- /dev/null > +++ b/tests/sources.list.d.expected/absolute_suite.list > @@ -0,0 +1,5 @@ > +# From Debian Administrator's Handbook > +deb http://packages.falcot.com/ updates/=20 > + > +deb http://user.name@packages.falcot.com:80/ internal/=20 > + > diff --git a/tests/sources.list.d.expected/absolute_suite.sources b/tests= /sources.list.d.expected/absolute_suite.sources > new file mode 100644 > index 0000000..51e4d56 > --- /dev/null > +++ b/tests/sources.list.d.expected/absolute_suite.sources > @@ -0,0 +1,5 @@ > +# From Debian Administrator's Handbook > +Types: deb > +URIs: http://packages.falcot.com/ > +Suites: updates/ internal/ > + > diff --git a/tests/sources.list.d.expected/case.sources b/tests/sources.l= ist.d.expected/case.sources > new file mode 100644 > index 0000000..307aab6 > --- /dev/null > +++ b/tests/sources.list.d.expected/case.sources > @@ -0,0 +1,16 @@ > +# comment in here > +Types: deb deb-src > +URIs: http://ftp.at.debian.org/debian > +Suites: buster-updates > +Components: main contrib > +languages: it de fr > +Enabled: false > +languages-Add: ja > +languages-Remove: de > + > +# comment in here > +Types: deb deb-src > +URIs: http://ftp.at.debian.org/debian > +Suites: buster > +Components: main contrib > + > diff --git a/tests/sources.list.d.expected/multiline.sources b/tests/sour= ces.list.d.expected/multiline.sources > new file mode 100644 > index 0000000..d96acea > --- /dev/null > +++ b/tests/sources.list.d.expected/multiline.sources > @@ -0,0 +1,10 @@ > +# comment in here > +Types: deb deb-src > +URIs: http://ftp.at.debian.org/debian > +Suites: buster buster-updates > +Components: main contrib > +Languages: it de fr > +Enabled: false > +Languages-Add: ja > +Languages-Remove: de > + > diff --git a/tests/sources.list.d.expected/options_comment.list b/tests/s= ources.list.d.expected/options_comment.list > new file mode 100644 > index 0000000..8c905c0 > --- /dev/null > +++ b/tests/sources.list.d.expected/options_comment.list > @@ -0,0 +1,6 @@ > +# comment > +deb [ lang=3Dit,de arch=3Damd64 ] http://ftp.at.debian.org/debian buster= main contrib > + > +# non-free :( > +deb [ lang=3Dit,de arch=3Damd64 lang+=3Dfr lang-=3Dde ] http://ftp.at.de= bian.org/debian buster non-free > + > diff --git a/tests/sources.list.d.expected/pbs-enterprise.list b/tests/so= urces.list.d.expected/pbs-enterprise.list > new file mode 100644 > index 0000000..acb2990 > --- /dev/null > +++ b/tests/sources.list.d.expected/pbs-enterprise.list > @@ -0,0 +1,2 @@ > +deb https://enterprise.proxmox.com/debian/pbs buster pbs-enterprise > + > diff --git a/tests/sources.list.d.expected/pve.list b/tests/sources.list.= d.expected/pve.list > new file mode 100644 > index 0000000..127a49a > --- /dev/null > +++ b/tests/sources.list.d.expected/pve.list > @@ -0,0 +1,13 @@ > +deb http://ftp.debian.org/debian buster main contrib > + > +deb http://ftp.debian.org/debian buster-updates main contrib > + > +# PVE pve-no-subscription repository provided by proxmox.com, > +# NOT recommended for production use > +deb http://download.proxmox.com/debian/pve buster pve-no-subscription > + > +# deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise > + > +# security updates > +deb http://security.debian.org/debian-security buster/updates main contr= ib > + > diff --git a/tests/sources.list.d.expected/standard.list b/tests/sources.= list.d.expected/standard.list > new file mode 100644 > index 0000000..63c1b60 > --- /dev/null > +++ b/tests/sources.list.d.expected/standard.list > @@ -0,0 +1,7 @@ > +deb http://ftp.at.debian.org/debian buster main contrib > + > +deb http://ftp.at.debian.org/debian buster-updates main contrib > + > +# security updates > +deb http://security.debian.org buster/updates main contrib > + > diff --git a/tests/sources.list.d.expected/standard.sources b/tests/sourc= es.list.d.expected/standard.sources > new file mode 100644 > index 0000000..56ce280 > --- /dev/null > +++ b/tests/sources.list.d.expected/standard.sources > @@ -0,0 +1,11 @@ > +Types: deb > +URIs: http://ftp.at.debian.org/debian > +Suites: buster buster-updates > +Components: main contrib > + > +# security updates > +Types: deb > +URIs: http://security.debian.org > +Suites: buster/updates > +Components: main contrib > + > diff --git a/tests/sources.list.d/absolute_suite.list b/tests/sources.lis= t.d/absolute_suite.list > new file mode 100644 > index 0000000..b690d30 > --- /dev/null > +++ b/tests/sources.list.d/absolute_suite.list > @@ -0,0 +1,4 @@ > +# From Debian Administrator's Handbook > +deb http://packages.falcot.com/ updates/ > + > +deb http://user.name@packages.falcot.com:80/ internal/ > diff --git a/tests/sources.list.d/absolute_suite.sources b/tests/sources.= list.d/absolute_suite.sources > new file mode 100644 > index 0000000..51e4d56 > --- /dev/null > +++ b/tests/sources.list.d/absolute_suite.sources > @@ -0,0 +1,5 @@ > +# From Debian Administrator's Handbook > +Types: deb > +URIs: http://packages.falcot.com/ > +Suites: updates/ internal/ > + > diff --git a/tests/sources.list.d/case.sources b/tests/sources.list.d/cas= e.sources > new file mode 100644 > index 0000000..8979d0c > --- /dev/null > +++ b/tests/sources.list.d/case.sources > @@ -0,0 +1,17 @@ > +tYpeS: deb deb-src > +uRis: http://ftp.at.debian.org/debian > +suiTes: buster-updates > +# comment in here > +CompOnentS: main contrib > +languages: it > + de > + fr > +Enabled: off > +languages-Add: ja > +languages-Remove: de > + > +types: deb deb-src > +Uris: http://ftp.at.debian.org/debian > +suites: buster > +# comment in here > +components: main contrib > diff --git a/tests/sources.list.d/multiline.sources b/tests/sources.list.= d/multiline.sources > new file mode 100644 > index 0000000..bdbce29 > --- /dev/null > +++ b/tests/sources.list.d/multiline.sources > @@ -0,0 +1,11 @@ > +Types: deb deb-src > +URIs: http://ftp.at.debian.org/debian > +Suites: buster buster-updates > +# comment in here > +Components: main contrib > +Languages: it > + de > + fr > +Enabled: off > +Languages-Add: ja > +Languages-Remove: de > diff --git a/tests/sources.list.d/options_comment.list b/tests/sources.li= st.d/options_comment.list > new file mode 100644 > index 0000000..6b73053 > --- /dev/null > +++ b/tests/sources.list.d/options_comment.list > @@ -0,0 +1,3 @@ > +deb [ lang=3Dit,de arch=3Damd64 ] http://ftp.at.debian.org/debian buster= main contrib # comment > +deb [ lang=3Dit,de arch=3Damd64 lang+=3Dfr lang-=3Dde ] http://ftp.at.de= bian.org/debian buster non-free # non-free :( > + > diff --git a/tests/sources.list.d/pbs-enterprise.list b/tests/sources.lis= t.d/pbs-enterprise.list > new file mode 100644 > index 0000000..5f8763c > --- /dev/null > +++ b/tests/sources.list.d/pbs-enterprise.list > @@ -0,0 +1 @@ > +deb https://enterprise.proxmox.com/debian/pbs buster pbs-enterprise > diff --git a/tests/sources.list.d/pve.list b/tests/sources.list.d/pve.lis= t > new file mode 100644 > index 0000000..6213f72 > --- /dev/null > +++ b/tests/sources.list.d/pve.list > @@ -0,0 +1,10 @@ > +deb http://ftp.debian.org/debian buster main contrib > +deb http://ftp.debian.org/debian buster-updates main contrib > + > +# PVE pve-no-subscription repository provided by proxmox.com, > +# NOT recommended for production use > +deb http://download.proxmox.com/debian/pve buster pve-no-subscription > +# deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise > + > +# security updates > +deb http://security.debian.org/debian-security buster/updates main contr= ib > diff --git a/tests/sources.list.d/standard.list b/tests/sources.list.d/st= andard.list > new file mode 100644 > index 0000000..26db887 > --- /dev/null > +++ b/tests/sources.list.d/standard.list > @@ -0,0 +1,6 @@ > +deb http://ftp.at.debian.org/debian buster main contrib > + > +deb http://ftp.at.debian.org/debian buster-updates main contrib > + > +# security updates > +deb http://security.debian.org buster/updates main contrib > diff --git a/tests/sources.list.d/standard.sources b/tests/sources.list.d= /standard.sources > new file mode 100644 > index 0000000..605202e > --- /dev/null > +++ b/tests/sources.list.d/standard.sources > @@ -0,0 +1,10 @@ > +Types: deb > +URIs: http://ftp.at.debian.org/debian > +Suites: buster buster-updates > +Components: main contrib > + > +# security updates > +Types: deb > +URIs: http://security.debian.org > +Suites: buster/updates > +Components: main contrib > --=20 > 2.20.1 >=20 >=20 >=20 > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >=20 >=20 >=20