From: Fabian Ebner <f.ebner@proxmox.com>
To: Wolfgang Bumiller <w.bumiller@proxmox.com>
Cc: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com
Subject: Re: [pve-devel] [PATCH v6 proxmox-apt 04/11] add check_repositories function
Date: Fri, 18 Jun 2021 08:42:47 +0200 [thread overview]
Message-ID: <035287b8-415b-d96d-7896-170edc4a7a77@proxmox.com> (raw)
In-Reply-To: <20210617083902.32lau57pd3gjt5qe@wobu-vie.proxmox.com>
Am 17.06.21 um 10:39 schrieb Wolfgang Bumiller:
> some non-blocking cleanups in case you do another version:
>
> On Fri, Jun 11, 2021 at 01:43:53PM +0200, Fabian Ebner wrote:
>> which checks for bad suites and official URIs.
>>
>> Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
>> ---
>>
>> Changes from v5:
>> * split out host_from_uri helper and also handle userinfo and port
>> * test an offical URI with port
>> * match all *.debian.org and *.proxmox.com as official to avoid (future)
>> false negatives.
>> * add bookworm and trixie codenames to the list of new_suites
>>
>> src/repositories/check.rs | 174 +++++++++++++++++++++-
>> src/repositories/mod.rs | 19 ++-
>> src/types.rs | 19 +++
>> tests/repositories.rs | 97 +++++++++++-
>> tests/sources.list.d.expected/bad.sources | 30 ++++
>> tests/sources.list.d/bad.sources | 29 ++++
>> 6 files changed, 364 insertions(+), 4 deletions(-)
>> create mode 100644 tests/sources.list.d.expected/bad.sources
>> create mode 100644 tests/sources.list.d/bad.sources
>>
>> diff --git a/src/repositories/check.rs b/src/repositories/check.rs
>> index a682b69..585c28d 100644
>> --- a/src/repositories/check.rs
>> +++ b/src/repositories/check.rs
>> @@ -1,6 +1,45 @@
>> use anyhow::{bail, Error};
>>
>> -use crate::types::{APTRepository, APTRepositoryFileType, APTRepositoryPackageType};
>> +use crate::types::{
>> + APTRepository, APTRepositoryFile, APTRepositoryFileType, APTRepositoryInfo,
>> + APTRepositoryPackageType,
>> +};
>> +
>> +/// Splits the suite into its base part and variant.
>> +fn suite_variant(suite: &str) -> (&str, &str) {
>> + let variants = ["-backports-sloppy", "-backports", "-updates", "/updates"];
>> +
>> + for variant in variants.iter() {
>> + if let Some(base) = suite.strip_suffix(variant) {
>> + return (base, variant);
>> + }
>> + }
>> +
>> + (suite, "")
>> +}
>> +
>> +/// Get the host part from a given URI.
>> +fn host_from_uri(uri: &str) -> Option<&str> {
>> + if let Some(begin) = uri.find("://") {
>
> You could shorten this via `?` (since the function itself also returns
> an `Option`):
>
> let begin = uri.find("://")?;
>
>> + let mut host = uri.split_at(begin + 3).1;
>> +
>> + if let Some(end) = host.find('/') {
>> + host = host.split_at(end).0;
>
> Personally I'd prefer `host = &host[..end]`, but it probably compiles to
> the same code in the end.
>
>> + }
>> +
>> + if let Some(begin) = host.find('@') {
>> + host = host.split_at(begin + 1).1;
>
> (Similarly: `host = &host[(begin + 1)..]`)
>
>> + }
>> +
>> + if let Some(end) = host.find(':') {
>> + host = host.split_at(end).0;
>> + }
>> +
>> + return Some(host);
>> + }
>> +
>> + None
>> +}
>>
>> impl APTRepository {
>> /// Makes sure that all basic properties of a repository are present and
>> @@ -102,4 +141,137 @@ impl APTRepository {
>> false
>> }
>> }
>> +
>> + /// Checks if old or unstable suites are configured and also that the
>> + /// `stable` keyword is not used.
>> + fn check_suites(&self, add_info: &mut dyn FnMut(String, String)) {
>> + let old_suites = [
>> + "lenny",
>> + "squeeze",
>> + "wheezy",
>> + "jessie",
>> + "stretch",
>> + "oldoldstable",
>> + "oldstable",
>> + ];
>> +
>> + let next_suite = "bullseye";
>> +
>> + let new_suites = [
>> + "bookworm",
>> + "trixie",
>> + "testing",
>> + "unstable",
>> + "sid",
>> + "experimental",
>> + ];
>> +
>> + if self
>> + .types
>> + .iter()
>> + .any(|package_type| *package_type == APTRepositoryPackageType::Deb)
>> + {
>> + for suite in self.suites.iter() {
>
> maybe cache `suite_variant(suite).0` at this point
>
> let variant = suite_variant(suite).0;
>
>> + if old_suites
>> + .iter()
>> + .any(|base_suite| suite_variant(suite).0 == *base_suite)
>
> ^ then this could be
>
> if old_suites.contains(&variant) {
>
> I think
>
>> + {
>> + add_info(
>> + "warning".to_string(),
>> + format!("old suite '{}' configured!", suite),
>> + );
>> + }
>> +
>> + if suite_variant(suite).0 == next_suite {
>> + add_info(
>> + "ignore-pre-upgrade-warning".to_string(),
>> + format!("suite '{}' should not be used in production!", suite),
>> + );
>> + }
>> +
>> + if new_suites
>> + .iter()
>> + .any(|base_suite| suite_variant(suite).0 == *base_suite)
>
> ^ same
>
>> + {
>> + add_info(
>> + "warning".to_string(),
>> + format!("suite '{}' should not be used in production!", suite),
>> + );
>> + }
>> +
>> + if suite_variant(suite).0 == "stable" {
>> + add_info(
>> + "warning".to_string(),
>> + "use the name of the stable distribution instead of 'stable'!".to_string(),
>> + );
>> + }
>> + }
>> + }
>> + }
>> +
>> + /// Checks if an official host is configured in the repository.
>> + fn check_uris(&self) -> Option<(String, String)> {
>> + let official_host = |domains: &Vec<&str>| match domains.split_last() {
>
> Drop this entire beast (see below), but as a review of it:
> You can use the slice[1] & rest[2] pattern syntax here:
>
> #[allow(clippy::match_like_matches_macro)]
> match domains[..] { // the `[..]` part is required here
> [.., "proxmox", "com"] => true,
> [.., "debian", "org"] => true,
> _ => false,
> }
>
> Or more concise (but I do find the above a bit quicker to glance over,
> hence the 'clippy' hint ;-) ):
>
> matches!(domains[..], [.., "proxmox", "com"] | [.., "debian", "org"]);
>
> [1] https://doc.rust-lang.org/reference/patterns.html#slice-patterns
> [2] https://doc.rust-lang.org/reference/patterns.html#rest-patterns
>
>> + Some((last, rest)) => match rest.split_last() {
>> + Some((second_to_last, _rest)) => {
>> + (*last == "org" && *second_to_last == "debian")
>> + || (*last == "com" && *second_to_last == "proxmox")
>> + }
>> + None => false,
>> + },
>> + None => false,
>> + };
>> +
>> + for uri in self.uris.iter() {
>> + if let Some(host) = host_from_uri(uri) {
>> + let domains = host.split('.').collect();
>
> ^ But instead of building a vector here, why not just do:
>
> if host == "proxmox.com" || host.ends_with(".proxmox.com")
> || host == "debian.org" || host.ends_with(".debian.org")
> {
> ...
> }
>
Misses FQDNs? Thanks for the tips, I was not aware that one can do
tail-matching with the matches! macro.
>> +
>> + if official_host(&domains) {
>> + return Some(("badge".to_string(), "official host name".to_string()));
>> + }
>> + }
>> + }
>> +
>> + None
>> + }
>> +}
>> +
>> +impl APTRepositoryFile {
>> + /// Checks if old or unstable suites are configured and also that the
>> + /// `stable` keyword is not used.
>> + pub fn check_suites(&self) -> Vec<APTRepositoryInfo> {
>> + let mut infos = vec![];
>> +
>> + for (n, repo) in self.repositories.iter().enumerate() {
>> + let mut add_info = |kind, message| {
>> + infos.push(APTRepositoryInfo {
>> + path: self.path.clone(),
>> + number: n + 1,
>> + kind,
>> + message,
>> + })
>> + };
>> + repo.check_suites(&mut add_info);
>
> ^ minor nit:
> the `check_suites` you're calling here is only called at this one spot
> and private, so personally I'd prefer an `impl FnMut` or generic over a
> trait object, (also you could inline the closure here)
>
next prev parent reply other threads:[~2021-06-18 6:42 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-11 11:43 [pve-devel] [PATCH-SERIES v6] APT repositories API/UI Fabian Ebner
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 01/11] initial commit Fabian Ebner
2021-06-18 8:14 ` Fabian Grünbichler
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 02/11] add files for Debian packaging Fabian Ebner
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 03/11] add functions to check for Proxmox repositories Fabian Ebner
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 04/11] add check_repositories function Fabian Ebner
2021-06-17 8:39 ` Wolfgang Bumiller
2021-06-18 6:42 ` Fabian Ebner [this message]
2021-06-17 14:16 ` Fabian Grünbichler
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 05/11] add common_digest helper Fabian Ebner
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 06/11] add release_upgrade function and constants for the current and upgrade suite Fabian Ebner
2021-06-17 14:16 ` [pve-devel] [pbs-devel] " Fabian Grünbichler
2021-06-18 6:50 ` Fabian Ebner
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 07/11] bump version to 0.1.1-1 Fabian Ebner
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 08/11] update for bullseye Fabian Ebner
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 09/11] bump version to 1.0.0-1 Fabian Ebner
2021-06-11 11:43 ` [pve-devel] [PATCH v6 proxmox-apt 10/11] allow upgrade to bullseye Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-apt 11/11] bump version to 0.2.0-1 Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-widget-toolkit 1/3] add UI for APT repositories Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-widget-toolkit 2/3] APT repositories: add warnings Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-widget-toolkit 3/3] add upgrade button Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-backup 1/6] depend on new proxmox-apt crate Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-backup 2/6] api: apt: add repositories call Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-backup 3/6] ui: add APT repositories Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-backup 4/6] api: apt: add check_repositories_call Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 proxmox-backup 5/6] add upgrade_repositories call Fabian Ebner
2021-06-18 8:21 ` Fabian Grünbichler
2021-06-11 11:44 ` [pve-devel] [RFC v6 proxmox-backup 6/6] enable release upgrade for package repositories Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-rs 1/4] initial commit Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-rs 2/4] add files for Debian packaging Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-rs 3/4] apt: add upgrade_repositories call Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-rs 4/4] depend on proxmox-apt 0.2.0 Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-manager 1/5] api: apt: add call to list repositories Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-manager 2/5] ui: add panel for listing APT repositories Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-manager 3/5] api: apt: add call for repository check Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-manager 4/5] api: apt: add upgrade repos call Fabian Ebner
2021-06-11 11:44 ` [pve-devel] [PATCH v6 pve-manager 5/5] ui: node config: enable release upgrade button for package repositories Fabian Ebner
2021-06-18 6:44 [pve-devel] [PATCH v6 proxmox-apt 04/11] add check_repositories function Wolfgang Bumiller
2021-06-18 6:53 ` Fabian Ebner
2021-06-18 6:56 Wolfgang Bumiller
2021-06-18 6:58 ` Fabian Ebner
2021-06-18 7:16 Wolfgang Bumiller
2021-06-18 7:26 ` Fabian Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=035287b8-415b-d96d-7896-170edc4a7a77@proxmox.com \
--to=f.ebner@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
--cc=pve-devel@lists.proxmox.com \
--cc=w.bumiller@proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox