all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements
@ 2022-09-15 13:09 Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 1/4] mirror: use xz multi decoder Fabian Grünbichler
                   ` (8 more replies)
  0 siblings, 9 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 13:09 UTC (permalink / raw)
  To: pve-devel

this series adds some features and fixes some issues that pop up when
attempting to mirror Ubuntu repositories, and should also improve
resilience with other third-party repositories.

tested with Ubuntu Jammy (main, security and updates repositories).




^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH proxmox-offline-mirror 1/4] mirror: use xz multi decoder
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
@ 2022-09-15 13:09 ` Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 1/3] release: add Commands file reference type Fabian Grünbichler
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 13:09 UTC (permalink / raw)
  To: pve-devel

Ubuntu's Packages.xz files require it, because they contain multiple
streams.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 src/mirror.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/mirror.rs b/src/mirror.rs
index 78a493b..14b0c6a 100644
--- a/src/mirror.rs
+++ b/src/mirror.rs
@@ -277,7 +277,7 @@ fn fetch_index_file(
             &buf[..]
         }
         Some(CompressionType::Lzma) | Some(CompressionType::Xz) => {
-            let mut xz = xz2::read::XzDecoder::new(raw);
+            let mut xz = xz2::read::XzDecoder::new_multi_decoder(raw);
             xz.read_to_end(&mut buf)?;
             &buf[..]
         }
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH proxmox-apt 1/3] release: add Commands file reference type
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 1/4] mirror: use xz multi decoder Fabian Grünbichler
@ 2022-09-15 13:09 ` Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 2/4] mirror: skip failed, non Packages references Fabian Grünbichler
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 13:09 UTC (permalink / raw)
  To: pve-devel

used by command-not-found to lookup which package ships which command.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---

Notes:
    this is technically a breaking change, but the only user of this already
    has a fallback match arm. I wonder whether we should mark this as
    non-exhaustive?

 src/deb822/release_file.rs | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/src/deb822/release_file.rs b/src/deb822/release_file.rs
index 6668450..2b7245b 100644
--- a/src/deb822/release_file.rs
+++ b/src/deb822/release_file.rs
@@ -51,6 +51,8 @@ pub type Component = String;
 /// `Packages` and `Sources` will contain further reference to binary or source package files.
 ///  These are handled in `PackagesFile` and `SourcesFile` respectively.
 pub enum FileReferenceType {
+    /// A `Commands` index listing command to package mappings
+    Commands(Architecture, Option<CompressionType>),
     /// A `Contents` index listing contents of binary packages
     Contents(Architecture, Option<CompressionType>),
     /// A `Contents` index listing contents of binary udeb packages
@@ -123,6 +125,20 @@ impl FileReferenceType {
                         Ok(FileReferenceType::Unknown)
                     }
                 }
+                "cnf" => {
+                    if let Some(rest) = rest.strip_prefix("Commands-") {
+                        if let Some((arch, ext)) = rest.rsplit_once('.') {
+                            Ok(FileReferenceType::Commands(
+                                arch.to_owned(),
+                                FileReferenceType::match_compression(ext).ok().flatten(),
+                            ))
+                        } else {
+                            Ok(FileReferenceType::Commands(rest.to_owned(), None))
+                        }
+                    } else {
+                        Ok(FileReferenceType::Unknown)
+                    }
+                },
                 "dep11" => {
                     if let Some((_path, ext)) = rest.rsplit_once('.') {
                         Ok(FileReferenceType::Dep11(
@@ -198,7 +214,8 @@ impl FileReferenceType {
 
     pub fn compression(&self) -> Option<CompressionType> {
         match *self {
-            FileReferenceType::Contents(_, comp)
+            FileReferenceType::Commands(_, comp)
+            | FileReferenceType::Contents(_, comp)
             | FileReferenceType::ContentsUdeb(_, comp)
             | FileReferenceType::Packages(_, comp)
             | FileReferenceType::Sources(comp)
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH proxmox-offline-mirror 2/4] mirror: skip failed, non Packages references
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 1/4] mirror: use xz multi decoder Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 1/3] release: add Commands file reference type Fabian Grünbichler
@ 2022-09-15 13:09 ` Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 2/3] release: add 'architecture' helper Fabian Grünbichler
                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 13:09 UTC (permalink / raw)
  To: pve-devel

these contain extra data that is not that important for the main
repository use case - providing deb packages.

if they are not retrievable (e.g., Ubuntu *only* provides some of they
via by-hash, which proxmox-offline-mirror doesn't yet support) a warning
should be enough, instead of failing the whole snapshot creation.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 src/mirror.rs | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/src/mirror.rs b/src/mirror.rs
index 14b0c6a..f910e6a 100644
--- a/src/mirror.rs
+++ b/src/mirror.rs
@@ -534,6 +534,7 @@ pub fn create_snapshot(
 
     let mut packages_size = 0_usize;
     let mut packages_indices = HashMap::new();
+    let mut failed_references = Vec::new();
     for (component, references) in per_component {
         println!("\nFetching indices for component '{component}'");
         let mut component_deb_size = 0;
@@ -555,7 +556,18 @@ pub fn create_snapshot(
                 }
 
                 // this will ensure the uncompressed file will be written locally
-                let res = fetch_index_file(&config, prefix, reference, uncompressed_ref)?;
+                let res = match fetch_index_file(&config, prefix, reference, uncompressed_ref) {
+                    Ok(res) => res,
+                    Err(err) if !reference.file_type.is_package_index() => {
+                        eprintln!(
+                            "Failed to fetch '{:?}' type reference '{}', skipping - {err}",
+                            reference.file_type, reference.path
+                        );
+                        failed_references.push(reference);
+                        continue;
+                    }
+                    Err(err) => bail!(err),
+                };
                 fetch_progress.update(&res);
 
                 if package_index_data.is_none() && reference.file_type.is_package_index() {
@@ -577,6 +589,12 @@ pub fn create_snapshot(
         total_progress += fetch_progress;
     }
     println!("Total deb size: {packages_size}");
+    if !failed_references.is_empty() {
+        eprintln!("Failed to download non-package-index references:");
+        for reference in failed_references {
+            eprintln!("\t{}", reference.path);
+        }
+    }
 
     println!("\nFetching packages..");
     for (basename, references) in packages_indices {
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH proxmox-apt 2/3] release: add 'architecture' helper
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
                   ` (2 preceding siblings ...)
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 2/4] mirror: skip failed, non Packages references Fabian Grünbichler
@ 2022-09-15 13:09 ` Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 3/4] mirror: support acquiring indices by hash Fabian Grünbichler
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 13:09 UTC (permalink / raw)
  To: pve-devel

which returns if a file reference is architecture specific, and for
which architecture it is relevant.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 src/deb822/release_file.rs | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/src/deb822/release_file.rs b/src/deb822/release_file.rs
index 2b7245b..5888728 100644
--- a/src/deb822/release_file.rs
+++ b/src/deb822/release_file.rs
@@ -228,6 +228,22 @@ impl FileReferenceType {
         }
     }
 
+    pub fn architecture(&self) -> Option<&Architecture> {
+        match self {
+            FileReferenceType::Commands(arch, _)
+            | FileReferenceType::Contents(arch, _)
+            | FileReferenceType::ContentsUdeb(arch, _)
+            | FileReferenceType::Packages(arch, _) => Some(arch),
+            FileReferenceType::PseudoRelease(arch) => arch.as_ref(),
+            FileReferenceType::Unknown
+            | FileReferenceType::PDiff
+            | FileReferenceType::Sources(_)
+            | FileReferenceType::Dep11(_)
+            | FileReferenceType::Translation(_)
+            | FileReferenceType::Ignored => None,
+        }
+    }
+
     pub fn is_package_index(&self) -> bool {
         matches!(self, FileReferenceType::Packages(_, _))
     }
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH proxmox-offline-mirror 3/4] mirror: support acquiring indices by hash
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
                   ` (3 preceding siblings ...)
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 2/3] release: add 'architecture' helper Fabian Grünbichler
@ 2022-09-15 13:09 ` Fabian Grünbichler
  2022-09-15 14:34   ` [pve-devel] [PATCH FIXUP proxmox-offline-mirror] clippy fix Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 3/3] release: fix typo in 'Acquire-By-Hash' Fabian Grünbichler
                   ` (3 subsequent siblings)
  8 siblings, 1 reply; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 13:09 UTC (permalink / raw)
  To: pve-devel

requires proxmox-apt > 0.9.1, since earlier versions misdetect by-hash
support in the release file.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 src/mirror.rs | 47 ++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 38 insertions(+), 9 deletions(-)

diff --git a/src/mirror.rs b/src/mirror.rs
index f910e6a..6cbd680 100644
--- a/src/mirror.rs
+++ b/src/mirror.rs
@@ -233,6 +233,7 @@ fn fetch_index_file(
     prefix: &Path,
     reference: &FileReference,
     uncompressed: &FileReference,
+    by_hash: bool,
 ) -> Result<FetchResult, Error> {
     let url = get_dist_url(&config.repository, &reference.path);
     let path = get_dist_path(&config.repository, prefix, &reference.path);
@@ -252,14 +253,36 @@ fn fetch_index_file(
         return Ok(FetchResult { data, fetched: 0 });
     }
 
-    let res = fetch_plain_file(
-        config,
-        &url,
-        &path,
-        reference.size,
-        &reference.checksums,
-        true,
-    )?;
+    let urls = if by_hash {
+        let mut urls = Vec::new();
+        if let Some((base_url, _file_name)) = url.rsplit_once('/') {
+            if let Some(sha512) = reference.checksums.sha512 {
+                urls.push(format!("{base_url}/by-hash/SHA512/{}", hex::encode(sha512)));
+            }
+            if let Some(sha256) = reference.checksums.sha256 {
+                urls.push(format!("{base_url}/by-hash/SHA256/{}", hex::encode(sha256)));
+            }
+        }
+        urls.push(url);
+        urls
+    } else {
+        vec![url]
+    };
+
+    let res = urls
+        .iter()
+        .fold(None, |res, url| match res {
+            Some(Ok(res)) => Some(Ok(res)),
+            _ => Some(fetch_plain_file(
+                config,
+                &url,
+                &path,
+                reference.size,
+                &reference.checksums,
+                true,
+            )),
+        })
+        .ok_or_else(|| format_err!("Failed to retrieve {}", reference.path))??;
 
     let mut buf = Vec::new();
     let raw = res.data_ref();
@@ -556,7 +579,13 @@ pub fn create_snapshot(
                 }
 
                 // this will ensure the uncompressed file will be written locally
-                let res = match fetch_index_file(&config, prefix, reference, uncompressed_ref) {
+                let res = match fetch_index_file(
+                    &config,
+                    prefix,
+                    reference,
+                    uncompressed_ref,
+                    release.aquire_by_hash,
+                ) {
                     Ok(res) => res,
                     Err(err) if !reference.file_type.is_package_index() => {
                         eprintln!(
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH proxmox-apt 3/3] release: fix typo in 'Acquire-By-Hash'
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
                   ` (4 preceding siblings ...)
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 3/4] mirror: support acquiring indices by hash Fabian Grünbichler
@ 2022-09-15 13:09 ` Fabian Grünbichler
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 4/4] mirror: use new architecture helper Fabian Grünbichler
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 13:09 UTC (permalink / raw)
  To: pve-devel

to allow detection of repositories that support downloading indices via
their hash instead of their filename.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 src/deb822/release_file.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/deb822/release_file.rs b/src/deb822/release_file.rs
index 5888728..355b246 100644
--- a/src/deb822/release_file.rs
+++ b/src/deb822/release_file.rs
@@ -342,7 +342,7 @@ impl TryFrom<ReleaseFileRaw> for ReleaseFile {
         parsed.suite = value.suite;
         parsed.version = value.version;
 
-        parsed.aquire_by_hash = match value.extra_fields.get("Aquire-By-Hash") {
+        parsed.aquire_by_hash = match value.extra_fields.get("Acquire-By-Hash") {
             Some(val) => *val == "yes",
             None => false,
         };
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH proxmox-offline-mirror 4/4] mirror: use new architecture helper
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
                   ` (5 preceding siblings ...)
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 3/3] release: fix typo in 'Acquire-By-Hash' Fabian Grünbichler
@ 2022-09-15 13:09 ` Fabian Grünbichler
  2022-09-15 14:36 ` [pve-devel] [PATCH follow-up proxmox-offline-mirror 8/7] mirror: handle indices which are only available compressed Fabian Grünbichler
  2022-09-16 12:31 ` [pve-devel] applied-series: [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Thomas Lamprecht
  8 siblings, 0 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 13:09 UTC (permalink / raw)
  To: pve-devel

in order to avoid having a list of arch-specific references on two
places.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
requires proxmox-apt > 0.9.1 with the new helper

 src/mirror.rs | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/src/mirror.rs b/src/mirror.rs
index 6cbd680..3370ca4 100644
--- a/src/mirror.rs
+++ b/src/mirror.rs
@@ -501,14 +501,14 @@ pub fn create_snapshot(
             || match &reference.file_type {
                 FileReferenceType::Ignored => true,
                 FileReferenceType::PDiff => true, // would require fetching the patches as well
-                FileReferenceType::Contents(arch, _)
-                | FileReferenceType::ContentsUdeb(arch, _)
-                | FileReferenceType::Packages(arch, _)
-                | FileReferenceType::PseudoRelease(Some(arch)) => {
-                    !binary || !config.architectures.contains(arch)
-                }
                 FileReferenceType::Sources(_) => !source,
-                _ => false,
+                _ => {
+                    if let Some(arch) = reference.file_type.architecture() {
+                        !binary || !config.architectures.contains(arch)
+                    } else {
+                        false
+                    }
+                }
             };
         if skip {
             println!("Skipping {}", reference.path);
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH FIXUP proxmox-offline-mirror] clippy fix
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 3/4] mirror: support acquiring indices by hash Fabian Grünbichler
@ 2022-09-15 14:34   ` Fabian Grünbichler
  0 siblings, 0 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 14:34 UTC (permalink / raw)
  To: pve-devel

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
not really important, but can be folded into the patch since it's not
yet applied ;)

 src/mirror.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/mirror.rs b/src/mirror.rs
index f5ee48e..a1fc1a0 100644
--- a/src/mirror.rs
+++ b/src/mirror.rs
@@ -280,7 +280,7 @@ fn fetch_index_file(
             Some(Ok(res)) => Some(Ok(res)),
             _ => Some(fetch_plain_file(
                 config,
-                &url,
+                url,
                 &path,
                 reference.size,
                 &reference.checksums,
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] [PATCH follow-up proxmox-offline-mirror 8/7] mirror: handle indices which are only available compressed
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
                   ` (6 preceding siblings ...)
  2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 4/4] mirror: use new architecture helper Fabian Grünbichler
@ 2022-09-15 14:36 ` Fabian Grünbichler
  2022-09-16 12:31 ` [pve-devel] applied-series: [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Thomas Lamprecht
  8 siblings, 0 replies; 11+ messages in thread
From: Fabian Grünbichler @ 2022-09-15 14:36 UTC (permalink / raw)
  To: pve-devel

there are repositories out there that not only skip serving the
uncompressed version, but not even reference it in their Release
file(s).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
best viewed with -w, the signal-desktop repository is one such example.

 src/mirror.rs | 56 ++++++++++++++++++++++++++++-----------------------
 1 file changed, 31 insertions(+), 25 deletions(-)

diff --git a/src/mirror.rs b/src/mirror.rs
index 3370ca4..f5ee48e 100644
--- a/src/mirror.rs
+++ b/src/mirror.rs
@@ -232,25 +232,30 @@ fn fetch_index_file(
     config: &ParsedMirrorConfig,
     prefix: &Path,
     reference: &FileReference,
-    uncompressed: &FileReference,
+    uncompressed: Option<&FileReference>,
     by_hash: bool,
 ) -> Result<FetchResult, Error> {
     let url = get_dist_url(&config.repository, &reference.path);
     let path = get_dist_path(&config.repository, prefix, &reference.path);
-    let uncompressed_path = get_dist_path(&config.repository, prefix, &uncompressed.path);
-
-    if config.pool.contains(&reference.checksums) && config.pool.contains(&uncompressed.checksums) {
-        let data = config
-            .pool
-            .get_contents(&uncompressed.checksums, config.verify)?;
-
-        // Ensure they're linked at current path
-        config.pool.lock()?.link_file(&reference.checksums, &path)?;
-        config
-            .pool
-            .lock()?
-            .link_file(&uncompressed.checksums, &uncompressed_path)?;
-        return Ok(FetchResult { data, fetched: 0 });
+
+    if let Some(uncompressed) = uncompressed {
+        let uncompressed_path = get_dist_path(&config.repository, prefix, &uncompressed.path);
+
+        if config.pool.contains(&reference.checksums)
+            && config.pool.contains(&uncompressed.checksums)
+        {
+            let data = config
+                .pool
+                .get_contents(&uncompressed.checksums, config.verify)?;
+
+            // Ensure they're linked at current path
+            config.pool.lock()?.link_file(&reference.checksums, &path)?;
+            config
+                .pool
+                .lock()?
+                .link_file(&uncompressed.checksums, &uncompressed_path)?;
+            return Ok(FetchResult { data, fetched: 0 });
+        }
     }
 
     let urls = if by_hash {
@@ -307,12 +312,15 @@ fn fetch_index_file(
     };
 
     let locked = &config.pool.lock()?;
-    if !locked.contains(&uncompressed.checksums) {
-        locked.add_file(decompressed, &uncompressed.checksums, config.sync)?;
-    }
+    if let Some(uncompressed) = uncompressed {
+        if !locked.contains(&uncompressed.checksums) {
+            locked.add_file(decompressed, &uncompressed.checksums, config.sync)?;
+        }
 
-    // Ensure it's linked at current path
-    locked.link_file(&uncompressed.checksums, &uncompressed_path)?;
+        // Ensure it's linked at current path
+        let uncompressed_path = get_dist_path(&config.repository, prefix, &uncompressed.path);
+        locked.link_file(&uncompressed.checksums, &uncompressed_path)?;
+    }
 
     Ok(FetchResult {
         data: decompressed.to_owned(),
@@ -566,15 +574,13 @@ pub fn create_snapshot(
         for basename in references {
             println!("\tFetching '{basename}'..");
             let files = release.files.get(basename).unwrap();
-            let uncompressed_ref = files
-                .iter()
-                .find(|reference| reference.path == *basename)
-                .ok_or_else(|| format_err!("Found derived reference without base reference."))?;
+            let uncompressed_ref = files.iter().find(|reference| reference.path == *basename);
+
             let mut package_index_data = None;
 
             for reference in files {
                 // if both compressed and uncompressed are referenced, the uncompressed file may not exist on the server
-                if reference == uncompressed_ref && files.len() > 1 {
+                if Some(reference) == uncompressed_ref && files.len() > 1 {
                     continue;
                 }
 
-- 
2.30.2





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [pve-devel] applied-series: [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements
  2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
                   ` (7 preceding siblings ...)
  2022-09-15 14:36 ` [pve-devel] [PATCH follow-up proxmox-offline-mirror 8/7] mirror: handle indices which are only available compressed Fabian Grünbichler
@ 2022-09-16 12:31 ` Thomas Lamprecht
  8 siblings, 0 replies; 11+ messages in thread
From: Thomas Lamprecht @ 2022-09-16 12:31 UTC (permalink / raw)
  To: Proxmox VE development discussion, Fabian Grünbichler

Am 15/09/2022 um 15:09 schrieb Fabian Grünbichler:
> this series adds some features and fixes some issues that pop up when
> attempting to mirror Ubuntu repositories, and should also improve
> resilience with other third-party repositories.
> 
> tested with Ubuntu Jammy (main, security and updates repositories).
> 

applied series, with the fixup squashed into the respective commit and a dependency
bump for proxmox-apt, thanks!




^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2022-09-16 12:31 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 1/4] mirror: use xz multi decoder Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 1/3] release: add Commands file reference type Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 2/4] mirror: skip failed, non Packages references Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 2/3] release: add 'architecture' helper Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 3/4] mirror: support acquiring indices by hash Fabian Grünbichler
2022-09-15 14:34   ` [pve-devel] [PATCH FIXUP proxmox-offline-mirror] clippy fix Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 3/3] release: fix typo in 'Acquire-By-Hash' Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 4/4] mirror: use new architecture helper Fabian Grünbichler
2022-09-15 14:36 ` [pve-devel] [PATCH follow-up proxmox-offline-mirror 8/7] mirror: handle indices which are only available compressed Fabian Grünbichler
2022-09-16 12:31 ` [pve-devel] applied-series: [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Thomas Lamprecht

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal