From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH proxmox-offline-mirror 2/4] mirror: skip failed, non Packages references
Date: Thu, 15 Sep 2022 15:09:14 +0200 [thread overview]
Message-ID: <20220915130918.727902-4-f.gruenbichler@proxmox.com> (raw)
In-Reply-To: <20220915130918.727902-1-f.gruenbichler@proxmox.com>
these contain extra data that is not that important for the main
repository use case - providing deb packages.
if they are not retrievable (e.g., Ubuntu *only* provides some of they
via by-hash, which proxmox-offline-mirror doesn't yet support) a warning
should be enough, instead of failing the whole snapshot creation.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
src/mirror.rs | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/src/mirror.rs b/src/mirror.rs
index 14b0c6a..f910e6a 100644
--- a/src/mirror.rs
+++ b/src/mirror.rs
@@ -534,6 +534,7 @@ pub fn create_snapshot(
let mut packages_size = 0_usize;
let mut packages_indices = HashMap::new();
+ let mut failed_references = Vec::new();
for (component, references) in per_component {
println!("\nFetching indices for component '{component}'");
let mut component_deb_size = 0;
@@ -555,7 +556,18 @@ pub fn create_snapshot(
}
// this will ensure the uncompressed file will be written locally
- let res = fetch_index_file(&config, prefix, reference, uncompressed_ref)?;
+ let res = match fetch_index_file(&config, prefix, reference, uncompressed_ref) {
+ Ok(res) => res,
+ Err(err) if !reference.file_type.is_package_index() => {
+ eprintln!(
+ "Failed to fetch '{:?}' type reference '{}', skipping - {err}",
+ reference.file_type, reference.path
+ );
+ failed_references.push(reference);
+ continue;
+ }
+ Err(err) => bail!(err),
+ };
fetch_progress.update(&res);
if package_index_data.is_none() && reference.file_type.is_package_index() {
@@ -577,6 +589,12 @@ pub fn create_snapshot(
total_progress += fetch_progress;
}
println!("Total deb size: {packages_size}");
+ if !failed_references.is_empty() {
+ eprintln!("Failed to download non-package-index references:");
+ for reference in failed_references {
+ eprintln!("\t{}", reference.path);
+ }
+ }
println!("\nFetching packages..");
for (basename, references) in packages_indices {
--
2.30.2
next prev parent reply other threads:[~2022-09-15 13:09 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-15 13:09 [pve-devel] [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 1/4] mirror: use xz multi decoder Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 1/3] release: add Commands file reference type Fabian Grünbichler
2022-09-15 13:09 ` Fabian Grünbichler [this message]
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 2/3] release: add 'architecture' helper Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 3/4] mirror: support acquiring indices by hash Fabian Grünbichler
2022-09-15 14:34 ` [pve-devel] [PATCH FIXUP proxmox-offline-mirror] clippy fix Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-apt 3/3] release: fix typo in 'Acquire-By-Hash' Fabian Grünbichler
2022-09-15 13:09 ` [pve-devel] [PATCH proxmox-offline-mirror 4/4] mirror: use new architecture helper Fabian Grünbichler
2022-09-15 14:36 ` [pve-devel] [PATCH follow-up proxmox-offline-mirror 8/7] mirror: handle indices which are only available compressed Fabian Grünbichler
2022-09-16 12:31 ` [pve-devel] applied-series: [PATCH proxmox-apt/proxmox-offline-mirror 0/7] misc improvements Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220915130918.727902-4-f.gruenbichler@proxmox.com \
--to=f.gruenbichler@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox