From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id C70BE6BD8E for ; Thu, 28 Jan 2021 08:34:43 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id BC2DA2AACB for ; Thu, 28 Jan 2021 08:34:43 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id E38C72AAC1 for ; Thu, 28 Jan 2021 08:34:39 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id AF6014612C for ; Thu, 28 Jan 2021 08:34:39 +0100 (CET) Date: Thu, 28 Jan 2021 08:34:30 +0100 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox Backup Server development discussion References: <20210127134314.22530-1-d.csapak@proxmox.com> In-Reply-To: <20210127134314.22530-1-d.csapak@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid) Message-Id: <1611818626.6jkzjorqup.astroid@nora.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.025 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [pbs-devel] [RFC proxmox-backup] backup/backup_info: ignore vanished backup dirs during listing X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2021 07:34:43 -0000 when looking at the call sites of list_backup_files, this seems wrong -=20 e.g. when listing a BackupGroup's backups, such a no-longer-existing=20 snapshot is now returned but has no files. when creating a new=20 BackupInfo from a BackupDir, or listing a BackupInfo's files, it returns=20 Ok but contains no files. the decision whether to ignore such an error must be made at the call=20 sites of the call sites above, since in most cases I'd want to ignore=20 the snapshot and not just the file listing.. list_snapshot_files already throws an error if it can't read the=20 manifest, list_snapshots and list_groups seem to include it with no=20 files? On January 27, 2021 2:43 pm, Dominik Csapak wrote: > this had the effect that that a snapshot listing errored out when called > during a prune action >=20 > we now ignore such errors and simply return no files >=20 > also add some context for the error if one happens >=20 > Signed-off-by: Dominik Csapak > --- > maybe we also want to filter such snapshots out one level higher? > so that we do not show 'empty' snapshots >=20 > src/backup/backup_info.rs | 28 ++++++++++++++++++++++------ > 1 file changed, 22 insertions(+), 6 deletions(-) >=20 > diff --git a/src/backup/backup_info.rs b/src/backup/backup_info.rs > index 5ff1a6f8..dba3fea5 100644 > --- a/src/backup/backup_info.rs > +++ b/src/backup/backup_info.rs > @@ -1,4 +1,7 @@ > -use crate::tools; > +use crate::tools::{ > + self, > + fs::FileIterOps, > +}; > =20 > use anyhow::{bail, format_err, Error}; > use std::os::unix::io::RawFd; > @@ -339,11 +342,24 @@ impl BackupInfo { > fn list_backup_files(dirfd: RawFd, path: &P) -= > Result, Error> { > let mut files =3D vec![]; > =20 > - tools::scandir(dirfd, path, &BACKUP_FILE_REGEX, |_, filename, file_t= ype| { > - if file_type !=3D nix::dir::Type::File { return Ok(()); } > - files.push(filename.to_owned()); > - Ok(()) > - })?; > + match tools::fs::read_subdir(dirfd, path) { > + Ok(iter) =3D> { > + for entry in iter.filter_file_name_regex(&BACKUP_FILE_REGEX)= { > + let entry =3D entry?; > + let file_type =3D entry.file_type().ok_or_else(|| format= _err!("unable to detect file type"))?; > + if file_type =3D=3D nix::dir::Type::File { > + // ok because we filtered with BACKUP_FILE_REGEX > + let filename =3D unsafe { entry.file_name_utf8_unche= cked() }; > + files.push(filename.to_owned()); > + } > + } > + }, > + Err(nix::Error::Sys(errno)) if errno =3D=3D nix::errno::Errno::E= NOENT =3D> { > + // ignore vanished backup dirs > + eprintln!("backup dir vanished during listing"); > + }, > + Err(err) =3D> bail!("error listing backup files: {}", err), > + } > =20 > Ok(files) > } > --=20 > 2.20.1 >=20 >=20 >=20 > _______________________________________________ > pbs-devel mailing list > pbs-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel >=20 >=20 >=20 =