all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Robert Obkircher <r.obkircher@proxmox.com>
To: Christian Ebner <c.ebner@proxmox.com>, pbs-devel@lists.proxmox.com
Subject: Re: [PATCH v5 proxmox-backup 10/16] client: treat minus sign as stdin
Date: Thu, 5 Feb 2026 14:40:18 +0100	[thread overview]
Message-ID: <c3384ee6-d113-428a-9d6d-70953a359864@proxmox.com> (raw)
In-Reply-To: <b64dcf8e-8d76-41bd-be5d-f6bb79b57229@proxmox.com>


On 2/4/26 15:42, Christian Ebner wrote:
> On 2/4/26 3:00 PM, Robert Obkircher wrote:
>>
>> On 2/2/26 13:14, Christian Ebner wrote:
>>> On 1/30/26 5:45 PM, Robert Obkircher wrote:
>>>> Treat "-" as an alias for "/dev/stdin". If there is an actual file
>>>> with that name it can still be read via "./-".
>>>>
>>>> Signed-off-by: Robert Obkircher <r.obkircher@proxmox.com>
>>>> ---
>>>>    proxmox-backup-client/src/main.rs | 7 +++++++
>>>>    1 file changed, 7 insertions(+)
>>>>
>>>> diff --git a/proxmox-backup-client/src/main.rs
>>>> b/proxmox-backup-client/src/main.rs
>>>> index 7fc711fd..37878b01 100644
>>>> --- a/proxmox-backup-client/src/main.rs
>>>> +++ b/proxmox-backup-client/src/main.rs
>>>> @@ -845,6 +845,13 @@ async fn create_backup(
>>>>            }
>>>>            target_set.insert(target.clone());
>>>>    +        // one can still use ./- to refer to an actual file with
>>>> that name
>>>> +        let filename = if filename == "-" {
>>>> +            String::from("/dev/stdin")
>>>
>>> should we add a warning here?
>>>
>>> This is actually a breaking change, but guarding it behind a flag is
>>> maybe a bit overkill. We could check if the file exists before
>>> continuing and if so fail with an appropriate warning/error?
>>
>> Another option would be to just require /dev/stdin and print a hint
>> about that if the user tries to specify "-". What do you think
>> about that?
>
> Might be the better option for the time being, yes. And must be
> documented correctly of course. The '-' shortcut could still be
> added as breaking change in a future version.
>
> But now I'm wondering, how is the multi archive case handled if
> reading from stdin?
>
> To extend your example in the cover letter: `ssh host cmd |
> proxmox-backup-client backup a.img:/dev/stdin b.img:/dev/stdin`
>
> Without having tested it I guess this will fail on the second
> archive creation since that will be empty? 
Yes, this specific example will fail for that reason.

But in general, running something like `cat - test.txt -` or `cat
/dev/stdin test.txt /dev/stdin` actually works if you run it
interactively in a terminal and hit `ctrl+d` to end the first input.
>
> This should be checked and fail on command invocation instead. 
This is not trivial because of symlinks:

$ file /dev/stdin
/dev/stdin: symbolic link to /proc/self/fd/0

$ file /proc/self/fd/0
/proc/self/fd/0: symbolic link to /dev/pts/5

$ echo | file /proc/self/fd/0
/proc/self/fd/0: symbolic link to pipe:[317534]


I can think of two solutions:

1) Since we already read the metadata of all inputs at the start, we
could check that the inode number is unique for fifo files.

2) Simply check if /dev/stdin and/or /proc/self/fd/0 appear multiple
times as strings. In this case a warning might be better than an error.


I also just realized that my patch didn't actually work interactively,
because pts devices are not fifo pipes. Somehow I've only ever tested
this with my script or non-interactive pipes.

I was able to fix this by allowing character devices, but initially I
got some really weird behavior. For example, a single ctrl+D was not
enough to end the stream and I also got errors about multiple smaller
chunks being sent. The problem was that FixedChunkStream continued to
call (try_)poll_next even after it got a None, which is not allowed.

>
> > The only real advantage of "-" is that it could be implemented in
> a> platform independent way by reading from io::stdin(). That would
>> require a lot of changes though.
>>
>>>
>>>> +        } else {
>>>> +            filename
>>>> +        };
>>>> +
>>>>            use std::os::unix::fs::FileTypeExt;
>>>>              let metadata = std::fs::metadata(&filename)
>>>
>




  reply	other threads:[~2026-02-05 13:39 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-30 16:45 [PATCH v5 proxmox-backup 00/16] fix: #3847 pipe from STDIN to proxmox-backup-client Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 01/16] datastore: remove Arc<ChunkStore> from FixedIndexWriter Robert Obkircher
2026-02-02 10:02   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 02/16] datastore: remove Arc<ChunkStore> from DynamicIndexWriter Robert Obkircher
2026-02-02 10:03   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 03/16] datastore: add TempTestDir that is automatically deleted on drop Robert Obkircher
2026-02-02  8:32   ` Lukas Wagner
2026-02-02 10:12     ` Robert Obkircher
2026-02-02 10:56       ` Lukas Wagner
2026-02-02 10:03   ` Christian Ebner
2026-02-02 10:17   ` Christian Ebner
2026-02-02 10:50     ` Robert Obkircher
2026-02-02 11:13       ` Christian Ebner
2026-02-02 11:21         ` Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 04/16] datastore: use temporary directory for chunk store test Robert Obkircher
2026-02-02 10:16   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 05/16] datastore: support writing fidx files of unknown size Robert Obkircher
2026-02-02 10:43   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 06/16] datastore: test FixedIndexWriter Robert Obkircher
2026-02-02 11:11   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 07/16] api: backup: make fixed index file size optional Robert Obkircher
2026-02-02 11:39   ` Christian Ebner
2026-02-02 13:20     ` Robert Obkircher
2026-02-02 13:57       ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 08/16] api: verify fixed index writer size on close Robert Obkircher
2026-02-02 11:48   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 09/16] fix #3847: client: support fifo pipe inputs for images Robert Obkircher
2026-02-02 12:09   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 10/16] client: treat minus sign as stdin Robert Obkircher
2026-02-02 12:15   ` Christian Ebner
2026-02-04 14:02     ` Robert Obkircher
2026-02-04 14:43       ` Christian Ebner
2026-02-05 13:40         ` Robert Obkircher [this message]
2026-02-05 14:45           ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 11/16] datastore: combine public FixedIndexWriter methods into add_chunk Robert Obkircher
2026-02-02 12:32   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 12/16] datastore: use u64 instead of usize for fidx writer content size Robert Obkircher
2026-02-02 12:48   ` Christian Ebner
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 13/16] datastore: compute fidx file size with overflow checks Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 14/16] datastore: support writing fidx files on systems with larger page size Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 15/16] datastore: FixedIndexWriter: switch public chunk_size to u32 Robert Obkircher
2026-01-30 16:45 ` [PATCH v5 proxmox-backup 16/16] datastore: FixedIndexWriter: switch internal " Robert Obkircher

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c3384ee6-d113-428a-9d6d-70953a359864@proxmox.com \
    --to=r.obkircher@proxmox.com \
    --cc=c.ebner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal