[PATCH v6 proxmox-backup 00/18] fix: #3847 pipe from STDIN to proxmox-backup-client

[Robert Obkircher <r.obkircher@proxmox.com>]

Add support for commands like:
    ssh host cmd | proxmox-backup-client backup data.img:/dev/stdin
    proxmox-backup-client backup a.img:<(mysqldump) b.img:<(pgdump)

Changes since v5:

Client:
- add IndexType::to_prefix_and_size helper and fix format string nit
- drop support for '-' but log helpful warnings
- fail early for duplicate pipes
- avoid calling poll_next on terminated streams

Server:
- mention why Arc<ChunkStore> can be removed from didx as well
- (Test)TempDir:
    - add to parent mod in earlier commit
    - drop "Test" from the name
    - improved API so it could be moved to a crate (no unwrap in new,
      no Deref, no impl From<&TempDir> for PathBuf)
    - added tests
- reorder add_chunk commit before grow_to_size
- support reuse-csum with arbitrary sizes
- made up my mind on u32 vs u64 and reordered commits
- check that chunk_size is power of two


Robert Obkircher (18):
  datastore: remove Arc<ChunkStore> from FixedIndexWriter
  datastore: remove Arc<ChunkStore> from DynamicIndexWriter
  datastore: add TempDir that is automatically deleted on drop
  datastore: use temporary directory for chunk store test
  datastore: combine public FixedIndexWriter methods into add_chunk
  datastore: use fixed size types for FixedIndexWriter
  datastore: verify that chunk_size is a power of two
  datastore: support writing fidx files of unknown size
  datastore: test FixedIndexWriter
  api: backup: make fixed index file size optional
  api: verify fixed index writer size on close
  client: don't poll terminated source in FixedChunkStream
  client: don't poll terminated source in ChunkStream
  fix #3847: client: support fifo pipe inputs for image backups
  client: Fail early if the same pipe is specified for multiple inputs
  datastore: compute fidx file size with overflow checks
  datastore: support writing fidx files on systems with larger page size
  datastore: support incremental fidx uploads with different size

 pbs-client/src/backup_writer.rs    |  37 ++-
 pbs-client/src/chunk_stream.rs     |  23 +-
 pbs-datastore/src/chunk_store.rs   |  14 +-
 pbs-datastore/src/datastore.rs     |  19 +-
 pbs-datastore/src/dynamic_index.rs |   5 +-
 pbs-datastore/src/fixed_index.rs   | 513 +++++++++++++++++++++++++----
 pbs-datastore/src/lib.rs           |   3 +
 pbs-datastore/src/temp_dir.rs      | 142 ++++++++
 proxmox-backup-client/src/main.rs  |  55 +++-
 src/api2/backup/environment.rs     |  37 ++-
 src/api2/backup/mod.rs             |   6 +-
 src/server/push.rs                 |  11 +-
 12 files changed, 725 insertions(+), 140 deletions(-)
 create mode 100644 pbs-datastore/src/temp_dir.rs

-- 
2.47.3