[PATCH v5 proxmox-backup 00/16] fix: #3847 pipe from STDIN to proxmox-backup-client

[Robert Obkircher <r.obkircher@proxmox.com>]

Add support for commands like:
    ssh host cmd | proxmox-backup-client backup data.img:/dev/stdin
    proxmox-backup-client backup a.img:<(mysqldump) b.img:<(pgdump)

Changes since v4:
- reordered the ChunkStore removal commit and adjusted dynamic writer
- moved TempTestDir into separate file, use it for test_chunk_store1
- improved commit messages and comments
- changed u32 param to u64
- removed unnecessary casts reported by clippy
- added two commits at the end which restrict chunk_size to u32
  (I'd fixup commit 12/16 if we actually want to keep those changes)

An open question is if we should maybe fsync the data before renaming.
Or is that too much overhead for slightly better crash consistency?


Robert Obkircher (16):
  datastore: remove Arc<ChunkStore> from FixedIndexWriter
  datastore: remove Arc<ChunkStore> from DynamicIndexWriter
  datastore: add TempTestDir that is automatically deleted on drop
  datastore: use temporary directory for chunk store test
  datastore: support writing fidx files of unknown size
  datastore: test FixedIndexWriter
  api: backup: make fixed index file size optional
  api: verify fixed index writer size on close
  fix #3847: client: support fifo pipe inputs for images
  client: treat minus sign as stdin
  datastore: combine public FixedIndexWriter methods into add_chunk.
  datastore: use u64 instead of usize for fidx writer content size
  datastore: compute fidx file size with overflow checks
  datastore: support writing fidx files on systems with larger page size
  datastore: FixedIndexWriter: switch public chunk_size to u32
  datastore: FixedIndexWriter: switch internal chunk_size to u32

 pbs-client/src/backup_writer.rs    |  38 ++-
 pbs-datastore/src/chunk_store.rs   |   7 +-
 pbs-datastore/src/datastore.rs     |  19 +-
 pbs-datastore/src/dynamic_index.rs |   5 +-
 pbs-datastore/src/fixed_index.rs   | 435 +++++++++++++++++++++++++----
 pbs-datastore/src/lib.rs           |   3 +
 pbs-datastore/src/temp_test_dir.rs |  33 +++
 proxmox-backup-client/src/main.rs  |  37 ++-
 src/api2/backup/environment.rs     |  33 ++-
 src/api2/backup/mod.rs             |   6 +-
 src/server/push.rs                 |  11 +-
 11 files changed, 499 insertions(+), 128 deletions(-)
 create mode 100644 pbs-datastore/src/temp_test_dir.rs

-- 
2.47.3