[pbs-devel] [PATCH v2 0/5] Fix PBS blockdriver for non-VM settings

[Stefan Reiter <s.reiter@proxmox.com>]

When using the PBS blockdriver with qemu-nbd (for example), it can happen that
enough read requests are issued to saturate the tokio thread pool. Not an issue
in general, but as me and Wolfgang painstakenly discovered a while back, it does
break block_on, which is used in BufferedFixedReader. This means that reading
larger amounts of data would hang the QEMU process [0].

Fix this by replacing the BufferedFixedReader with an AsyncIndexReader,
implementing AsyncSeek for it in the process. This makes the entire API async,
requiring no block_on anymore.

Incidentally, this also gave me my best benchmark results yet, coming in at
above 1.6 Gb/s read speed via NBD on my local machine.

Additionally I discovered a seperate bug (fixed by patch 5), wherein read
requests that we're not aligned to the chunk size would return bogus data. This
too only seems to happen in non-VM connections (e.g. nbd, etc...).

v2:
* Remove BufferedFixedReader entirely, use AsyncIndexReader instead
* Implement AsyncSeek for AsyncIndexReader
* Fix the second bug in Rust instead of QEMU C


[0] ...and since the NBD kernel driver appears to be horribly broken, this often
also crashes most of the system, but that's a different story. If you ever get
in this situation, 'nbd-client -d /dev/nbdX' works (sometimes) to force
disconnect the device ('qemu-nbd -d' intelligently issues a read before
disconnecting, thus hanging before getting anything done...)


backup: Stefan Reiter (3):
  add and implement chunk_from_offset for IndexFile
  implement AsyncSeek for AsyncIndexReader
  remove BufferedFixedReader interface

 src/backup/async_index_reader.rs | 116 ++++++++++++++++++----
 src/backup/dynamic_index.rs      |  18 ++++
 src/backup/fixed_index.rs        | 165 +++----------------------------
 src/backup/index.rs              |   4 +
 4 files changed, 129 insertions(+), 174 deletions(-)

backup-qemu: Stefan Reiter (2):
  use AsyncIndexReader for read_image_at
  read_image_at: iterate until buffer is filled

 current-api.h  |  4 ++--
 src/lib.rs     |  4 ++--
 src/restore.rs | 35 +++++++++++++++++++++++------------
 3 files changed, 27 insertions(+), 16 deletions(-)

-- 
2.20.1