Ceph

We are doing benchmarks with ceph to optimize settings (including striping)
and monitor various items, including the per image throughput using
'rbd perf image iostat not working'
On some tests this monitoring fails, i.e. despite workload being handled (generated by fio and
reported to have succeeded) and iostat showing corresponding activity on the disks,
no output is produced.
This is the case in one out of 30-50 tests, in all others, 'rbd perf image iostat' produces reasonable results.

The only output is 'rbd: waiting for initial image stats'.
The iostat module in mgr is enabled in all cases.
I could create a setup that reproduces the problem on our system.
When doing so,
ceph iostat
also reports only 0s for read and write, despite fio creating load.
Since the mgr collect the data from osds I restarted some OSD containers without change.
I created histogram dumps from two osds in both a reproduction state and a reference state where the rbd perf image iostat works.
E.g.
sudo ceph daemon /var/run/ceph/33faa8ca-980e-11ed-8944-d9a483780148/ceph-osd.9.asok perf histogram dump > osd9_histogram_dump_while_fio_read_noiostat.json
(I had to use the path to the admin socket).

There was a clear difference between the two cases.
In the case where iostat did not work nearly all values in the histogram file were 0, with few exceptions of some very low entries.