Ceph » RADOS

2017-08-24T23:58:41.191 INFO:teuthology.orchestra.run.smithi191:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty'

but mon log only shows

remote/smithi191/log/ceph-mon.c.log:2017-08-24 23:56:51.860202 7f6088666700  1 -- 172.21.15.191:6790/0 <== client.4823 172.21.15.191:0/3085641752 8 ==== mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 ==== 87+0+0 (3881750295 0 0) 0x56485357b600 con 0x5648536252a0
remote/smithi191/log/ceph-mon.c.log:2017-08-24 23:56:51.860280 7f6088666700  0 mon.c@2(peon) e1 handle_command mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1
remote/smithi191/log/ceph-mon.c.log:2017-08-24 23:56:51.860326 7f6088666700  0 log_channel(audit) log [DBG] : from='client.? 172.21.15.191:0/3085641752' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch
remote/smithi191/log/ceph-mon.c.log:2017-08-24 23:56:51.860571 7f6088666700  2 mon.c@2(peon) e1 send_reply 0x564851307680 0x5648519b8300 mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0  v0) v1
remote/smithi191/log/ceph-mon.c.log:2017-08-24 23:56:51.860578 7f6088666700  1 -- 172.21.15.191:6790/0 --> 172.21.15.191:0/3085641752 -- mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0  v0) v1 -- ?+3565 0x5648519b8300 con 0x5648536252a0

as the last ceph status command (2 minutes earlier!).

/a/sage-2017-08-24_16:14:07-rados-wip-sage-testing-20170824a-distro-basic-smithi/1560155

this caused a hang in the thrasher, which led to a recovery timeout:

failure_reason: failed to recover before timeout expired