Bug #51069
Updated by Patrick Donnelly almost 3 years ago
<pre>
2021-06-02T03:57:06.273 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo bash -c '
2021-06-02T03:57:06.273 DEBUG:teuthology.orchestra.run.smithi005:> set -ex
2021-06-02T03:57:06.274 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p tree
2021-06-02T03:57:06.274 DEBUG:teuthology.orchestra.run.smithi005:>
2021-06-02T03:57:06.274 DEBUG:teuthology.orchestra.run.smithi005:> setfattr -n ceph.dir.pin.distributed -v 1 tree
2021-06-02T03:57:06.275 DEBUG:teuthology.orchestra.run.smithi005:>
2021-06-02T03:57:06.275 DEBUG:teuthology.orchestra.run.smithi005:> for ((i = 0; i < 1000; i++)); do
2021-06-02T03:57:06.275 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p "tree/$i"
2021-06-02T03:57:06.276 DEBUG:teuthology.orchestra.run.smithi005:> echo file > "tree/$i/file"
2021-06-02T03:57:06.276 DEBUG:teuthology.orchestra.run.smithi005:> done
2021-06-02T03:57:06.276 DEBUG:teuthology.orchestra.run.smithi005:> ')
...
2021-06-02T03:57:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
2021-06-02T03:57:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i < 1000 ))
2021-06-02T03:57:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:+ mkdir -p tree/347
2021-06-02T03:57:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
2021-06-02T03:57:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i < 1000 ))
2021-06-02T03:57:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:+ mkdir -p tree/348
2021-06-02T03:57:17.592 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:20.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2021-06-02T03:57:20.846 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2021-06-02T03:57:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
2021-06-02T03:57:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i < 1000 ))
2021-06-02T03:57:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:+ mkdir -p tree/349
2021-06-02T03:57:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
2021-06-02T03:57:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i < 1000 ))
2021-06-02T03:57:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:+ mkdir -p tree/350
2021-06-02T03:57:22.593 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:22.606 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
2021-06-02T03:57:22.606 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i < 1000 ))
2021-06-02T03:57:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:+ mkdir -p tree/351
2021-06-02T03:57:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:22.619 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
2021-06-02T03:57:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i < 1000 ))
2021-06-02T03:57:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:+ mkdir -p tree/352
2021-06-02T03:57:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
2021-06-02T03:57:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i < 1000 ))
2021-06-02T03:57:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:+ mkdir -p tree/353
2021-06-02T03:57:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
2021-06-02T03:57:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i < 1000 ))
2021-06-02T03:57:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:+ mkdir -p tree/354
2021-06-02T03:57:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:+ echo file
2021-06-02T03:57:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:+ (( i++ ))
...
</pre>
From: /ceph/teuthology-archive/pdonnell-2021-06-02_03:31:59-fs-master-distro-basic-smithi/6145427/teuthology.log
The 5 second turnaround usually is a symptom of something blocking on journal flush.