Bug #17984
Updated by Nathan Cutler over 7 years ago
description: powercycle/osd/{clusters/3osd-1per-target.yaml fs/ext4.yaml powercycle/default.yaml tasks/cfuse_workunit_kernel_untar_build.yaml}
Test failed ("dead") twice in a row on smithi:
* http://pulpito.ceph.com/smithfarm-2016-11-18_15:27:08-powercycle-hammer-backports-testing-basic-smithi/560554/
* http://pulpito.ceph.com/smithfarm-2016-11-20_21:53:44-powercycle-hammer-backports-testing-basic-smithi/565543/
<pre>
2016-11-21T15:44:10.000 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients...
2016-11-21T15:44:10.000 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {}
2016-11-21T15:44:10.000 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi102.front.sepia.ceph.com /home/ubuntu/cephtest
/mnt.0...
2016-11-21T15:44:10.000 INFO:teuthology.orchestra.run.smithi102:Running: 'mkdir -- /home/ubuntu/cephtest/mnt.0'
2016-11-21T15:44:10.001 INFO:tasks.thrashosds.thrasher:starting do_thrash
2016-11-21T15:44:10.001 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [1, 0, 2]
2016-11-21T15:44:10.002 INFO:tasks.thrashosds.thrasher:choose_action: min_in 3 min_out 0 min_live 2 min_dead 0
2016-11-21T15:44:10.002 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.253337561323
2016-11-21T15:44:10.002 INFO:teuthology.orchestra.run.smithi102:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage c
eph osd reweight 2 0.253337561323'
2016-11-21T15:44:10.015 INFO:teuthology.orchestra.run.smithi102:Running: 'sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connectio
ns'
2016-11-21T15:44:10.103 INFO:teuthology.orchestra.run.smithi102.stderr:mount: /sys/fs/fuse/connections already mounted or /sys/fs/fuse/connecti
ons busy
2016-11-21T15:44:10.103 INFO:teuthology.orchestra.run.smithi102.stderr:mount: according to mtab, none is already mounted on /sys/fs/fuse/connec
tions
2016-11-21T15:44:10.105 INFO:teuthology.orchestra.run.smithi102:Running: 'ls /sys/fs/fuse/connections'
2016-11-21T15:44:10.179 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: []
2016-11-21T15:44:10.179 INFO:teuthology.orchestra.run.smithi102:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/cover
age daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0'
2016-11-21T15:44:10.246 INFO:teuthology.orchestra.run.smithi102:Running: 'sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections'
2016-11-21T15:44:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:mount: /sys/fs/fuse/connections already mounted or /sys/fs/fuse/connections busy
2016-11-21T15:44:10.269 INFO:teuthology.orchestra.run.smithi102.stderr:mount: according to mtab, none is already mounted on /sys/fs/fuse/connections
2016-11-21T15:44:10.271 INFO:teuthology.orchestra.run.smithi102:Running: 'ls /sys/fs/fuse/connections'
2016-11-21T15:44:10.319 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stdout:ceph-fuse[15058]: starting ceph client
2016-11-21T15:44:10.320 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:2016-11-21 15:44:10.316967 7f87f3b307c0 -1 init, newargv = 0x399c700 newargc=9
2016-11-21T15:44:10.336 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:ceph-fuse[15058]: starting fuse
2016-11-21T15:44:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:31
2016-11-21T15:44:10.346 INFO:teuthology.orchestra.run.smithi102:Running: 'sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections'
2016-11-21T15:44:10.433 INFO:teuthology.orchestra.run.smithi102.stderr:mount: /sys/fs/fuse/connections already mounted or /sys/fs/fuse/connections busy
2016-11-21T15:44:10.433 INFO:teuthology.orchestra.run.smithi102.stderr:mount: according to mtab, none is already mounted on /sys/fs/fuse/connections
2016-11-21T15:44:10.435 INFO:teuthology.orchestra.run.smithi102:Running: 'ls /sys/fs/fuse/connections'
2016-11-21T15:44:10.510 INFO:teuthology.orchestra.run.smithi102.stdout:31
2016-11-21T15:44:10.510 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [31]
2016-11-21T15:44:10.510 INFO:teuthology.orchestra.run.smithi102:Running: "stat --file-system '--printf=%T\n' -- /home/ubuntu/cephtest/mnt.0"
2016-11-21T15:44:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:fuseblk
2016-11-21T15:44:10.587 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0
2016-11-21T15:44:10.587 INFO:teuthology.orchestra.run.smithi102:Running: 'sudo chmod 1777 /home/ubuntu/cephtest/mnt.0'
2016-11-21T16:00:29.846 ERROR:paramiko.transport:Socket exception: No route to host (113)
</pre>