Actions
Bug #5641
closedoccasional crush_ops.sh failure
Status:
Resolved
Priority:
High
Assignee:
-
Category:
Monitor
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
teuthology-2013-07-15_20:00:18-rados-cuttlefish-testing-basic/68030
2013-07-15T20:40:21.190 INFO:teuthology.task.workunit.client.0.err:+ ceph osd crush add 7 1 host=host2 root=foo
2013-07-15T20:40:21.544 DEBUG:teuthology.misc:with jobid basedir: 68030
2013-07-15T20:40:21.544 DEBUG:teuthology.orchestra.run:Running [10.214.133.37]: '/home/ubuntu/cephtest/68030/enable-coredump ceph-coverage /home/ubuntu/cephtest/68030/archive/coverage ceph --concise -m 10.214.133.37:6792 mon_status'
2013-07-15T20:40:21.571 DEBUG:teuthology.misc:with jobid basedir: 68030
2013-07-15T20:40:21.571 DEBUG:teuthology.orchestra.run:Running [10.214.133.37]: '/home/ubuntu/cephtest/68030/enable-coredump ceph-coverage /home/ubuntu/cephtest/68030/archive/coverage ceph --concise -m 10.214.133.37:6793 mon_status'
2013-07-15T20:40:21.663 INFO:teuthology.task.mon_thrash.mon_thrasher:monitors to thrash: ['c', 'a', 'b']
2013-07-15T20:40:21.663 INFO:teuthology.task.mon_thrash.mon_thrasher:monitors to freeze: ['g']
2013-07-15T20:40:21.663 INFO:teuthology.task.mon_thrash.mon_thrasher:thrashing mon.c
2013-07-15T20:40:21.664 INFO:teuthology.task.mon_thrash.mon_thrasher:killing mon.c
2013-07-15T20:40:21.664 DEBUG:teuthology.task.ceph.mon.c:waiting for process to exit
2013-07-15T20:40:21.704 INFO:teuthology.task.ceph.mon.c:Stopped
2013-07-15T20:40:21.704 INFO:teuthology.task.mon_thrash.mon_thrasher:thrashing mon.a
2013-07-15T20:40:21.704 INFO:teuthology.task.mon_thrash.mon_thrasher:killing mon.a
2013-07-15T20:40:21.704 DEBUG:teuthology.task.ceph.mon.a:waiting for process to exit
2013-07-15T20:40:21.743 INFO:teuthology.task.ceph.mon.a:Stopped
2013-07-15T20:40:21.743 INFO:teuthology.task.mon_thrash.mon_thrasher:thrashing mon.b
2013-07-15T20:40:21.743 INFO:teuthology.task.mon_thrash.mon_thrasher:killing mon.b
2013-07-15T20:40:21.743 DEBUG:teuthology.task.ceph.mon.b:waiting for process to exit
2013-07-15T20:40:21.779 INFO:teuthology.task.workunit.client.0.err:2013-07-15 20:40:47.984631 7fe0bdf41700 0 monclient: hunting for new mon
2013-07-15T20:40:21.779 INFO:teuthology.task.workunit.client.0.err:2013-07-15 20:40:47.984948 7fe0bc63d700 0 -- 10.214.133.32:0/26602 >> 10.214.133.37:6789/0 pipe(0x7fe0a0004680 sd=8 :0 s=1 pgs=0 cs=0 l=1).fault
2013-07-15T20:40:21.783 INFO:teuthology.task.ceph.mon.b:Stopped
2013-07-15T20:40:21.783 INFO:teuthology.task.mon_thrash:Sending STOP to mon g
2013-07-15T20:40:21.784 INFO:teuthology.task.ceph.mon.g:Sent signal 19
2013-07-15T20:40:21.784 INFO:teuthology.task.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons
2013-07-15T20:40:30.208 INFO:teuthology.task.workunit.client.0.err:2013-07-15 20:40:56.413366 7fe0bc73e700 0 -- 10.214.133.32:0/26602 >> 10.214.133.37:6791/0 pipe(0x7fe0b0002c50 sd=8 :0 s=1 pgs=0 cs=0 l=1).fault
2013-07-15T20:40:33.208 INFO:teuthology.task.workunit.client.0.err:2013-07-15 20:40:59.413872 7fe0bc63d700 0 -- 10.214.133.32:0/26602 >> 10.214.133.37:6789/0 pipe(0x7fe0b00044f0 sd=7 :0 s=1 pgs=0 cs=0 l=1).fault
2013-07-15T20:40:36.783 INFO:teuthology.task.mon_thrash:Sending CONT to mon g
2013-07-15T20:40:36.784 INFO:teuthology.task.ceph.mon.g:Sent signal 18
2013-07-15T20:40:36.784 INFO:teuthology.task.mon_thrash.ceph_manager:waiting for quorum size 6
2013-07-15T20:40:36.784 DEBUG:teuthology.misc:with jobid basedir: 68030
2013-07-15T20:40:36.784 DEBUG:teuthology.orchestra.run:Running [10.214.133.37]: '/home/ubuntu/cephtest/68030/enable-coredump ceph-coverage /home/ubuntu/cephtest/68030/archive/coverage ceph --concise quorum_status'
2013-07-15T20:40:36.815 INFO:teuthology.task.mon_thrash.ceph_manager:quorum_status is { "election_epoch": 18,
"quorum": [
0,
4,
6,
7,
8],
"monmap": { "epoch": 1,
"fsid": "0a3ef0eb-6237-4d7d-91d1-d3aae59a542a",
"modified": "2013-07-15 20:37:53.307446",
"created": "2013-07-15 20:37:53.307446",
"mons": [
{ "rank": 0,
"name": "f",
"addr": "10.214.133.32:6789\/0"},
{ "rank": 1,
"name": "a",
"addr": "10.214.133.37:6789\/0"},
{ "rank": 2,
"name": "g",
"addr": "10.214.133.32:6790\/0"},
{ "rank": 3,
"name": "b",
"addr": "10.214.133.37:6790\/0"},
{ "rank": 4,
"name": "h",
"addr": "10.214.133.32:6791\/0"},
{ "rank": 5,
"name": "c",
"addr": "10.214.133.37:6791\/0"},
{ "rank": 6,
"name": "i",
"addr": "10.214.133.32:6792\/0"},
{ "rank": 7,
"name": "d",
"addr": "10.214.133.37:6792\/0"},
{ "rank": 8,
"name": "e",
"addr": "10.214.133.37:6793\/0"}]}}
2013-07-15T20:40:39.815 DEBUG:teuthology.misc:with jobid basedir: 68030
2013-07-15T20:40:39.816 DEBUG:teuthology.orchestra.run:Running [10.214.133.37]: '/home/ubuntu/cephtest/68030/enable-coredump ceph-coverage /home/ubuntu/cephtest/68030/archive/coverage ceph --concise quorum_status'
2013-07-15T20:40:42.372 INFO:teuthology.task.workunit.client.0.err:(22) Invalid argument
Actions