Actions
Bug #3966
closedosdthrasher: does tell on osd just after restarting it
Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
figured out where the thrasher errors are coming from:
2013-01-30T10:48:02.890 INFO:teuthology.task.thrashosds.thrasher:in_osds: [1, 2, 0, 4] out_osds: [3, 5] dead_osds: [3] live_osds: [1, 0, 2, 5, 4] 2013-01-30T10:48:02.890 INFO:teuthology.task.thrashosds.thrasher:Reviving osd 3 2013-01-30T10:48:02.890 INFO:teuthology.task.ceph.osd.3:Restarting 2013-01-30T10:48:02.891 DEBUG:teuthology.orchestra.run:Running: '/tmp/cephtest/enable-coredump /tmp/cephtest/binary/usr/local/bin/ceph-coverage /tmp/cephtest/archive/coverage /tmp/cephtest/daemon-helper term /tmp/cephtest/binary/usr/local/bin/ceph-osd -f -i 3 -c /tmp/cephtest/ceph.conf' 2013-01-30T10:48:02.894 INFO:teuthology.task.ceph.osd.3:Started 2013-01-30T10:48:02.894 INFO:teuthology.task.thrashosds.ceph_manager:waiting for recovery to complete 2013-01-30T10:48:02.894 DEBUG:teuthology.orchestra.run:Running: 'LD_LIBRARY_PRELOAD=/tmp/cephtest/binary/usr/local/lib /tmp/cephtest/enable-coredump /tmp/cephtest/binary/usr/local/bin/ceph-coverage /tmp/cephtest/archive/coverage /tmp/cephtest/binary/usr/local/bin /ceph -k /tmp/cephtest/ceph.keyring -c /tmp/cephtest/ceph.conf --concise -- pg dump --format=json' 2013-01-30T10:48:02.947 DEBUG:teuthology.orchestra.run:Running: 'LD_LIBRARY_PRELOAD=/tmp/cephtest/binary/usr/local/lib /tmp/cephtest/enable-coredump /tmp/cephtest/binary/usr/local/bin/ceph-coverage /tmp/cephtest/archive/coverage /tmp/cephtest/binary/usr/local/bin /ceph -k /tmp/cephtest/ceph.keyring -c /tmp/cephtest/ceph.conf --concise -- pg dump --format=json' 2013-01-30T10:48:02.948 INFO:teuthology.task.ceph.osd.3.out:starting osd.3 at :/0 osd_data /tmp/cephtest/data/osd.3.data /tmp/cephtest/data/osd.3.journal 2013-01-30T10:48:03.027 INFO:teuthology.task.ceph.osd.3.err:2013-01-30 10:47:47.704046 7f8a55f09780 -1 filestore(/tmp/cephtest/data/osd.3.data) limited size xattrs -- filestore_xattr_use_omap enabled 2013-01-30T10:48:03.037 DEBUG:teuthology.orchestra.run:Running: 'LD_LIBRARY_PRELOAD=/tmp/cephtest/binary/usr/local/lib /tmp/cephtest/enable-coredump /tmp/cephtest/binary/usr/local/bin/ceph-coverage /tmp/cephtest/archive/coverage /tmp/cephtest/binary/usr/local/bin /ceph -k /tmp/cephtest/ceph.keyring -c /tmp/cephtest/ceph.conf --concise -s' 2013-01-30T10:48:03.108 INFO:teuthology.task.thrashosds.ceph_manager: health HEALTH_OK monmap e1: 3 mons at {a=10.214.131.15:6789/0,b=10.214.131.15:6790/0,c=10.214.131.24:6789/0}, election epoch 6, quorum 0,1,2 a,b,c osdmap e104: 6 osds: 5 up, 4 in pgmap v142: 72 pgs: 72 active+clean; 108 MB data, 1519 MB used, 2609 GB / 2750 GB avail mdsmap e5: 1/1/1 up {0=a=up:active} 2013-01-30T10:48:03.108 INFO:teuthology.task.thrashosds.ceph_manager:recovered! 2013-01-30T10:48:03.108 INFO:teuthology.task.thrashosds.thrasher:choose_action: min_in 2 min_out 0 min_live 2 min_dead 0 2013-01-30T10:48:03.112 INFO:teuthology.task.thrashosds.thrasher:inject_pause on 3 2013-01-30T10:48:03.112 INFO:teuthology.task.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2013-01-30T10:48:03.112 INFO:teuthology.task.thrashosds.thrasher:Checking after 0, should_be_down=False
i'm guessing it's just that osd.3 hasn't started up yet, causing the ceph tell osd.3 to fail.
Updated by Samuel Just over 11 years ago
- Status changed from New to 7
pushed fix to master, fadc22c0b9e1755b1d1826fcfe8be71e28574bc9 (teuthology)
Updated by Sage Weil over 11 years ago
- Status changed from 7 to Resolved
fixed in tuethology fadc22c0b9e1755b1d1826fcfe8be71e28574bc9
Actions