Actions
Bug #3460
closedceph monitor crashed in the nightlies
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Log: ubuntu@teuthology:/a/teuthology-2012-11-06_02:00:02-regression-testing-master-basic/10579
2012-11-07T04:46:56.568 INFO:teuthology.task.ceph.mon.b.err:common/Thread.cc: 110: FAILED assert(ret == 0) 2012-11-07T04:46:56.568 INFO:teuthology.task.ceph.mon.b.err: ceph version 0.53 (commit:2528b5ee105b16352c91af064af5c0b5a7d45d7c) 2012-11-07T04:46:56.569 INFO:teuthology.task.ceph.mon.b.err: 1: (Thread::create(unsigned long)+0x8a) [0x5e274a] 2012-11-07T04:46:56.569 INFO:teuthology.task.ceph.mon.b.err: 2: (SimpleMessenger::add_accept_pipe(int)+0x6a) [0x5ddb7a] 2012-11-07T04:46:56.569 INFO:teuthology.task.ceph.mon.b.err: 3: (Accepter::entry()+0x279) [0x63b789] 2012-11-07T04:46:56.569 INFO:teuthology.task.ceph.mon.b.err: 4: (()+0x7e9a) [0x7f03b5aa3e9a] 2012-11-07T04:46:56.569 INFO:teuthology.task.ceph.mon.b.err: 5: (clone()+0x6d) [0x7f03b425c4bd] 2012-11-07T04:46:56.569 INFO:teuthology.task.ceph.mon.b.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2012-11-07T04:46:56.569 INFO:teuthology.task.ceph.mon.b.err:2012-11-07 04:46:39.921454 7f03b274d700 -1 common/Thread.cc: In function 'void Thread::create(size_t)' thread 7f03b274d700 time 2012-11-07 04:46:39.920768 2012-11-07T04:46:56.570 INFO:teuthology.task.ceph.mon.b.err:common/Thread.cc: 110: FAILED assert(ret == 0) 2012-11-07T04:46:56.570 INFO:teuthology.task.ceph.mon.b.err: 2012-11-07T04:46:56.570 INFO:teuthology.task.ceph.mon.b.err: ceph version 0.53 (commit:2528b5ee105b16352c91af064af5c0b5a7d45d7c) 2012-11-07T04:46:56.570 INFO:teuthology.task.ceph.mon.b.err: 1: (Thread::create(unsigned long)+0x8a) [0x5e274a] 2012-11-07T04:46:56.570 INFO:teuthology.task.ceph.mon.b.err: 2: (SimpleMessenger::add_accept_pipe(int)+0x6a) [0x5ddb7a] 2012-11-07T04:46:56.570 INFO:teuthology.task.ceph.mon.b.err: 3: (Accepter::entry()+0x279) [0x63b789] 2012-11-07T04:46:56.570 INFO:teuthology.task.ceph.mon.b.err: 4: (()+0x7e9a) [0x7f03b5aa3e9a] 2012-11-07T04:46:56.571 INFO:teuthology.task.ceph.mon.b.err: 5: (clone()+0x6d) [0x7f03b425c4bd] 2012-11-07T04:46:56.571 INFO:teuthology.task.ceph.mon.b.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2012-11-07T04:46:56.571 INFO:teuthology.task.ceph.mon.b.err: 2012-11-07T04:46:56.700 INFO:teuthology.task.ceph.mon.b.err: 0> 2012-11-07 04:46:39.921454 7f03b274d700 -1 common/Thread.cc: In function 'void Thread::create(size_t)' thread 7f03b274d700 time 2012-11-07 04:46:39.920768 2012-11-07T04:46:56.700 INFO:teuthology.task.ceph.mon.b.err:common/Thread.cc: 110: FAILED assert(ret == 0) 2012-11-07T04:46:56.700 INFO:teuthology.task.ceph.mon.b.err: 2012-11-07T04:46:56.700 INFO:teuthology.task.ceph.mon.b.err: ceph version 0.53 (commit:2528b5ee105b16352c91af064af5c0b5a7d45d7c) 2012-11-07T04:46:56.700 INFO:teuthology.task.ceph.mon.b.err: 1: (Thread::create(unsigned long)+0x8a) [0x5e274a] 2012-11-07T04:46:56.701 INFO:teuthology.task.ceph.mon.b.err: 2: (SimpleMessenger::add_accept_pipe(int)+0x6a) [0x5ddb7a] 2012-11-07T04:46:56.701 INFO:teuthology.task.ceph.mon.b.err: 3: (Accepter::entry()+0x279) [0x63b789] 2012-11-07T04:46:56.701 INFO:teuthology.task.ceph.mon.b.err: 4: (()+0x7e9a) [0x7f03b5aa3e9a] 2012-11-07T04:46:56.701 INFO:teuthology.task.ceph.mon.b.err: 5: (clone()+0x6d) [0x7f03b425c4bd] 2012-11-07T04:46:56.701 INFO:teuthology.task.ceph.mon.b.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2012-11-07T04:46:56.701 INFO:teuthology.task.ceph.mon.b.err: 2012-11-07T04:46:56.702 INFO:teuthology.task.ceph.mon.b.err:terminate called after throwing an instance of 'ceph::FailedAssertion' 2012-11-07T04:46:56.702 INFO:teuthology.task.ceph.mon.b.err:*** Caught signal (Aborted) ** 2012-11-07T04:46:56.703 INFO:teuthology.task.ceph.mon.b.err: in thread 7f03b274d700 2012-11-07T04:46:56.703 INFO:teuthology.task.ceph.mon.b.err: ceph version 0.53 (commit:2528b5ee105b16352c91af064af5c0b5a7d45d7c) 2012-11-07T04:46:56.703 INFO:teuthology.task.ceph.mon.b.err: 1: /tmp/cephtest/binary/usr/local/bin/ceph-mon() [0x540a61] 2012-11-07T04:46:56.703 INFO:teuthology.task.ceph.mon.b.err: 2: (()+0xfcb0) [0x7f03b5aabcb0] 2012-11-07T04:46:56.703 INFO:teuthology.task.ceph.mon.b.err: 3: (gsignal()+0x35) [0x7f03b41a0445] 2012-11-07T04:46:56.703 INFO:teuthology.task.ceph.mon.b.err: 4: (abort()+0x17b) [0x7f03b41a3bab] 2012-11-07T04:46:56.703 INFO:teuthology.task.ceph.mon.b.err: 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f03b4aee69d] 2012-11-07T04:46:56.704 INFO:teuthology.task.ceph.mon.b.err: 6: (()+0xb5846) [0x7f03b4aec846] 2012-11-07T04:46:56.704 INFO:teuthology.task.ceph.mon.b.err: 7: (()+0xb5873) [0x7f03b4aec873] 2012-11-07T04:46:56.704 INFO:teuthology.task.ceph.mon.b.err: 8: (()+0xb596e) [0x7f03b4aec96e] 2012-11-07T04:46:56.704 INFO:teuthology.task.ceph.mon.b.err: 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ea) [0x5f0caa] 2012-11-07T04:46:56.704 INFO:teuthology.task.ceph.mon.b.err: 10: (Thread::create(unsigned long)+0x8a) [0x5e274a] 2012-11-07T04:46:56.704 INFO:teuthology.task.ceph.mon.b.err: 11: (SimpleMessenger::add_accept_pipe(int)+0x6a) [0x5ddb7a] 2012-11-07T04:46:56.705 INFO:teuthology.task.ceph.mon.b.err: 12: (Accepter::entry()+0x279) [0x63b789] 2012-11-07T04:46:56.705 INFO:teuthology.task.ceph.mon.b.err: 13: (()+0x7e9a) [0x7f03b5aa3e9a] 2012-11-07T04:46:56.705 INFO:teuthology.task.ceph.mon.b.err: 14: (clone()+0x6d) [0x7f03b425c4bd] 2012-11-07T04:46:56.705 INFO:teuthology.task.ceph.mon.b.err:2012-11-07 04:46:40.055458 7f03b274d700 -1 *** Caught signal (Aborted) ** 2012-11-07T04:46:56.706 INFO:teuthology.task.ceph.mon.b.err: in thread 7f03b274d700 2012-11-07T04:46:56.706 INFO:teuthology.task.ceph.mon.b.err: 2012-11-07T04:46:56.706 INFO:teuthology.task.ceph.mon.b.err: ceph version 0.53 (commit:2528b5ee105b16352c91af064af5c0b5a7d45d7c) 2012-11-07T04:46:56.706 INFO:teuthology.task.ceph.mon.b.err: 1: /tmp/cephtest/binary/usr/local/bin/ceph-mon() [0x540a61] 2012-11-07T04:46:56.706 INFO:teuthology.task.ceph.mon.b.err: 2: (()+0xfcb0) [0x7f03b5aabcb0] 2012-11-07T04:46:56.707 INFO:teuthology.task.ceph.mon.b.err: 3: (gsignal()+0x35) [0x7f03b41a0445] 2012-11-07T04:46:56.707 INFO:teuthology.task.ceph.mon.b.err: 4: (abort()+0x17b) [0x7f03b41a3bab] 2012-11-07T04:46:56.707 INFO:teuthology.task.ceph.mon.b.err: 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f03b4aee69d] 2012-11-07T04:46:56.707 INFO:teuthology.task.ceph.mon.b.err: 6: (()+0xb5846) [0x7f03b4aec846] 2012-11-07T04:46:56.707 INFO:teuthology.task.ceph.mon.b.err: 7: (()+0xb5873) [0x7f03b4aec873] 2012-11-07T04:46:56.707 INFO:teuthology.task.ceph.mon.b.err: 8: (()+0xb596e) [0x7f03b4aec96e] 2012-11-07T04:46:56.708 INFO:teuthology.task.ceph.mon.b.err: 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ea) [0x5f0caa] 2012-11-07T04:46:56.708 INFO:teuthology.task.ceph.mon.b.err: 10: (Thread::create(unsigned long)+0x8a) [0x5e274a] 2012-11-07T04:46:56.708 INFO:teuthology.task.ceph.mon.b.err: 11: (SimpleMessenger::add_accept_pipe(int)+0x6a) [0x5ddb7a] 2012-11-07T04:46:56.708 INFO:teuthology.task.ceph.mon.b.err: 12: (Accepter::entry()+0x279) [0x63b789] 2012-11-07T04:46:56.708 INFO:teuthology.task.ceph.mon.b.err: 13: (()+0x7e9a) [0x7f03b5aa3e9a] 2012-11-07T04:46:56.708 INFO:teuthology.task.ceph.mon.b.err: 14: (clone()+0x6d) [0x7f03b425c4bd] 2012-11-07T04:46:56.709 INFO:teuthology.task.ceph.mon.b.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2012-11-07T04:46:56.709 INFO:teuthology.task.ceph.mon.b.err: 2012-11-07T04:46:56.709 INFO:teuthology.task.ceph.mon.b.err: 0> 2012-11-07 04:46:40.055458 7f03b274d700 -1 *** Caught signal (Aborted) ** 2012-11-07T04:46:56.709 INFO:teuthology.task.ceph.mon.b.err: in thread 7f03b274d700 2012-11-07T04:46:56.709 INFO:teuthology.task.ceph.mon.b.err: 2012-11-07T04:46:56.709 INFO:teuthology.task.ceph.mon.b.err: ceph version 0.53 (commit:2528b5ee105b16352c91af064af5c0b5a7d45d7c) 2012-11-07T04:46:56.709 INFO:teuthology.task.ceph.mon.b.err: 1: /tmp/cephtest/binary/usr/local/bin/ceph-mon() [0x540a61] 2012-11-07T04:46:56.710 INFO:teuthology.task.ceph.mon.b.err: 2: (()+0xfcb0) [0x7f03b5aabcb0] 2012-11-07T04:46:56.710 INFO:teuthology.task.ceph.mon.b.err: 3: (gsignal()+0x35) [0x7f03b41a0445] 2012-11-07T04:46:56.710 INFO:teuthology.task.ceph.mon.b.err: 4: (abort()+0x17b) [0x7f03b41a3bab] 2012-11-07T04:46:56.710 INFO:teuthology.task.ceph.mon.b.err: 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f03b4aee69d] 2012-11-07T04:46:56.710 INFO:teuthology.task.ceph.mon.b.err: 6: (()+0xb5846) [0x7f03b4aec846] 2012-11-07T04:46:56.710 INFO:teuthology.task.ceph.mon.b.err: 7: (()+0xb5873) [0x7f03b4aec873] 2012-11-07T04:46:56.710 INFO:teuthology.task.ceph.mon.b.err: 8: (()+0xb596e) [0x7f03b4aec96e] 2012-11-07T04:46:56.711 INFO:teuthology.task.ceph.mon.b.err: 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ea) [0x5f0caa] 2012-11-07T04:46:56.711 INFO:teuthology.task.ceph.mon.b.err: 10: (Thread::create(unsigned long)+0x8a) [0x5e274a] 2012-11-07T04:46:56.711 INFO:teuthology.task.ceph.mon.b.err: 11: (SimpleMessenger::add_accept_pipe(int)+0x6a) [0x5ddb7a] 2012-11-07T04:46:56.711 INFO:teuthology.task.ceph.mon.b.err: 12: (Accepter::entry()+0x279) [0x63b789] 2012-11-07T04:46:56.711 INFO:teuthology.task.ceph.mon.b.err: 13: (()+0x7e9a) [0x7f03b5aa3e9a] 2012-11-07T04:46:56.711 INFO:teuthology.task.ceph.mon.b.err: 14: (clone()+0x6d) [0x7f03b425c4bd] 2012-11-07T04:46:56.711 INFO:teuthology.task.ceph.mon.b.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. ubuntu@teuthology:/a/teuthology-2012-11-06_02:00:02-regression-testing-master-basic/10579$ cat config.yaml kernel: &id001 kdb: true sha1: 35152979e6181b1fbb4b61c3ff641c14df53ad66 nuke-on-error: true overrides: ceph: conf: global: ms inject socket failures: 500 log-whitelist: - slow request sha1: 2528b5ee105b16352c91af064af5c0b5a7d45d7c s3tests: branch: testing workunit: sha1: 2528b5ee105b16352c91af064af5c0b5a7d45d7c roles: - - mon.a - mon.d - mon.g - mon.j - mon.m - mon.p - mon.s - osd.0 - - mon.b - mon.e - mon.h - mon.k - mon.n - mon.q - mon.t - mds.a - - mon.c - mon.f - mon.i - mon.l - mon.o - mon.r - mon.u - osd.1 targets: ubuntu@plana54.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC60+3L8IN2WBpWHY94YAuCOMVdKs3xUqYQpO1ie127fBomk7fiEhR0RovhmDHWIzNr3qvvNkIp9Y+MHUcpZ7C4MFFMYsy2+zq026Ag3XLEOyZWDSyPfMapd5+nmuvxJqEvAx4wAWBhYVEB3aPFmDmz4mayZ9aSYoA1lhsClxfYpAHZ0zRWX3kY1KxXlk6UrZy0igYGvKIvmubkYcmFzOPsI3aWpgWU1rEXGWsFHOlwaor0KJPnpEsZYTrlPyLZqJcKbI/EcHgti0ak22vsDT7LVMKoyPXeUFL5ZGUEpuqQ+IMiECCMKa8X8vPG2MN9V6DK3gQezF+lo5CRCAu7DYdn ubuntu@plana73.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxoJnvRI1V0OJuQI9SosOedC7mj9O627LjoQWPKilJiBbHduPe1byBaKrgwTeEghl43VNf+EBs1+MwVH7zlDolnwN4tAlW9bRpC2SzURJfhZskp2CSQY3l8ca7a5f0J3hdOhx47oSSapN7O2cqmPzwlL/+MrFKGi+ITT613nUtzCjduZRPdhjyqZ0cQWeb0p1neDw5hbDBKd+HAH+ek/E6DK2PaqN6YAtmIgP76q0fQ85Omd0oDlmGXpKe3jlxlPT0W/5KD1+mpobPsh/EF2qar7IG/WqHHJ6NZAcXbdZ4KiMf9erP+Pk4KkD5SJ+e3GF7OEOwXtahKIIR1An4P2GD ubuntu@plana83.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPiffDjAR8Vr7B1C/WQ4ZiY0rnm4DSvknWpdrM1NMvHhxdrnceyfi6K8y0d+NdQKAkRXzCMbYFM/LU1gUEoIgT6k15wzz/KBUyWPbZ4BCNQNLng3z2aQ80FSA+CWYir0UVCOuj+glDN/MAqWmOwfJ7zHkej1/+hMH5mBTy9Nc8/VS7/yDIQgHLdHmeeMTlGVvBs+uc0b4rBd5kSzhzdwegxO+J86aR/HVsdL7/kEOVHeZQ0yWCT5mRa1+nTveZWYXBz+8ZIFzIx4e7QIhGw/XjDYqqhLD+ZjaUXuKNWG3GqLrQntt42LRvC6flIF11PoL1g0wsxW4DV5Vwdg+zkX71 tasks: - internal.lock_machines: 3 - internal.save_config: null - internal.check_lock: null - internal.connect: null - internal.check_conflict: null - kernel: *id001 - internal.base: null - internal.archive: null - internal.coredump: null - internal.syslog: null - internal.timer: null - chef: null - clock: null - ceph: null - mon_recovery: null
Actions