Actions
Subtask #4973
closedsepia - Fix #4977: bad disk in plana49
nightlies: Failed assert at MonitorStore::write_bl_ss in bobtail
Status:
Duplicate
Priority:
High
Assignee:
-
Category:
Monitor
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:
Description
logs: ubuntu@teuthology:/a/teuthology-2013-05-09_01:30:04-upgrade-master-testing-basic/9552
2013-05-09T05:47:02.223 INFO:teuthology.task.workunit.client.0.out:waiting for all operations to complete 2013-05-09T05:47:02.223 INFO:teuthology.task.workunit.client.0.out:cleaning up objects 2013-05-09T05:55:03.362 INFO:teuthology.task.ceph.mon.a.err:mon/MonitorStore.cc: In function 'void MonitorStore::write_bl_ss(ceph::bufferlist&, const char*, const char*, bool)' thread 7f505d278700 time 2013-05-09 05:55:19.270060 2013-05-09T05:55:03.362 INFO:teuthology.task.ceph.mon.a.err:mon/MonitorStore.cc: 384: FAILED assert(!err) 2013-05-09T05:55:03.363 INFO:teuthology.task.ceph.mon.a.err: ceph version 0.56.5-4-g95a0bda (95a0bda7f007a33b0dc7adf4b330778fa1e5d70c) 2013-05-09T05:55:03.363 INFO:teuthology.task.ceph.mon.a.err: 1: (MonitorStore::write_bl_ss(ceph::buffer::list&, char const*, char const*, bool)+0xdfa) [0x533aca] 2013-05-09T05:55:03.363 INFO:teuthology.task.ceph.mon.a.err: 2: (Paxos::stash_latest(unsigned long, ceph::buffer::list&)+0x28c) [0x4a2f2c] 2013-05-09T05:55:03.363 INFO:teuthology.task.ceph.mon.a.err: 3: (PGMonitor::update_from_paxos()+0xb80) [0x5005e0] 2013-05-09T05:55:03.364 INFO:teuthology.task.ceph.mon.a.err: 4: (PaxosService::_active()+0x202) [0x4adbd2] 2013-05-09T05:55:03.364 INFO:teuthology.task.ceph.mon.a.err: 5: (Context::complete(int)+0xa) [0x4910ca] 2013-05-09T05:55:03.364 INFO:teuthology.task.ceph.mon.a.err: 6: (finish_contexts(CephContext*, std::list<Context*, std::allocator<Context*> >&, int)+0x11d) [0x492f0d] 2013-05-09T05:55:03.364 INFO:teuthology.task.ceph.mon.a.err: 7: (Paxos::handle_accept(MMonPaxos*)+0xa6a) [0x4a67fa] 2013-05-09T05:55:03.364 INFO:teuthology.task.ceph.mon.a.err: 8: (Paxos::dispatch(PaxosServiceMessage*)+0x2bb) [0x4aa7fb] 2013-05-09T05:55:03.364 INFO:teuthology.task.ceph.mon.a.err: 9: (Monitor::_ms_dispatch(Message*)+0xfdf) [0x4901af] 2013-05-09T05:55:03.365 INFO:teuthology.task.ceph.mon.a.err: 10: (Monitor::ms_dispatch(Message*)+0x32) [0x49fc22] 2013-05-09T05:55:03.365 INFO:teuthology.task.ceph.mon.a.err: 11: (DispatchQueue::entry()+0x349) [0x64ad09] 2013-05-09T05:55:03.365 INFO:teuthology.task.ceph.mon.a.err: 12: (DispatchQueue::DispatchThread::entry()+0xd) [0x5e40dd] 2013-05-09T05:55:03.365 INFO:teuthology.task.ceph.mon.a.err: 13: (()+0x7e9a) [0x7f5061dd5e9a] 2013-05-09T05:55:03.365 INFO:teuthology.task.ceph.mon.a.err: 14: (clone()+0x6d) [0x7f506058bccd] 2013-05-09T05:55:03.365 INFO:teuthology.task.ceph.mon.a.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2013-05-09T05:55:03.380 INFO:teuthology.task.ceph.mon.a.err:problem writing to /var/log/ceph/ceph-mon.a.log: (30) Read-only file system 2013-05-09T05:55:03.380 INFO:teuthology.task.ceph.mon.a.err:2013-05-09 05:55:19.272561 7f505d278700 -1 mon/MonitorStore.cc: In function 'void MonitorStore::write_bl_ss(ceph::bufferlist&, const char*, const char*, bool)' thread 7f505d278700 time 2013-05-09 05:55:19.270060 2013-05-09T05:55:03.380 INFO:teuthology.task.ceph.mon.a.err:mon/MonitorStore.cc: 384: FAILED assert(!err) 2013-05-09T05:55:03.380 INFO:teuthology.task.ceph.mon.a.err: 2013-05-09T05:55:03.380 INFO:teuthology.task.ceph.mon.a.err: ceph version 0.56.5-4-g95a0bda (95a0bda7f007a33b0dc7adf4b330778fa1e5d70c) 2013-05-09T05:55:03.381 INFO:teuthology.task.ceph.mon.a.err: 1: (MonitorStore::write_bl_ss(ceph::buffer::list&, char const*, char const*, bool)+0xdfa) [0x533aca] 2013-05-09T05:55:03.381 INFO:teuthology.task.ceph.mon.a.err: 2: (Paxos::stash_latest(unsigned long, ceph::buffer::list&)+0x28c) [0x4a2f2c] 2013-05-09T05:55:03.381 INFO:teuthology.task.ceph.mon.a.err: 3: (PGMonitor::update_from_paxos()+0xb80) [0x5005e0] 2013-05-09T05:55:03.381 INFO:teuthology.task.ceph.mon.a.err: 4: (PaxosService::_active()+0x202) [0x4adbd2] 2013-05-09T05:55:03.381 INFO:teuthology.task.ceph.mon.a.err: 5: (Context::complete(int)+0xa) [0x4910ca] 2013-05-09T05:55:03.381 INFO:teuthology.task.ceph.mon.a.err: 6: (finish_contexts(CephContext*, std::list<Context*, std::allocator<Context*> >&, int)+0x11d) [0x492f0d] 2013-05-09T05:55:03.382 INFO:teuthology.task.ceph.mon.a.err: 7: (Paxos::handle_accept(MMonPaxos*)+0xa6a) [0x4a67fa] 2013-05-09T05:55:03.382 INFO:teuthology.task.ceph.mon.a.err: 8: (Paxos::dispatch(PaxosServiceMessage*)+0x2bb) [0x4aa7fb] 2013-05-09T05:55:03.382 INFO:teuthology.task.ceph.mon.a.err: 9: (Monitor::_ms_dispatch(Message*)+0xfdf) [0x4901af] 2013-05-09T05:55:03.382 INFO:teuthology.task.ceph.mon.a.err: 10: (Monitor::ms_dispatch(Message*)+0x32) [0x49fc22] 2013-05-09T05:55:03.382 INFO:teuthology.task.ceph.mon.a.err: 11: (DispatchQueue::entry()+0x349) [0x64ad09] 2013-05-09T05:55:03.382 INFO:teuthology.task.ceph.mon.a.err: 12: (DispatchQueue::DispatchThread::entry()+0xd) [0x5e40dd] 2013-05-09T05:55:03.383 INFO:teuthology.task.ceph.mon.a.err: 13: (()+0x7e9a) [0x7f5061dd5e9a] 2013-05-09T05:55:03.383 INFO:teuthology.task.ceph.mon.a.err: 14: (clone()+0x6d) [0x7f506058bccd] 2013-05-09T05:55:03.383 INFO:teuthology.task.ceph.mon.a.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. ubuntu@teuthology:/a/teuthology-2013-05-09_01:30:04-upgrade-master-testing-basic/9552$ cat config.yaml kernel: &id001 kdb: true sha1: b5b09be30cf99f9c699e825629f02e3bce555d44 machine_type: plana nuke-on-error: true overrides: ceph: conf: mon: debug mon: 20 debug ms: 20 debug paxos: 20 log-whitelist: - slow request sha1: 5177fcb6bf384d5c9caed9b28f28f656823b3d64 s3tests: branch: master workunit: sha1: 5177fcb6bf384d5c9caed9b28f28f656823b3d64 roles: - - mon.a - mds.a - osd.0 - osd.1 - - mon.b - mon.c - osd.2 - osd.3 - - client.0 targets: ubuntu@plana09.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsROY/kuvJRrw3yOos6aHDtveOLPgZ+mBs4ul/O6LTm0swWIpeiilmIeILba12z98XmkoqIGTAgZqcGx6FfNpwrOg8P2pdDg9YKT9YrdESlFZryHyu3Rjyn+lZcskMdHAoXhTUzRkHh7t9/vM5cuhq+BoO5+nAebtFCQ1mcn9w8jn6ZqnXkQiplB93UBmXqGKhbqnaGok6xJfYtV3NtmXOFBJ0keja0rOT4ylnSBExG3YBdZ4qSEaWVNeBtoDslu6K0hq5/mmNtZwJf2iXQWOkhfM3tVUbcMfwaGVEe77gzg3CeU0lT7ag1oJan9MAxzVBqOXO1ZS2h8w4zBrX7eYt ubuntu@plana46.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrD6s9otJ5xCNH4nyv0iJu6AoqmlNTFd8D0X9RfFBnmOMrMBWU9kwsFzPIOsuJGbSYbA8LCtjWUwaWoXmbEFtTMitxaDXp47gbVNXknHq7TGZHkWWOwKKu+tlSQBpCVzO/rzBbvJ9fcG7tewq5XcIHz0IUXsUFuEuXR1HaTUJKic2twBpaeAGNvdd6IZ9Sz9TMkfiRV/aVdcHJ/yF8bsXi3pfRPR3puMK/Nyfq5Hz/aabQo1TSyK2o0weoWV7D8vD6S8f3D7p5/5ScBhL3zUcP85SsV47W+/hTFbU8kN1Grlv2sx0fVMB/TUB/UNVdsHKGn5Nv6zb/qMqBEx9nSeZ9 ubuntu@plana49.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC66Qzm2AumkBR4o9bhYjn1rhOLYTrBdzqECL3wOefSl+Ehkky+8orkWwlqGvaKRbOnTuU5VZ6uUVhXCzdGX+ERtxfO11/SFjt8DIp9ayOD/9vnVPzZMSum3aqkMOEDtMSu7pSENJYGZdoETBqT4bmxyPIUcobQEKjTqKTRt5gv206uLDCQLNiHS2jsrD6BRDwm3rSLe+S+dQvmhX4OhBN27bnSPw6IZIdsN6o2aR2GQYe4RwGMYMS+RktTRKKGUmhmlbzDqApwZCCqpqCZzoeHkYC1acd9BSUnCsWOZBlw8/LsIVeua7VZUmnT9UqUwbKSKAbxzD6nsvs+z6IxWY71 tasks: - internal.lock_machines: - 3 - plana - internal.save_config: null - internal.check_lock: null - internal.connect: null - internal.check_conflict: null - internal.check_ceph_data: null - internal.vm_setup: null - kernel: *id001 - internal.base: null - internal.archive: null - internal.coredump: null - internal.syslog: null - internal.timer: null - chef: null - clock.check: null - install: branch: bobtail - ceph: null - workunit: clients: client.0: - rados/load-gen-mix.sh - install.upgrade: all: branch: next - ceph.restart: - mon.a - mon.b - mon.c - osd.0 - osd.1 - osd.2 - osd.3 - mds.a - workunit: clients: client.0: - rados/test.sh
Updated by Tamilarasi muthamizhan almost 11 years ago
- Source changed from other to Q/A
Updated by Tamilarasi muthamizhan almost 11 years ago
- Assignee set to Joao Eduardo Luis
Updated by Tamilarasi muthamizhan almost 11 years ago
- Priority changed from Normal to Urgent
Updated by Tamilarasi muthamizhan almost 11 years ago
- Status changed from New to Duplicate
- Assignee deleted (
Joao Eduardo Luis) - Priority changed from Urgent to Normal
- Parent task set to #4977
Actions