Project

General

Profile

Actions

Bug #18415

closed

"terminate called after throwing an instance of 'ceph::buffer::end_of_buffer'" in upgrade:jewel-x-kraken-distro-basic-smithi

Added by Yuri Weinstein over 7 years ago. Updated about 7 years ago.

Status:
Can't reproduce
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
upgrade/jewel-x
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Run: http://pulpito.ceph.com/teuthology-2017-01-04_01:47:00-upgrade:jewel-x-kraken-distro-basic-smithi/
Job: 686577
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2017-01-04_01:47:00-upgrade:jewel-x-kraken-distro-basic-smithi/686577/teuthology.log

2017-01-04T02:13:44.183 INFO:teuthology.orchestra.run.smithi037:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops'
2017-01-04T02:13:44.324 INFO:tasks.ceph.mon.c.smithi037.stderr:terminate called after throwing an instance of 'ceph::buffer::end_of_buffer'
2017-01-04T02:13:44.327 INFO:tasks.ceph.mon.c.smithi037.stderr:  what():  buffer::end_of_buffer
2017-01-04T02:13:44.330 INFO:tasks.ceph.mon.c.smithi037.stderr:*** Caught signal (Aborted) **
2017-01-04T02:13:44.332 INFO:tasks.ceph.mon.c.smithi037.stderr: in thread 7f806d1a1700 thread_name:ms_dispatch
2017-01-04T02:13:44.335 INFO:tasks.ceph.mon.c.smithi037.stderr: ceph version 10.2.5-5648-g5b402f8 (5b402f8a7b5a763852e93cd0a5decd34572f4518)
2017-01-04T02:13:44.337 INFO:tasks.ceph.mon.c.smithi037.stderr: 1: (()+0x508fea) [0x7f8074e99fea]
2017-01-04T02:13:44.340 INFO:tasks.ceph.mon.c.smithi037.stderr: 2: (()+0xf370) [0x7f807350b370]
2017-01-04T02:13:44.343 INFO:tasks.ceph.mon.c.smithi037.stderr: 3: (gsignal()+0x37) [0x7f8071d0a1d7]
2017-01-04T02:13:44.346 INFO:tasks.ceph.mon.c.smithi037.stderr: 4: (abort()+0x148) [0x7f8071d0b8c8]
2017-01-04T02:13:44.349 INFO:tasks.ceph.mon.c.smithi037.stderr: 5: (__gnu_cxx::__verbose_terminate_handler()+0x165) [0x7f807260e9d5]
2017-01-04T02:13:44.352 INFO:tasks.ceph.mon.c.smithi037.stderr: 6: (()+0x5e946) [0x7f807260c946]
2017-01-04T02:13:44.355 INFO:tasks.ceph.mon.c.smithi037.stderr: 7: (()+0x5e973) [0x7f807260c973]
2017-01-04T02:13:44.358 INFO:tasks.ceph.mon.c.smithi037.stderr: 8: (()+0x5eb93) [0x7f807260cb93]
2017-01-04T02:13:44.362 INFO:tasks.ceph.mon.c.smithi037.stderr: 9: (()+0x63a119) [0x7f8074fcb119]
2017-01-04T02:13:44.365 INFO:tasks.ceph.mon.c.smithi037.stderr: 10: (MonMap::decode(ceph::buffer::list::iterator&)+0x1f3) [0x7f8074f76123]
2017-01-04T02:13:44.369 INFO:tasks.ceph.mon.c.smithi037.stderr: 11: (MonmapMonitor::update_from_paxos(bool*)+0x21f) [0x7f8074d0706f]
2017-01-04T02:13:44.372 INFO:tasks.ceph.mon.c.smithi037.stderr: 12: (PaxosService::refresh(bool*)+0x1a5) [0x7f8074c67a85]
2017-01-04T02:13:44.375 INFO:tasks.ceph.mon.c.smithi037.stderr: 13: (Monitor::refresh_from_paxos(bool*)+0x15b) [0x7f8074c0141b]
2017-01-04T02:13:44.377 INFO:tasks.ceph.mon.c.smithi037.stderr: 14: (Paxos::do_refresh()+0x41) [0x7f8074c53f11]
2017-01-04T02:13:44.380 INFO:tasks.ceph.mon.c.smithi037.stderr: 15: (Paxos::handle_commit(std::shared_ptr<MonOpRequest>)+0x3c3) [0x7f8074c5b0f3]
2017-01-04T02:13:44.382 INFO:tasks.ceph.mon.c.smithi037.stderr: 16: (Paxos::dispatch(std::shared_ptr<MonOpRequest>)+0x3c4) [0x7f8074c62e34]
2017-01-04T02:13:44.385 INFO:tasks.ceph.mon.c.smithi037.stderr: 17: (Monitor::dispatch_op(std::shared_ptr<MonOpRequest>)+0xb8a) [0x7f8074c31b8a]
2017-01-04T02:13:44.387 INFO:tasks.ceph.mon.c.smithi037.stderr: 18: (Monitor::_ms_dispatch(Message*)+0x6c9) [0x7f8074c326d9]
2017-01-04T02:13:44.390 INFO:tasks.ceph.mon.c.smithi037.stderr: 19: (Monitor::ms_dispatch(Message*)+0x23) [0x7f8074c52f13]
2017-01-04T02:13:44.392 INFO:tasks.ceph.mon.c.smithi037.stderr: 20: (DispatchQueue::entry()+0x78a) [0x7f80750afb6a]
2017-01-04T02:13:44.395 INFO:tasks.ceph.mon.c.smithi037.stderr: 21: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f8074fa365d]
2017-01-04T02:13:44.397 INFO:tasks.ceph.mon.c.smithi037.stderr: 22: (()+0x7dc5) [0x7f8073503dc5]
2017-01-04T02:13:44.400 INFO:tasks.ceph.mon.c.smithi037.stderr: 23: (clone()+0x6d) [0x7f8071dcc73d]
Actions #1

Updated by Josh Durgin over 7 years ago

  • Priority changed from Normal to Immediate
Actions #2

Updated by Samuel Just over 7 years ago

  • Assignee set to Kefu Chai
Actions #3

Updated by Kefu Chai over 7 years ago

the test archive was nuked, rescheduling at http://pulpito.ceph.com/kchai-2017-01-10_10:55:32-upgrade-kraken---basic-mira/. including tests under ubuntu also.

tested with

teuthology-suite --filter="upgrade/jewel-x/stress-split-erasure-code/0-cluster/openstack.yaml" --priority 101 --suite upgrade --machine-type=mira --ceph kraken  --ceph-repo https://github.com/ceph/ceph.git --suite-repo https://github.com/ceph/ceph.git --suite-branch kraken

but the ubuntu tests passed.

Actions #4

Updated by Kefu Chai about 7 years ago

  • Status changed from New to Can't reproduce
  • Priority changed from Immediate to Urgent
Actions

Also available in: Atom PDF