Project

General

Profile

Actions
Copy link

Bug #14776

closed

"FAILED assert(0 == "old msgs despite reconnect_seq feature")" in rados-hammer-distro-basic-vps

Added by Yuri Weinstein about 8 years ago. Updated over 7 years ago.

Status:
Can't reproduce
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rados
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Run: http://pulpito.ceph.com/teuthology-2016-02-15_09:00:01-rados-hammer-distro-basic-vps/
Job: 9742
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2016-02-15_09:00:01-rados-hammer-distro-basic-vps/9742/teuthology.log

2016-02-15T11:06:39.600 INFO:tasks.ceph.osd.3.vpm101.stderr:     0> 2016-02-15 19:06:39.327571 7f649fda2700 -1 msg/simple/Pipe.cc: In function 'void Pipe::reader()' thread 7f649fda2700 time 2016-02-15 19:06:39.283230
2016-02-15T11:06:39.600 INFO:tasks.ceph.osd.3.vpm101.stderr:msg/simple/Pipe.cc: 1611: FAILED assert(0 == "old msgs despite reconnect_seq feature")
2016-02-15T11:06:39.601 INFO:tasks.ceph.osd.3.vpm101.stderr:
2016-02-15T11:06:39.601 INFO:tasks.ceph.osd.3.vpm101.stderr: ceph version 0.94.5-305-g7abb6ae (7abb6ae8f3cba67009bd022aaeee0a87cdfc6477)
2016-02-15T11:06:39.601 INFO:tasks.ceph.osd.3.vpm101.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0xba89eb]
2016-02-15T11:06:39.602 INFO:tasks.ceph.osd.3.vpm101.stderr: 2: (Pipe::reader()+0x1831) [0xc836a1]
2016-02-15T11:06:39.602 INFO:tasks.ceph.osd.3.vpm101.stderr: 3: (Pipe::Reader::entry()+0xd) [0xc8c34d]
2016-02-15T11:06:39.602 INFO:tasks.ceph.osd.3.vpm101.stderr: 4: (()+0x8182) [0x7f64c4b63182]
2016-02-15T11:06:39.602 INFO:tasks.ceph.osd.3.vpm101.stderr: 5: (clone()+0x6d) [0x7f64c30ce47d]
2016-02-15T11:06:39.603 INFO:tasks.ceph.osd.3.vpm101.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2016-02-15T11:06:39.603 INFO:tasks.ceph.osd.3.vpm101.stderr:
2016-02-15T11:06:39.604 INFO:tasks.rados.rados.0.vpm154.stdout:345: write initial oid 345
2016-02-15T11:06:39.604 INFO:tasks.rados.rados.0.vpm154.stdout:345:  seq_num 344 ranges {767097=580277,2094348=272749}
2016-02-15T11:06:39.646 INFO:tasks.ceph.osd.3.vpm101.stderr:terminate called after throwing an instance of 'ceph::FailedAssertion'
2016-02-15T11:06:39.661 INFO:tasks.ceph.osd.3.vpm101.stderr:*** Caught signal (Aborted) **
2016-02-15T11:06:39.661 INFO:tasks.ceph.osd.3.vpm101.stderr: in thread 7f649fda2700
2016-02-15T11:06:39.679 INFO:tasks.ceph.osd.3.vpm101.stderr: ceph version 0.94.5-305-g7abb6ae (7abb6ae8f3cba67009bd022aaeee0a87cdfc6477)
2016-02-15T11:06:39.679 INFO:tasks.ceph.osd.3.vpm101.stderr: 1: ceph-osd() [0xab010a]
2016-02-15T11:06:39.679 INFO:tasks.ceph.osd.3.vpm101.stderr: 2: (()+0x10340) [0x7f64c4b6b340]
2016-02-15T11:06:39.680 INFO:tasks.ceph.osd.3.vpm101.stderr: 3: (gsignal()+0x39) [0x7f64c300acc9]
2016-02-15T11:06:39.680 INFO:tasks.ceph.osd.3.vpm101.stderr: 4: (abort()+0x148) [0x7f64c300e0d8]
2016-02-15T11:06:39.680 INFO:tasks.ceph.osd.3.vpm101.stderr: 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f64c3915535]
2016-02-15T11:06:39.680 INFO:tasks.ceph.osd.3.vpm101.stderr: 6: (()+0x5e6d6) [0x7f64c39136d6]
2016-02-15T11:06:39.680 INFO:tasks.ceph.osd.3.vpm101.stderr: 7: (()+0x5e703) [0x7f64c3913703]
2016-02-15T11:06:39.680 INFO:tasks.ceph.osd.3.vpm101.stderr: 8: (()+0x5e922) [0x7f64c3913922]
2016-02-15T11:06:39.681 INFO:tasks.ceph.osd.3.vpm101.stderr: 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0xba8bd8]
2016-02-15T11:06:39.681 INFO:tasks.ceph.osd.3.vpm101.stderr: 10: (Pipe::reader()+0x1831) [0xc836a1]
2016-02-15T11:06:39.681 INFO:tasks.ceph.osd.3.vpm101.stderr: 11: (Pipe::Reader::entry()+0xd) [0xc8c34d]
2016-02-15T11:06:39.681 INFO:tasks.ceph.osd.3.vpm101.stderr: 12: (()+0x8182) [0x7f64c4b63182]
2016-02-15T11:06:39.681 INFO:tasks.ceph.osd.3.vpm101.stderr: 13: (clone()+0x6d) [0x7f64c30ce47d]
Actions
Copy link
 #1

Updated by Samuel Just about 8 years ago

  • Priority changed from Normal to Urgent
Actions
Copy link
 #2

Updated by Yuri Weinstein about 8 years ago

Run: http://pulpito.ceph.com/teuthology-2016-04-02_22:00:02-rados-jewel-distro-basic-smithi/
Job: 104901
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2016-04-02_22:00:02-rados-jewel-distro-basic-smithi/104901/teuthology.log

2016-04-03T05:40:46.667 INFO:tasks.ceph.osd.5.smithi048.stderr:osd/ReplicatedPG.cc: In function 'void ReplicatedPG::execute_ctx(ReplicatedPG::OpContext*)' thread 7fa809493700 time 2016-04-03 12:40:46.661134
2016-04-03T05:40:46.667 INFO:tasks.ceph.osd.5.smithi048.stderr:osd/ReplicatedPG.cc: 2986: FAILED assert(0 == "out of order op")
2016-04-03T05:40:46.685 INFO:tasks.ceph.osd.5.smithi048.stderr: ceph version 10.1.0-351-gdbdbacf (dbdbacfc13e2ff243ade4bf15608ff9a170c686f)
2016-04-03T05:40:46.686 INFO:tasks.ceph.osd.5.smithi048.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7fa82b6c688b]
2016-04-03T05:40:46.686 INFO:tasks.ceph.osd.5.smithi048.stderr: 2: (ReplicatedPG::execute_ctx(ReplicatedPG::OpContext*)+0x1a15) [0x7fa82b1f0e15]
2016-04-03T05:40:46.686 INFO:tasks.ceph.osd.5.smithi048.stderr: 3: (ReplicatedPG::do_op(std::shared_ptr<OpRequest>&)+0x28bb) [0x7fa82b1f3c1b]
2016-04-03T05:40:46.686 INFO:tasks.ceph.osd.5.smithi048.stderr: 4: (ReplicatedPG::do_request(std::shared_ptr<OpRequest>&, ThreadPool::TPHandle&)+0x70c) [0x7fa82b1b1a9c]
2016-04-03T05:40:46.687 INFO:tasks.ceph.osd.5.smithi048.stderr: 5: (OSD::dequeue_op(boost::intrusive_ptr<PG>, std::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x3f5) [0x7fa82b071f55]
2016-04-03T05:40:46.687 INFO:tasks.ceph.osd.5.smithi048.stderr: 6: (PGQueueable::RunVis::operator()(std::shared_ptr<OpRequest>&)+0x5d) [0x7fa82b07217d]
2016-04-03T05:40:46.687 INFO:tasks.ceph.osd.5.smithi048.stderr: 7: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x869) [0x7fa82b076b99]
2016-04-03T05:40:46.687 INFO:tasks.ceph.osd.5.smithi048.stderr: 8: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x877) [0x7fa82b6b6d87]
2016-04-03T05:40:46.688 INFO:tasks.ceph.osd.5.smithi048.stderr: 9: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x7fa82b6b8cb0]
2016-04-03T05:40:46.688 INFO:tasks.ceph.osd.5.smithi048.stderr: 10: (()+0x8182) [0x7fa82998c182]
2016-04-03T05:40:46.688 INFO:tasks.ceph.osd.5.smithi048.stderr: 11: (clone()+0x6d) [0x7fa827aba47d]
Actions
Copy link
 #3

Updated by Samuel Just over 7 years ago

  • Status changed from New to Can't reproduce
Actions
Copy link

Also available in: Atom PDF