Bug #7696: osd/ECUtil.cc: 23: FAILED assert(i->second.length() == total_chunk_size) - Ceph - Ceph

Actions

Copy link

Bug #7696

closed

osd/ECUtil.cc: 23: FAILED assert(i->second.length() == total_chunk_size)

Added by Samuel Just about 10 years ago. Updated about 10 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Samuel Just

Category:

Target version:

% Done:

Source:

other

Tags:

Backport:

Regression:

Severity:

2 - major

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Primary and replica disagree on last_backfill for some reason.

3> 2014-03-12 03:31:58.711369 7feaff6c9700 10 osd.0 pg_epoch: 1094 pg[5.9s1( v 1094'443 (0'0,1094'443] local-les=1030 n=8 ec=8 les/c 1030/1036 1029/1029/49) [1,0,3] r=1 lpr=1029 pi=8-1028/6 luod=0'0 crt=0'0 active] op_applied on replica on version 1094'443
    -2> 2014-03-12 03:31:58.711380 7feaff6c9700  1 - 10.214.134.34:6801/30581 --> osd.1 10.214.134.34:6806/30582 -- MOSDECSubOpWriteReply(5.9s0 1094 ECSubWriteReply(tid=9776, last_complete=0'0, committed=0, applied=1)) v1 -- ?+0 0x7c1d000
    1> 2014-03-12 03:31:58.711403 7feaff6c9700  5 - op tracker -- , seq: 85033, time: 2014-03-12 03:31:58.711403, event: done, request: MOSDECSubOpWrite(5.9s1 1094 ECSubWrite(tid=9776, reqid=client.4130.0:4078, at_version=1094'443, trim_to=0'0)) v1
     0> 2014-03-12 03:31:58.770190 7feaf5587700 -1 ** Caught signal (Aborted) *
 in thread 7feaf5587700

ceph version 0.77-809-g205c29e (205c29ea7581d075997d6fb5a9a49216dee8e6f6)
 1: ceph-osd() [0x98205a]
 2: (()+0xfcb0) [0x7feb0a432cb0]
 3: (gsignal()+0x35) [0x7feb08b31425]
 4: (abort()+0x17b) [0x7feb08b34b8b]
 5: (_gnu_cxx::_verbose_terminate_handler()+0x11d) [0x7feb0948469d]
 6: (()+0xb5846) [0x7feb09482846]
 7: (()+0xb5873) [0x7feb09482873]
 8: (()+0xb596e) [0x7feb0948296e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0xa60d8f]
 10: (ECUtil::decode(ECUtil::stripe_info_t const&, std::tr1::shared_ptr&lt;ceph::ErasureCodeInterface&gt;&, std::map&lt;int, ceph::buffer::list, std::less&lt;int&gt;, std::allocator&lt;std::pair&lt;int const, ceph::buffer::list&gt; > >&, ceph::buffer::list*)+0xa46) [0x97d8d6]
 11: (CallClientContexts::finish(std::pair&lt;RecoveryMessages*, ECBackend::read_result_t&&gt;&)+0x248) [0x96b258]
 12: (GenContext&lt;std::pair&lt;RecoveryMessages*, ECBackend::read_result_t&&gt;&>::complete(std::pair&lt;RecoveryMessages*, ECBackend::read_result_t&&gt;&)+0x9) [0x9627a9]
 13: (ECBackend::complete_read_op(ECBackend::ReadOp&, RecoveryMessages*)+0x5b) [0x95203b]
 14: (ECBackend::handle_sub_read_reply(pg_shard_t, ECSubReadReply&, RecoveryMessages*)+0xcd5) [0x95c8a5]
 15: (ECBackend::handle_message(std::tr1::shared_ptr&lt;OpRequest&gt;)+0x506) [0x960456]
 16: (ReplicatedPG::do_request(std::tr1::shared_ptr&lt;OpRequest&gt;, ThreadPool::TPHandle&)+0x1ee) [0x856dbe]
 17: (OSD::dequeue_op(boost::intrusive_ptr&lt;PG&gt;, std::tr1::shared_ptr&lt;OpRequest&gt;, ThreadPool::TPHandle&)+0x34a) [0x61916a]
 18: (OSD::OpWQ::_process(boost::intrusive_ptr&lt;PG&gt;, ThreadPool::TPHandle&)+0x4a0) [0x6341a0]
 19: (ThreadPool::WorkQueueVal&lt;std::pair&lt;boost::intrusive_ptr&lt;PG&gt;, std::tr1::shared_ptr&lt;OpRequest&gt; >, boost::intrusive_ptr&lt;PG&gt; >::_void_process(void*, ThreadPool::TPHandle&)+0x9c) [0x67976c]
 20: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0xa515c6]
 21: (ThreadPool::WorkThread::entry()+0x10) [0xa533d0]
 22: (()+0x7e9a) [0x7feb0a42ae9a]
 23: (clone()+0x6d) [0x7feb08bef3fd]
 NOTE: a copy of the executable, or `objdump -rdS &lt;executable&gt;` is needed to interpret this.