Project

General

Profile

Actions

Bug #19191

closed

osd/ReplicatedBackend.cc: 1109: FAILED assert(!parent->get_log().get_missing().is_missing(soid))

Added by Sage Weil about 7 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
PG Split
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
OSD
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

  -334> 2017-03-04 02:19:34.154279 7f9a40a60700 10 osd.1 pg_epoch: 93 pg[1.e( v 93'2357 lc 0'0 (72'1265,93'2357] lb MIN (bitwise) local-les=65 n=33 ec=7 les/c/f 65/48/0 84/88/52) [2,1]/[4,5] r=-1 lpr=88 pi=7-87/8 luod=0'0 lua=93'2356 crt=93'2357 active+remapped m=17] handle_message: 0x7f9a71228f00
  -333> 2017-03-04 02:19:34.154292 7f9a40a60700 10 osd.1 pg_epoch: 93 pg[1.e( v 93'2357 lc 0'0 (72'1265,93'2357] lb MIN (bitwise) local-les=65 n=33 ec=7 les/c/f 65/48/0 84/88/52) [2,1]/[4,5] r=-1 lpr=88 pi=7-87/8 luod=0'0 lua=93'2356 crt=93'2357 active+remapped m=17] sub_op_modify trans 1:70ffb74a:::benchmark_data_

     0> 2017-03-04 02:19:34.157419 7f9a40a60700 -1 /build/ceph-12.0.0-1023-gfcbb6f6/src/osd/ReplicatedBackend.cc: In function 'void ReplicatedBackend::sub_op_modify(OpRequestRef)' thread 7f9a40a60700 time 2017-03-04 02:19:34.154304
/build/ceph-12.0.0-1023-gfcbb6f6/src/osd/ReplicatedBackend.cc: 1109: FAILED assert(!parent->get_log().get_missing().is_missing(soid))

 ceph version 12.0.0-1023-gfcbb6f6 (fcbb6f681122104df058e33b18d1d007dc4a2d0f)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x7f9a5d8c6a2e]
 2: (ReplicatedBackend::sub_op_modify(boost::intrusive_ptr<OpRequest>)+0x10f3) [0x7f9a5d62be13]
 3: (ReplicatedBackend::handle_message(boost::intrusive_ptr<OpRequest>)+0x324) [0x7f9a5d630e84]
 4: (PrimaryLogPG::do_request(boost::intrusive_ptr<OpRequest>&, ThreadPool::TPHandle&)+0x445) [0x7f9a5d4fa105]
 5: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x212) [0x7f9a5d3b0d42]
 6: (PGQueueable::RunVis::operator()(boost::intrusive_ptr<OpRequest> const&)+0x47) [0x7f9a5d3b1177]
 7: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xff5) [0x7f9a5d3d9f25]
 8: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x955) [0x7f9a5d8cc355]
 9: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x7f9a5d8ce4b0]
 10: (()+0x8184) [0x7f9a5b65a184]
 11: (clone()+0x6d) [0x7f9a5a74a37d]

it's a backfill target that got an empty opt

2017-03-04 02:19:34.158019 7f94b8215700 10 osd.4 pg_epoch: 93 pg[1.e( v 93'2357 (72'1265,93'2357] local-les=89 n=102 ec=7 les/c/f 89/48/0 84/88/52) [2,1]/[4,5] r=0 lpr=88 pi=7-87/8 luod=93'2356 bft=1,2 crt=93'2357 lcod 93'2355 mlcod 0'0 active+remapped+backfill_wait] issue_repop shipping empty opt to osd.1, object 1:70ffb74a:::benchmark_data_smithi157_18125_object21406:head beyond MAX(last_backfill_started , pinfo.last_backfill MIN)

but hit this assert
  // we better not be missing this.
  assert(!parent->get_log().get_missing().is_missing(soid));

/a/sage-2017-03-03_17:51:47-rados-wip-osd-full---basic-smithi/878986

Actions

Also available in: Atom PDF