Actions
Bug #13185
closedosd/ReplicatedPG.cc: 11062: FAILED assert(obc) on hammer -> infernalis upgrade
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
-4> 2015-09-19 23:19:01.650164 7f6d6f930700 15 filestore(/var/lib/ceph/osd/ceph-3) getattr 429.6_head/429/00000006:.ceph-internal/hit_set_429.6_archive_2015-09-19 23:18:49.366823Z_2015-09-19 23:18:52.551666Z/head '_' -3> 2015-09-19 23:19:01.650242 7f6d6f930700 10 filestore(/var/lib/ceph/osd/ceph-3) error opening file /var/lib/ceph/osd/ceph-3/current/429.6_head/hit\uset\u429.6\uarchive\u2015-09-19 23:18:49.366823Z\u2015-09-19 23:18:52.551666Z__head_00000006_.ceph-internal_1ad with flags=2: (2) No such file or directory -2> 2015-09-19 23:19:01.650254 7f6d6f930700 10 filestore(/var/lib/ceph/osd/ceph-3) getattr 429.6_head/429/00000006:.ceph-internal/hit_set_429.6_archive_2015-09-19 23:18:49.366823Z_2015-09-19 23:18:52.551666Z/head '_' = -2 -1> 2015-09-19 23:19:01.650259 7f6d6f930700 10 osd.3 pg_epoch: 2554 pg[429.6( v 2554'19 (0'0,2554'19] local-les=2554 n=4 ec=2464 les/c 2554/2554 2553/2553/2480) [3,5] r=0 lpr=2553 crt=2554'17 lcod 2554'18 mlcod 2554'18 active+clean NIBBLEWISE] get_object_context: no obc for soid 429/00000006:.ceph-internal/hit_set_429.6_archive_2015-09-19 23:18:49.366823Z_2015-09-19 23:18:52.551666Z/head and !can_create 0> 2015-09-19 23:19:01.668845 7f6d6f930700 -1 osd/ReplicatedPG.cc: In function 'void ReplicatedPG::hit_set_trim(ReplicatedPG::RepGather*, unsigned int)' thread 7f6d6f930700 time 2015-09-19 23:19:01.650276 osd/ReplicatedPG.cc: 11062: FAILED assert(obc) ceph version 9.0.3-1735-g210156f (210156f3d22c491e8e7f6e5c797464c45978f8db) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7f6d9099df6b] 2: (ReplicatedPG::hit_set_trim(ReplicatedPG::RepGather*, unsigned int)+0x54e) [0x7f6d905f80fe] 3: (ReplicatedPG::hit_set_persist()+0xcc1) [0x7f6d905fb611] 4: (ReplicatedPG::do_op(std::shared_ptr<OpRequest>&)+0xf8f) [0x7f6d90603d1f] 5: (ReplicatedPG::do_request(std::shared_ptr<OpRequest>&, ThreadPool::TPHandle&)+0x6dd) [0x7f6d9059da4d] 6: (OSD::dequeue_op(boost::intrusive_ptr<PG>, std::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x3bd) [0x7f6d903fff2d] 7: (PGQueueable::RunVis::operator()(std::shared_ptr<OpRequest>&)+0x5d) [0x7f6d9040014d] 8: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x8c4) [0x7f6d90424ed4] 9: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x85f) [0x7f6d9098e98f] 10: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x7f6d90990890] 11: (()+0x8182) [0x7f6d8ed55182] 12: (clone()+0x6d) [0x7f6d8d09c47d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
i thought this was mismatched timezones but it's not ... the vms all have the same tz.
/a/sage-2015-09-19_14:20:08-upgrade:hammer-x-master---basic-vps/1063065
Updated by Yuri Weinstein about 8 years ago
- Related to Bug #14399: "ReplicatedPG.cc: 10483: FAILED assert(obc)" in rados-hammer-distro-basic-mira added
Actions