Actions
Bug #42328
closedosd/PrimaryLogPG.cc: 3962: ceph_abort_msg("out of order op")
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Observing on the recent master when running rbd suite [1]:
2019-10-15T09:03:57.023 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr:2019-10-15T09:03:57.025+0000 7fe9d0384700 -1 osd.0 pg_epoch: 34 pg[2.3b( v 34'214 (0'0,34'214] local-lis/les=13/14 n=14 ec=13/13 lis/c=13/13 les/c/f=14/14/0 sis=13) [0,1] r=0 lpr=13 luod=34'213 crt=34'213 lcod 34'212 mlcod 34'212 active+clean ps=[1b~5]] bad op order, already applied 31182 > this 31181
2019-10-15T09:03:57.026 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/15.0.0-6149-g3ebc546/rpm/el7/BUILD/ceph-15.0.0-6149-g3ebc546/src/osd/PrimaryLogPG.cc: In function 'void PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)' thread 7fe9d0384700 time 2019-10-15T09:03:57.025331+0000
2019-10-15T09:03:57.026 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/15.0.0-6149-g3ebc546/rpm/el7/BUILD/ceph-15.0.0-6149-g3ebc546/src/osd/PrimaryLogPG.cc: 3962: ceph_abort_msg("out of order op")
2019-10-15T09:03:57.027 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: ceph version 15.0.0-6149-g3ebc546 (3ebc5465d42ca4df6f6d8453da743de24a676228) octopus (dev)
2019-10-15T09:03:57.027 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::string const&)+0xdd) [0x558ccdc73cc0]
2019-10-15T09:03:57.027 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 2: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0x187e) [0x558ccdeabdce]
2019-10-15T09:03:57.027 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 3: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x324f) [0x558ccdeaf16f]
2019-10-15T09:03:57.027 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 4: (PrimaryLogPG::do_request(boost::intrusive_ptr<OpRequest>&, ThreadPool::TPHandle&)+0xc8d) [0x558ccdeb4f6d]
2019-10-15T09:03:57.027 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 5: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x2f9) [0x558ccdd533c9]
2019-10-15T09:03:57.028 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 6: (PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x558ccdf98299]
2019-10-15T09:03:57.028 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 7: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x143a) [0x558ccdd6e58a]
2019-10-15T09:03:57.028 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 8: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x5b6) [0x558cce33ff56]
2019-10-15T09:03:57.028 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 9: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x558cce342aa0]
2019-10-15T09:03:57.028 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 10: (()+0x7ea5) [0x7fe9f5608ea5]
2019-10-15T09:03:57.028 INFO:tasks.ceph.cluster1.osd.0.smithi045.stderr: 11: (clone()+0x6d) [0x7fe9f44cc8cd]
[1] http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/
Actions