Project

General

Profile

Actions

Bug #10658

closed

"osd/osd_types.h: 3011: FAILED assert(rwstate.empty())" in upgrade:dumpling-firefly-x-giant-distro-basic-multi run

Added by Yuri Weinstein about 9 years ago. Updated about 9 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Run: http://pulpito.ceph.com/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/
Job: 722247
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/

In /a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz -

1634 pg[272.5( v 1634'7 (0'0,1634'7] local-les=1624 n=2 ec=1623 les/c 1624/1624 1623/1623/1623) [1,0] r=0 lpr=1623 crt=1632'2 lcod 1633'5 mlcod 1633'5 active+clean]  snapset_obc obc(7c6ccaf5/foo/snapdir/test-rados-api-plana48-11810-11/272 rwstate(write n=2 w=0))
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803008402-2015-01-26 13:26:58.893415 7f77fa789700 15 osd.1 pg_epoch: 1634 pg[272.5( v 1634'7 (0'0,1634'7] local-les=1624 n=2 ec=1623 les/c 1624/1624 1623/1623/1623) [1,0] r=0 lpr=1623 crt=1632'2 lcod 1633'5 mlcod 1633'5 active+clean]  requeue_ops 
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803008640-2015-01-26 13:26:58.897910 7f77fa789700 -1 osd/osd_types.h: In function 'ObjectContext::~ObjectContext()' thread 7f77fa789700 time 2015-01-26 13:26:58.893432
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803008798-osd/osd_types.h: 3011: FAILED assert(rwstate.empty())
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803008852-
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803008853: ceph version 0.87-81-ga463b92 (a463b92e475cd1f4cdb963e402033ebc9d37dbdc)
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803008927- 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7f) [0xaa031f]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009018- 2: (SharedPtrRegistry<hobject_t, ObjectContext>::OnRemoval::operator()(ObjectContext*)+0x27d) [0x86c48d]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009124- 3: (std::tr1::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count()+0x49) [0x67daf9]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009218- 4: (ReplicatedPG::OpContext::~OpContext()+0xf7) [0x850d07]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009278- 5: (intrusive_ptr_release(ReplicatedPG::RepGather*)+0x33) [0x7d1683]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009348- 6: (C_OSD_RepopCommit::~C_OSD_RepopCommit()+0x24) [0x851394]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009410- 7: (ReplicatedBackend::sub_op_modify_reply(std::tr1::shared_ptr<OpRequest>)+0x28a) [0x9365da]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009505- 8: (ReplicatedBackend::handle_message(std::tr1::shared_ptr<OpRequest>)+0x504) [0x936f54]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009595- 9: (ReplicatedPG::do_request(std::tr1::shared_ptr<OpRequest>&, ThreadPool::TPHandle&)+0x15a) [0x7ce21a]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009700- 10: (OSD::dequeue_op(boost::intrusive_ptr<PG>, std::tr1::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x17f) [0x64ddbf]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009822- 11: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x65f) [0x64e81f]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009914- 12: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x652) [0xa915d2]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803009996- 13: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0xa92d00]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803010065- 14: (()+0x7e9a) [0x7f78150d8e9a]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803010099- 15: (clone()+0x6d) [0x7f7813a8a3fd]
/a/teuthology-2015-01-25_18:10:02-upgrade:dumpling-firefly-x-giant-distro-basic-multi/722247/remote/plana51/log/ceph-osd.1.log.gz:803010136- NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Actions #1

Updated by Samuel Just about 9 years ago

  • Priority changed from Normal to Urgent
Actions #2

Updated by Sage Weil about 9 years ago

  • Status changed from New to Resolved
Actions

Also available in: Atom PDF