https://tracker.ceph.com/
https://tracker.ceph.com/favicon.ico
2017-05-17T14:29:50Z
Ceph
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91137
2017-05-17T14:29:50Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-05-16_22:22:40-rados-wip-sage-testing2---basic-smithi/1186082</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91285
2017-05-21T10:10:37Z
Kefu Chai
tchaikov@gmail.com
<ul></ul><p>/a/kchai-2017-05-21_04:27:34-rados-wip-bigbang---basic-mira/1203867/remote/mira104/log/ceph-osd.0.log.gz</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91472
2017-05-24T02:42:28Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-05-23_19:52:01-rados-wip-sage-testing2---basic-smithi/1221342</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91714
2017-05-26T13:49:59Z
Sage Weil
sage@newdream.net
<ul><li><strong>File</strong> <a href="/attachments/download/2818/gdb.txt">gdb.txt</a> <a class="icon-only icon-magnifier" title="View" href="/attachments/2818/gdb.txt">View</a> added</li></ul><p>attached gdb dump</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91718
2017-05-26T14:02:24Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-05-26_06:56:21-rados-wip-sage-testing---basic-smithi/1231195</p>
<p>core file in test dir along with a copy of the centos7 packages that were installed.</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91766
2017-05-28T02:06:59Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-05-27_03:43:09-rados-wip-sage-testing2---basic-smithi/1235284</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91767
2017-05-28T02:33:57Z
Kefu Chai
tchaikov@gmail.com
<ul></ul><p>/a/kchai-2017-05-27_07:55:19-rados-wip-mgr-stats-kefu---basic-mira/1237260/remote/mira100/log/ceph-osd.2.log.gz</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91781
2017-05-29T19:37:08Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-05-28_05:00:18-rados-wip-sage-testing---basic-smithi/1238213</p>
<p>rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml workloads/cache-pool-snaps.yaml}</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91819
2017-05-31T02:45:27Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-05-30_17:22:40-rados-wip-sage-testing---basic-smithi/1244707</p>
<p>rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml workloads/pool-snaps-few-objects.yaml}</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=91949
2017-06-01T16:46:04Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-06-01_02:27:12-rados-wip-sage-testing2---basic-smithi/1249818</p>
<p>rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml fast/fast.yaml leveldb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml}</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92040
2017-06-02T15:15:55Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-06-02_08:32:01-rados-wip-sage-testing-distro-basic-smithi/1255551</p>
<p>rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml workloads/pool-snaps-few-objects.yaml}</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92193
2017-06-05T18:32:22Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-06-05_14:47:27-rados-wip-sage-testing-distro-basic-smithi/1260429</p>
<p>rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml fast/normal.yaml leveldb.yaml msgr-failures/osd-delay.yaml rados.yaml thrashers/default.yaml workloads/ec-small-objects-fast-read-overwrites.yaml}</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92196
2017-06-05T21:34:39Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-06-05_18:36:01-rados-wip-sage-testing2-distro-basic-smithi/1261877<br />rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml workloads/pool-snaps-few-objects.yaml}</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92197
2017-06-05T21:35:13Z
Sage Weil
sage@newdream.net
<ul></ul><p>/a/sage-2017-06-05_18:36:01-rados-wip-sage-testing2-distro-basic-smithi/1261940</p>
<p>rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml workloads/cache-pool-snaps.yaml}</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92199
2017-06-05T22:31:59Z
Sage Weil
sage@newdream.net
<ul></ul><p>trim_object() returns an OpContextUPtr that we feed to simple_opc_submit(), but we aren't calling finish_ctx() like everyone else. As a result, when we trim a whiteout, the oi isn't getting cleared, and the version field isn't cleared. This means the prior_version is screwed up for subsequent events.</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92200
2017-06-05T22:32:07Z
Sage Weil
sage@newdream.net
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>Assignee</strong> set to <i>Sage Weil</i></li></ul>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92260
2017-06-06T18:02:54Z
Sage Weil
sage@newdream.net
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Fix Under Review</i></li></ul><p><a class="external" href="https://github.com/ceph/ceph/pull/15519">https://github.com/ceph/ceph/pull/15519</a></p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92398
2017-06-07T20:59:05Z
Sage Weil
sage@newdream.net
<ul></ul><p>Guessing this is related, since it's also an incorrect prior_version:<br /><pre>
2017-06-07T18:59:40.728 INFO:tasks.ceph.osd.2.smithi037.stderr:/mnt/jenkins/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.0.2-2485-gc8340cd/rpm/el7/BUILD/ceph-12.0.2-2485-gc8340cd/src/osd/PGLog.h: In function 'static void PGLog::_merg
e_object_divergent_entries(const PGLog::IndexedLog&, const hobject_t&, mempool::osd::list<pg_log_entry_t>&, const pg_info_t&, eversion_t, missing_type&, PGLog::LogEntryHandler*, const DoutPrefixProvider*) [with missing_type = pg_missing_set<true>; mempool::osd::list<pg_log_entry_t> = std::list<pg_log_entry_t, mempo
ol::pool_allocator<(mempool::pool_index_t)12u, pg_log_entry_t> >]' thread 7f6961877700 time 2017-06-07 18:59:40.734025
2017-06-07T18:59:40.728 INFO:tasks.ceph.osd.2.smithi037.stderr:/mnt/jenkins/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.0.2-2485-gc8340cd/rpm/el7/BUILD/ceph-12.0.2-2485-gc8340cd/src/osd/PGLog.h: 775: FAILED assert(i->prior_version =
= last)
2017-06-07T18:59:40.733 INFO:tasks.ceph.osd.2.smithi037.stderr: ceph version 12.0.2-2485-gc8340cd (c8340cde85674f8d9506d602368c2fd9a6307580) luminous (dev)
2017-06-07T18:59:40.733 INFO:tasks.ceph.osd.2.smithi037.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7f697bce79d0]
2017-06-07T18:59:40.733 INFO:tasks.ceph.osd.2.smithi037.stderr: 2: (void PGLog::_merge_object_divergent_entries<pg_missing_set<true> >(PGLog::IndexedLog const&, hobject_t const&, std::list<pg_log_entry_t, mempool::pool_allocator<(mempool::pool_index_t)12, pg_log_entry_t> > const&, pg_info_t const&, eversion_t, pg_m
issing_set<true>&, PGLog::LogEntryHandler*, DoutPrefixProvider const*)+0x620) [0x7f697b8c1470]
2017-06-07T18:59:40.733 INFO:tasks.ceph.osd.2.smithi037.stderr: 3: (void PGLog::_merge_divergent_entries<pg_missing_set<true> >(PGLog::IndexedLog const&, std::list<pg_log_entry_t, mempool::pool_allocator<(mempool::pool_index_t)12, pg_log_entry_t> >&, pg_info_t const&, eversion_t, pg_missing_set<true>&, PGLog::LogEn
tryHandler*, DoutPrefixProvider const*)+0x1cc) [0x7f697b8c33bc]
2017-06-07T18:59:40.733 INFO:tasks.ceph.osd.2.smithi037.stderr: 4: (PGLog::rewind_divergent_log(eversion_t, pg_info_t&, PGLog::LogEntryHandler*, bool&, bool&)+0x349) [0x7f697b8ba909]
2017-06-07T18:59:40.733 INFO:tasks.ceph.osd.2.smithi037.stderr: 5: (PGLog::merge_log(pg_info_t&, pg_log_t&, pg_shard_t, pg_info_t&, PGLog::LogEntryHandler*, bool&, bool&)+0xd48) [0x7f697b8bb808]
2017-06-07T18:59:40.733 INFO:tasks.ceph.osd.2.smithi037.stderr: 6: (PG::merge_log(ObjectStore::Transaction&, pg_info_t&, pg_log_t&, pg_shard_t)+0x6c) [0x7f697b82063c]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 7: (PG::proc_master_log(ObjectStore::Transaction&, pg_info_t&, pg_log_t&, pg_missing_set<false>&, pg_shard_t)+0x1bc) [0x7f697b8694cc]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 8: (PG::RecoveryState::GetLog::react(PG::RecoveryState::GotLog const&)+0xd1) [0x7f697b8697d1]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 9: (boost::statechart::simple_state<PG::RecoveryState::GetLog, PG::RecoveryState::Peering, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_
::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0x242) [0x7f697b8b48a2]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 10: (boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine, PG::RecoveryState::Initial, std::allocator<void>, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x6b) [0x7f697b88eebb]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 11: (boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine, PG::RecoveryState::Initial, std::allocator<void>, boost::statechart::null_exception_translator>::process_queued_events()+0x91) [0x7f697b88f011]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 12: (PG::handle_peering_event(std::shared_ptr<PG::CephPeeringEvt>, PG::RecoveryCtx*)+0x1fe) [0x7f697b85aeae]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 13: (OSD::process_peering_events(std::list<PG*, std::allocator<PG*> > const&, ThreadPool::TPHandle&)+0x22a) [0x7f697b7abdda]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 14: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> > const&, ThreadPool::TPHandle&)+0x17) [0x7f697b809da7]
2017-06-07T18:59:40.734 INFO:tasks.ceph.osd.2.smithi037.stderr: 15: (ThreadPool::worker(ThreadPool::WorkThread*)+0xb59) [0x7f697bcee6d9]
2017-06-07T18:59:40.735 INFO:tasks.ceph.osd.2.smithi037.stderr: 16: (ThreadPool::WorkThread::entry()+0x10) [0x7f697bcef6b0]
2017-06-07T18:59:40.735 INFO:tasks.ceph.osd.2.smithi037.stderr: 17: (()+0x7dc5) [0x7f697892bdc5]
2017-06-07T18:59:40.735 INFO:tasks.ceph.osd.2.smithi037.stderr: 18: (clone()+0x6d) [0x7f6977a1f73d]
</pre><br />/a/sage-2017-06-07_16:25:35-rados-wip-sage-testing2-distro-basic-smithi/1268235</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92471
2017-06-08T22:14:41Z
Sage Weil
sage@newdream.net
<ul><li><strong>Status</strong> changed from <i>Fix Under Review</i> to <i>7</i></li></ul>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92682
2017-06-13T03:01:16Z
Sage Weil
sage@newdream.net
<ul><li><strong>Status</strong> changed from <i>7</i> to <i>Resolved</i></li></ul>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92926
2017-06-14T21:54:22Z
Sage Weil
sage@newdream.net
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>7</i></li></ul><p>DIdn't fix it fully... see<br /> /a/sage-2017-06-14_19:30:57-rados:thrash-wip-19943---basic-smithi/1286357</p>
Ceph - Bug #19947: osd/PGLog.h: 1235: FAILED assert(miter->second.have == eversion_t())
https://tracker.ceph.com/issues/19947?journal_id=92960
2017-06-15T21:33:19Z
Sage Weil
sage@newdream.net
<ul><li><strong>Status</strong> changed from <i>7</i> to <i>Resolved</i></li></ul>