Ceph

I'm not sure if this is the same issue as #326, that's why i open a new issue.

At the same moment #326 took place, this OSD went down too.

Core was generated by `/usr/bin/cosd -i 21 -c /etc/ceph/ceph.conf'.
Program terminated with signal 6, Aborted.
#0  0x00007fa1a483ba75 in raise () from /lib/libc.so.6
(gdb) bt
#0  0x00007fa1a483ba75 in raise () from /lib/libc.so.6
#1  0x00007fa1a483f5c0 in abort () from /lib/libc.so.6
#2  0x00007fa1a50f08e5 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/libstdc++.so.6
#3  0x00007fa1a50eed16 in ?? () from /usr/lib/libstdc++.so.6
#4  0x00007fa1a50eed43 in std::terminate() () from /usr/lib/libstdc++.so.6
#5  0x00007fa1a50eee3e in __cxa_throw () from /usr/lib/libstdc++.so.6
#6  0x00000000005bdc08 in ceph::__ceph_assert_fail(char const*, char const*, int, char const*) ()
#7  0x00000000005233b3 in PG::IndexedLog::print(std::ostream&) const ()
#8  0x00000000004de8f7 in OSD::_process_pg_info(unsigned int, int, PG::Info&, PG::Log&, PG::Missing&, std::map<int, MOSDPGInfo*, std::less<int>, std::allocator<std::pair<int const, MOSDPGInfo*> > >*, int&) ()
#9  0x00000000004e8261 in OSD::handle_pg_log(MOSDPGLog*) ()
#10 0x00000000004ef20d in OSD::_dispatch(Message*) ()
#11 0x00000000004efb19 in OSD::ms_dispatch(Message*) ()
#12 0x0000000000461d59 in SimpleMessenger::dispatch_entry() ()
#13 0x0000000000458d1c in SimpleMessenger::DispatchThread::entry() ()
#14 0x000000000046ccca in Thread::_entry_func(void*) ()
#15 0x00007fa1a56ce9ca in start_thread () from /lib/libpthread.so.0
#16 0x00007fa1a48ee6cd in clone () from /lib/libc.so.6
#17 0x0000000000000000 in ?? ()
(gdb)

3765'8211 (2931'8207) m 10000108158.00000000/head by mds0.86:1064 10.07.28_21:36:12.467067 indexed
osd/PG.cc: In function 'std::ostream& PG::IndexedLog::print(std::ostream&) const':
osd/PG.cc:698: FAILED assert(!p->reqid_is_indexed() || logged_req(p->reqid))
 1: (OSD::_process_pg_info(unsigned int, int, PG::Info&, PG::Log&, PG::Missing&, std::map<int, MOSDPGInfo*, std::less<int>, std::allocator<std::pair<int const, MOSDPGInfo*> > >*, int&)+0x8a7) [0x4de8f7]
 2: (OSD::handle_pg_log(MOSDPGLog*)+0xb1) [0x4e8261]
 3: (OSD::_dispatch(Message*)+0x35d) [0x4ef20d]
 4: (OSD::ms_dispatch(Message*)+0x39) [0x4efb19]
 5: (SimpleMessenger::dispatch_entry()+0x749) [0x461d59]
 6: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x458d1c]
 7: (Thread::_entry_func(void*)+0xa) [0x46ccca]
 8: (()+0x69ca) [0x7fa1a56ce9ca]
 9: (clone()+0x6d) [0x7fa1a48ee6cd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

The core, binary and logs are available on logger.ceph.widodh.nl in the directory /srv/ceph/issues/osd_crash_pg_indexedlog_print