Ceph

   -18> 2017-08-12 01:08:34.874814 7fe9be5ad700  1 -- 172.21.15.146:6789/0 --> 172.21.15.146:6801/19308 -- auth_reply(proto 2 0 (0) Success) v1 -- 0x7fe9d3d45980 con 0
   -17> 2017-08-12 01:08:34.875124 7fe9be5ad700  1 -- 172.21.15.146:6789/0 <== osd.0 172.21.15.146:6801/19308 4 ==== mon_subscribe({mgrmap=0+,monmap=2+,osd_pg_creates=0+}) v2 ==== 69+0+0 (4278242086 0 0) 0x7fe9d20bfd40 con 0x7fe9d2039800
   -16> 2017-08-12 01:08:34.875139 7fe9be5ad700 20 mon.a@0(leader) e1 _ms_dispatch existing session 0x7fe9d2aca700 for osd.0 172.21.15.146:6801/19308
   -15> 2017-08-12 01:08:34.875143 7fe9be5ad700 20 mon.a@0(leader) e1  caps allow *
   -14> 2017-08-12 01:08:34.875145 7fe9be5ad700 20 is_capable service=mon command= read on cap allow *
   -13> 2017-08-12 01:08:34.875147 7fe9be5ad700 20  allow so far , doing grant allow *
   -12> 2017-08-12 01:08:34.875148 7fe9be5ad700 20  allow all
   -11> 2017-08-12 01:08:34.875149 7fe9be5ad700 10 mon.a@0(leader) e1 handle_subscribe mon_subscribe({mgrmap=0+,monmap=2+,osd_pg_creates=0+}) v2
   -10> 2017-08-12 01:08:34.875155 7fe9be5ad700 20 mon.a@0(leader).mgr e11 Sending map to subscriber 0x7fe9d2039800 172.21.15.146:6801/19308
    -9> 2017-08-12 01:08:34.875160 7fe9be5ad700  1 -- 172.21.15.146:6789/0 --> 172.21.15.146:6801/19308 -- mgrmap(e 11) v1 -- 0x7fe9d3e36940 con 0
    -8> 2017-08-12 01:08:34.875169 7fe9be5ad700 10 mon.a@0(leader).monmap v1 check_sub monmap next 2 have 1
    -7> 2017-08-12 01:08:34.875173 7fe9be5ad700 20 is_capable service=osd command= write on cap allow *
    -6> 2017-08-12 01:08:34.875174 7fe9be5ad700 20  allow so far , doing grant allow *
    -5> 2017-08-12 01:08:34.875175 7fe9be5ad700 20  allow all
    -4> 2017-08-12 01:08:34.875177 7fe9be5ad700 20 mon.a@0(leader).osd e671 check_pg_creates_sub .. osd.0 172.21.15.146:6801/19308
    -3> 2017-08-12 01:08:34.875185 7fe9be5ad700 20 mon.a@0(leader).osd e671 send_pg_creates osd.0 from 0 : epoch 670 34 pgs
    -2> 2017-08-12 01:08:34.875949 7fe9bfdb0700 10 osdmap epoch 671 mapping took 0.003478 seconds
    -1> 2017-08-12 01:08:34.875956 7fe9bfdb0700 10 mon.a@0(leader).osd e671 update_creating_pgs 0 pgs creating, 0 pools in queue
     0> 2017-08-12 01:08:34.877197 7fe9be5ad700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.1.2-724-ge28899a/rpm/el7/BUILD/ceph-12.1.2-724-ge28899a/src/mon/OSDMonitor.cc: In function 'epoch_t OSDMonitor:
:send_pg_creates(int, Connection*, epoch_t) const' thread 7fe9be5ad700 time 2017-08-12 01:08:34.875189
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.1.2-724-ge28899a/rpm/el7/BUILD/ceph-12.1.2-724-ge28899a/src/mon/OSDMonitor.cc: 3284: FAILED assert(create != creating_pgs.pgs.end())

 ceph version 12.1.2-724-ge28899a (e28899a14ce6de67db4320464888553a1ddb92f9) mimic (dev)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7fe9c85ca990]
 2: (OSDMonitor::send_pg_creates(int, Connection*, unsigned int) const+0xcac) [0x7fe9c84d03fc]
 3: (OSDMonitor::check_pg_creates_sub(Subscription*)+0x285) [0x7fe9c84d09a5]
 4: (Monitor::handle_subscribe(boost::intrusive_ptr<MonOpRequest>)+0xc4b) [0x7fe9c838361b]
 5: (Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0x7fd) [0x7fe9c83a810d]
 6: (Monitor::_ms_dispatch(Message*)+0x7eb) [0x7fe9c83a947b]
 7: (Monitor::ms_dispatch(Message*)+0x23) [0x7fe9c83d3a83]
 8: (DispatchQueue::entry()+0x792) [0x7fe9c88663a2]
 9: (DispatchQueue::DispatchThread::entry()+0xd) [0x7fe9c866a61d]
 10: (()+0x7dc5) [0x7fe9c737cdc5]

(gdb) bt
#0  0x00007fceda0a523b in raise () from /lib64/libpthread.so.0
#1  0x00007fcedb5d9fb6 in reraise_fatal (signum=6) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/global/signal_handler.cc:74
#2  handle_fatal_signal (signum=6) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/global/signal_handler.cc:138
#3  <signal handler called>
#4  0x00007fced73751d7 in raise () from /lib64/libc.so.6
#5  0x00007fced73768c8 in abort () from /lib64/libc.so.6
#6  0x00007fcedb2ebb04 in ceph::__ceph_assert_fail (assertion=assertion@entry=0x7fcedb7c8aa8 "create != creating_pgs.pgs.end()", 
    file=file@entry=0x7fcedb7c8300 "/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.1.2-724-ge28899a/rpm/el7/BUILD/ceph-12.1.2-724"..., line=line@entry=3284, 
    func=func@entry=0x7fcedb7d1ee0 <OSDMonitor::send_pg_creates(int, Connection*, unsigned int) const::__PRETTY_FUNCTION__> "epoch_t OSDMonitor::send_pg_creates(int, Connection*, epoch_t) const") at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/common/assert.cc:66
#7  0x00007fcedb1f13fc in OSDMonitor::send_pg_creates (this=this@entry=0x7fcee63a5400, osd=<optimized out>, con=<optimized out>, next=<optimized out>) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/mon/OSDMonitor.cc:3284
#8  0x00007fcedb1f19a5 in OSDMonitor::check_pg_creates_sub (this=0x7fcee63a5400, sub=0x7fcee6648180) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/mon/OSDMonitor.cc:3152
#9  0x00007fcedb0a461b in Monitor::handle_subscribe (this=this@entry=0x7fcee63a4000, op=...) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/mon/Monitor.cc:4911
#10 0x00007fcedb0c910d in Monitor::dispatch_op (this=this@entry=0x7fcee63a4000, op=...) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/mon/Monitor.cc:4268
#11 0x00007fcedb0ca47b in Monitor::_ms_dispatch (this=this@entry=0x7fcee63a4000, m=m@entry=0x7fcee79e1cc0) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/mon/Monitor.cc:4130
#12 0x00007fcedb0f4a83 in Monitor::ms_dispatch (this=0x7fcee63a4000, m=0x7fcee79e1cc0) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/mon/Monitor.h:880
#13 0x00007fcedb5873a2 in ms_deliver_dispatch (m=0x7fcee79e1cc0, this=0x7fcee5b30e00) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/msg/Messenger.h:625
#14 DispatchQueue::entry (this=0x7fcee5b30f58) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/msg/DispatchQueue.cc:197
#15 0x00007fcedb38b61d in DispatchQueue::DispatchThread::entry (this=<optimized out>) at /usr/src/debug/ceph-12.1.2-724-ge28899a/src/msg/DispatchQueue.h:101
#16 0x00007fceda09ddc5 in start_thread () from /lib64/libpthread.so.0

/a/sage-2017-08-11_21:54:25-rados-wip-sage-testing2-20170811a-distro-basic-smithi/1512767