Bug #12968
closed
mon/OSDMonitor.cc: 1864: FAILED assert(osdmap.get_num_up_osds() == 0 || osdmap.get_up_osd_features() & (1ULL<<54))
Added by Sage Weil over 8 years ago.
Updated over 8 years ago.
Description
2015-09-04T14:10:23.156 INFO:tasks.ceph.mon.a.plana79.stderr:mon/OSDMonitor.cc: In function 'bool OSDMonitor::preprocess_boot(MonOpRequestRef)' thread 7f36f243a700 time 2015-09-04 14:10:23.153087
2015-09-04T14:10:23.157 INFO:tasks.ceph.mon.a.plana79.stderr:mon/OSDMonitor.cc: 1864: FAILED assert(osdmap.get_num_up_osds() == 0 || osdmap.get_up_osd_features() & (1ULL<<54))
2015-09-04T14:10:23.167 INFO:tasks.ceph.mon.a.plana79.stderr: ceph version 9.0.3-1372-g210a4b7 (210a4b7dbb0f914d45e2c194a3ec683782f37c4f)
2015-09-04T14:10:23.167 INFO:tasks.ceph.mon.a.plana79.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7f36fb91888b]
2015-09-04T14:10:23.167 INFO:tasks.ceph.mon.a.plana79.stderr: 2: (OSDMonitor::preprocess_boot(std::shared_ptr<MonOpRequest>)+0x1be6) [0x7f36fb758046]
2015-09-04T14:10:23.168 INFO:tasks.ceph.mon.a.plana79.stderr: 3: (OSDMonitor::preprocess_query(std::shared_ptr<MonOpRequest>)+0x357) [0x7f36fb7824f7]
2015-09-04T14:10:23.168 INFO:tasks.ceph.mon.a.plana79.stderr: 4: (PaxosService::dispatch(std::shared_ptr<MonOpRequest>)+0x833) [0x7f36fb737e03]
2015-09-04T14:10:23.168 INFO:tasks.ceph.mon.a.plana79.stderr: 5: (Monitor::dispatch_op(std::shared_ptr<MonOpRequest>)+0x3cd) [0x7f36fb7014ed]
2015-09-04T14:10:23.168 INFO:tasks.ceph.mon.a.plana79.stderr: 6: (Monitor::dispatch(std::shared_ptr<MonOpRequest>)+0x205) [0x7f36fb702225]
2015-09-04T14:10:23.168 INFO:tasks.ceph.mon.a.plana79.stderr: 7: (Monitor::_ms_dispatch(Message*)+0x380) [0x7f36fb7033c0]
2015-09-04T14:10:23.169 INFO:tasks.ceph.mon.a.plana79.stderr: 8: (Monitor::ms_dispatch(Message*)+0x23) [0x7f36fb7212d3]
2015-09-04T14:10:23.169 INFO:tasks.ceph.mon.a.plana79.stderr: 9: (DispatchQueue::entry()+0x63a) [0x7f36fba98dfa]
2015-09-04T14:10:23.169 INFO:tasks.ceph.mon.a.plana79.stderr: 10: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f36fb902e9d]
2015-09-04T14:10:23.170 INFO:tasks.ceph.mon.a.plana79.stderr: 11: (()+0x8182) [0x7f36fa353182]
2015-09-04T14:10:23.170 INFO:tasks.ceph.mon.a.plana79.stderr: 12: (clone()+0x6d) [0x7f36f88c047d]
/a/sage-2015-09-04_11:01:27-rados-wip-sage-testing---basic-multi/1043917
- Assignee set to Kefu Chai
the osd.3 is not upgraded, so the only chance is that some of existing pool is using gmt. but the pools were created when all 6 OSDs were up.
http://pulpito.ceph.com/loic-2015-09-07_01:06:53-rados-infernalis---basic-multi/1045219/
2015-09-06T20:19:01.079 INFO:tasks.ceph.osd.3.mira097.stderr:2015-09-06 20:19:01.081461 7fa8b1454900 -1 osd.3 79 log_to_monitors {default=true}
2015-09-06T20:19:01.243 INFO:tasks.ceph.mon.b.plana44.stderr:mon/OSDMonitor.cc: In function 'bool OSDMonitor::preprocess_boot(MonOpRequestRef)' thread 7ff5dc65b700 time 2015-09-06 20:19:01.240274
2015-09-06T20:19:01.244 INFO:tasks.ceph.mon.b.plana44.stderr:mon/OSDMonitor.cc: 1864: FAILED assert(osdmap.get_num_up_osds() == 0 || osdmap.get_up_osd_features() & (1ULL<<54))
2015-09-06T20:19:01.268 INFO:tasks.ceph.mon.b.plana44.stderr: ceph version 9.0.3-1455-ga51c133 (a51c133bff3b6a22edd4a9a448afed51f5f4836d)
2015-09-06T20:19:01.268 INFO:tasks.ceph.mon.b.plana44.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7ff5e5ec77fb]
2015-09-06T20:19:01.268 INFO:tasks.ceph.mon.b.plana44.stderr: 2: (OSDMonitor::preprocess_boot(std::shared_ptr<MonOpRequest>)+0x1be6) [0x7ff5e5d06d86]
2015-09-06T20:19:01.268 INFO:tasks.ceph.mon.b.plana44.stderr: 3: (OSDMonitor::preprocess_query(std::shared_ptr<MonOpRequest>)+0x357) [0x7ff5e5d31537]
2015-09-06T20:19:01.268 INFO:tasks.ceph.mon.b.plana44.stderr: 4: (PaxosService::dispatch(std::shared_ptr<MonOpRequest>)+0x833) [0x7ff5e5ce6b93]
2015-09-06T20:19:01.268 INFO:tasks.ceph.mon.b.plana44.stderr: 5: (Monitor::dispatch_op(std::shared_ptr<MonOpRequest>)+0x3cd) [0x7ff5e5cb029d]
2015-09-06T20:19:01.269 INFO:tasks.ceph.mon.b.plana44.stderr: 6: (Monitor::dispatch(std::shared_ptr<MonOpRequest>)+0x205) [0x7ff5e5cb0fd5]
2015-09-06T20:19:01.269 INFO:tasks.ceph.mon.b.plana44.stderr: 7: (Monitor::_ms_dispatch(Message*)+0x380) [0x7ff5e5cb2170]
2015-09-06T20:19:01.269 INFO:tasks.ceph.mon.b.plana44.stderr: 8: (Monitor::ms_dispatch(Message*)+0x23) [0x7ff5e5cd0063]
2015-09-06T20:19:01.269 INFO:tasks.ceph.mon.b.plana44.stderr: 9: (DispatchQueue::entry()+0x63a) [0x7ff5e6047d1a]
2015-09-06T20:19:01.269 INFO:tasks.ceph.mon.b.plana44.stderr: 10: (DispatchQueue::DispatchThread::entry()+0xd) [0x7ff5e5eb1e0d]
2015-09-06T20:19:01.269 INFO:tasks.ceph.mon.b.plana44.stderr: 11: (()+0x8182) [0x7ff5e4902182]
2015-09-06T20:19:01.269 INFO:tasks.ceph.mon.b.plana44.stderr: 12: (clone()+0x6d) [0x7ff5e2e7038d]
2015-09-06T20:19:01.270 INFO:tasks.ceph.mon.b.plana44.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2015-09-06T20:19:01.270 INFO:tasks.ceph.mon.b.plana44.stderr:2015-09-06 20:19:01.267419 7ff5dc65b700 -1 mon/OSDMonitor.cc: In function 'bool OSDMonitor::preprocess_boot(MonOpRequestRef)' thread 7ff5dc65b700 time
$ ./ceph osd lspools
0 rbd,1 metadata,2 data,4 unique_pool_0,5 unique_pool_1,6 unique_pool_2,7 test-rados-api-mira122-31528-1,8 test-rados-api-mira122-31528-2,9 test-rados-api-mira122-31528-3,10 test-rados-api-mira122-31528-4,
$ ./ceph osd pool get data all| grep use_gmt_hitset
use_gmt_hitset: 0
$ ./ceph osd pool get unique_pool_0 all| grep use_gmt_hitset
use_gmt_hitset: 1
all pools after (inclusive) unique_pool_0 are using gmt.
- Status changed from New to Fix Under Review
will put this fix into the backport pull request of #9732, so no need to create a backport for this.
- Backport deleted (
hammer)
Your comment is good enough. Removing the hammer tag.
- Status changed from Fix Under Review to Resolved
Also available in: Atom
PDF