2016-12-07 08:59:38.164863 7f8dfe4f9480 0 set uid:gid to 64045:64045 (ceph:ceph) 2016-12-07 08:59:38.164885 7f8dfe4f9480 0 ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b), process ceph-mon, pid 3695 2016-12-07 08:59:38.164955 7f8dfe4f9480 0 pidfile_write: ignore empty --pid-file 2016-12-07 08:59:38.173417 7f8dfe4f9480 10 load: jerasure load: lrc load: isa 2016-12-07 08:59:38.173757 7f8dfe4f9480 1 leveldb: Recovering log #8837 2016-12-07 08:59:38.173794 7f8dfe4f9480 1 leveldb: Level-0 table #8839: started 2016-12-07 08:59:38.179928 7f8dfe4f9480 1 leveldb: Level-0 table #8839: 142 bytes OK 2016-12-07 08:59:38.193276 7f8dfe4f9480 1 leveldb: Delete type=2 #8838 2016-12-07 08:59:38.193681 7f8dfe4f9480 1 leveldb: Delete type=3 #8835 2016-12-07 08:59:38.193713 7f8dfe4f9480 1 leveldb: Delete type=0 #8837 2016-12-07 08:59:38.194394 7f8df349c700 1 leveldb: Compacting 68@0 + 0@1 files 2016-12-07 08:59:38.194626 7f8dfe4f9480 10 obtain_monmap 2016-12-07 08:59:38.194977 7f8dfe4f9480 10 obtain_monmap read last committed monmap ver 3 2016-12-07 08:59:38.195031 7f8dfe4f9480 0 starting mon.ceph3 rank 0 at 10.10.38.42:6789/0 mon_data /var/lib/ceph/mon/ceph-ceph3 fsid 7fddfc10-5e7d-44c6-ac0b-fdd129e4219b 2016-12-07 08:59:38.195807 7f8dfe4f9480 20 log_channel(cluster) update_config log_to_monitors {default=true} log_to_syslog {default=false} log_channels {audit=local0,default=daemon} log_prios {default=info} 2016-12-07 08:59:38.195816 7f8dfe4f9480 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) 2016-12-07 08:59:38.195819 7f8dfe4f9480 20 log_channel(audit) update_config log_to_monitors {default=true} log_to_syslog {default=false} log_channels {audit=local0,default=daemon} log_prios {default=info} 2016-12-07 08:59:38.195821 7f8dfe4f9480 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) 2016-12-07 08:59:38.195923 7f8dfe4f9480 1 mon.ceph3@-1(probing) e3 preinit fsid 7fddfc10-5e7d-44c6-ac0b-fdd129e4219b 2016-12-07 08:59:38.196067 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 check_fsid cluster_uuid contains '7fddfc10-5e7d-44c6-ac0b-fdd129e4219b' 2016-12-07 08:59:38.196154 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 features compat={},rocompat={},incompat={1=initial feature set (~v.18),3=single paxos with k/v store (v0.?),4=support erasure code pools,5=new-style osdmap encoding,6=support isa/lrc erasure code,7=support shec erasure code} 2016-12-07 08:59:38.196164 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 apply_compatset_features_to_quorum_requirements required_features 9025616074506240 2016-12-07 08:59:38.196167 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 required_features 9025616074506240 2016-12-07 08:59:38.196241 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 has_ever_joined = 1 2016-12-07 08:59:38.196472 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 sync_last_committed_floor 0 2016-12-07 08:59:38.196479 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 init_paxos 2016-12-07 08:59:38.196693 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxos(paxos recovering c 89357..90053) init last_pn: 0 accepted_pn: 20401 last_committed: 90053 first_committed: 89357 2016-12-07 08:59:38.196707 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxos(paxos recovering c 89357..90053) init 2016-12-07 08:59:38.196842 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 refresh_from_paxos 2016-12-07 08:59:38.197075 7f8dfe4f9480 1 mon.ceph3@-1(probing).paxosservice(pgmap 36912..37652) refresh upgraded, format 0 -> 1 2016-12-07 08:59:38.197086 7f8dfe4f9480 1 mon.ceph3@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2016-12-07 08:59:38.197100 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(pgmap 36912..37652) refresh 2016-12-07 08:59:38.197103 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v0 update_from_paxos v0, read_full 2016-12-07 08:59:38.197106 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v0 read_pgmap_meta 2016-12-07 08:59:38.197425 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.0 2016-12-07 08:59:38.197440 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1 2016-12-07 08:59:38.197446 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.10 2016-12-07 08:59:38.197451 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.11 2016-12-07 08:59:38.197455 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.12 2016-12-07 08:59:38.197462 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.13 2016-12-07 08:59:38.197466 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.14 2016-12-07 08:59:38.197471 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.15 2016-12-07 08:59:38.197475 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.16 2016-12-07 08:59:38.197480 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.17 2016-12-07 08:59:38.197489 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.18 2016-12-07 08:59:38.197497 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.19 2016-12-07 08:59:38.197504 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1a 2016-12-07 08:59:38.197513 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1b 2016-12-07 08:59:38.197518 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1c 2016-12-07 08:59:38.197523 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1d 2016-12-07 08:59:38.197527 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1e 2016-12-07 08:59:38.197534 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1f 2016-12-07 08:59:38.197539 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2 2016-12-07 08:59:38.197548 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.20 2016-12-07 08:59:38.197557 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.21 2016-12-07 08:59:38.197562 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.22 2016-12-07 08:59:38.197567 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.23 2016-12-07 08:59:38.197577 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.24 2016-12-07 08:59:38.197584 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.25 2016-12-07 08:59:38.197589 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.26 2016-12-07 08:59:38.197598 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.27 2016-12-07 08:59:38.197606 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.28 2016-12-07 08:59:38.197611 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.29 2016-12-07 08:59:38.197632 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2a 2016-12-07 08:59:38.197639 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2b 2016-12-07 08:59:38.197660 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2c 2016-12-07 08:59:38.197670 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2d 2016-12-07 08:59:38.197682 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2e 2016-12-07 08:59:38.197693 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2f 2016-12-07 08:59:38.197703 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3 2016-12-07 08:59:38.197708 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.30 2016-12-07 08:59:38.197719 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.31 2016-12-07 08:59:38.197728 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.32 2016-12-07 08:59:38.197733 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.33 2016-12-07 08:59:38.197738 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.34 2016-12-07 08:59:38.197747 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.35 2016-12-07 08:59:38.197752 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.36 2016-12-07 08:59:38.197760 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.37 2016-12-07 08:59:38.197765 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.38 2016-12-07 08:59:38.197778 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.39 2016-12-07 08:59:38.197795 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3a 2016-12-07 08:59:38.197800 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3b 2016-12-07 08:59:38.197810 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3c 2016-12-07 08:59:38.197815 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3d 2016-12-07 08:59:38.197820 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3e 2016-12-07 08:59:38.197824 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3f 2016-12-07 08:59:38.197829 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.4 2016-12-07 08:59:38.197836 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.5 2016-12-07 08:59:38.197841 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.6 2016-12-07 08:59:38.197849 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.7 2016-12-07 08:59:38.197858 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.8 2016-12-07 08:59:38.197863 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.9 2016-12-07 08:59:38.197868 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.a 2016-12-07 08:59:38.197872 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.b 2016-12-07 08:59:38.197881 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.c 2016-12-07 08:59:38.197886 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.d 2016-12-07 08:59:38.197893 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.e 2016-12-07 08:59:38.197903 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.f 2016-12-07 08:59:38.197909 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.0 2016-12-07 08:59:38.197914 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1 2016-12-07 08:59:38.197923 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.10 2016-12-07 08:59:38.197928 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.11 2016-12-07 08:59:38.197933 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.12 2016-12-07 08:59:38.197941 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.13 2016-12-07 08:59:38.197948 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.14 2016-12-07 08:59:38.197956 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.15 2016-12-07 08:59:38.197961 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.16 2016-12-07 08:59:38.197966 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.17 2016-12-07 08:59:38.197975 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.18 2016-12-07 08:59:38.197982 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.19 2016-12-07 08:59:38.197987 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1a 2016-12-07 08:59:38.197995 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1b 2016-12-07 08:59:38.198004 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1c 2016-12-07 08:59:38.198009 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1d 2016-12-07 08:59:38.198014 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1e 2016-12-07 08:59:38.198024 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1f 2016-12-07 08:59:38.198032 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2 2016-12-07 08:59:38.198037 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.20 2016-12-07 08:59:38.198048 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.21 2016-12-07 08:59:38.198055 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.22 2016-12-07 08:59:38.198064 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.23 2016-12-07 08:59:38.198069 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.24 2016-12-07 08:59:38.198074 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.25 2016-12-07 08:59:38.198081 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.26 2016-12-07 08:59:38.198086 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.27 2016-12-07 08:59:38.198092 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.28 2016-12-07 08:59:38.198103 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.29 2016-12-07 08:59:38.198108 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2a 2016-12-07 08:59:38.198116 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2b 2016-12-07 08:59:38.198121 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2c 2016-12-07 08:59:38.198144 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2d 2016-12-07 08:59:38.198154 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2e 2016-12-07 08:59:38.198165 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2f 2016-12-07 08:59:38.198172 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3 2016-12-07 08:59:38.198180 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.30 2016-12-07 08:59:38.198185 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.31 2016-12-07 08:59:38.198190 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.32 2016-12-07 08:59:38.198199 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.33 2016-12-07 08:59:38.198204 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.34 2016-12-07 08:59:38.198218 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.35 2016-12-07 08:59:38.198223 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.36 2016-12-07 08:59:38.198232 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.37 2016-12-07 08:59:38.198239 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.38 2016-12-07 08:59:38.198244 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.39 2016-12-07 08:59:38.198255 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3a 2016-12-07 08:59:38.198261 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3b 2016-12-07 08:59:38.198270 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3c 2016-12-07 08:59:38.198277 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3d 2016-12-07 08:59:38.198284 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3e 2016-12-07 08:59:38.198292 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3f 2016-12-07 08:59:38.198298 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4 2016-12-07 08:59:38.198302 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.40 2016-12-07 08:59:38.198311 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.41 2016-12-07 08:59:38.198319 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.42 2016-12-07 08:59:38.198326 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.43 2016-12-07 08:59:38.198336 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.44 2016-12-07 08:59:38.198341 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.45 2016-12-07 08:59:38.198346 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.46 2016-12-07 08:59:38.198355 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.47 2016-12-07 08:59:38.198362 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.48 2016-12-07 08:59:38.198366 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.49 2016-12-07 08:59:38.198374 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4a 2016-12-07 08:59:38.198383 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4b 2016-12-07 08:59:38.198389 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4c 2016-12-07 08:59:38.198394 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4d 2016-12-07 08:59:38.198401 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4e 2016-12-07 08:59:38.198406 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4f 2016-12-07 08:59:38.198415 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5 2016-12-07 08:59:38.198425 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.50 2016-12-07 08:59:38.198435 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.51 2016-12-07 08:59:38.198441 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.52 2016-12-07 08:59:38.198446 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.53 2016-12-07 08:59:38.198456 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.54 2016-12-07 08:59:38.198460 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.55 2016-12-07 08:59:38.198467 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.56 2016-12-07 08:59:38.198477 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.57 2016-12-07 08:59:38.198482 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.58 2016-12-07 08:59:38.198487 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.59 2016-12-07 08:59:38.198500 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5a 2016-12-07 08:59:38.198505 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5b 2016-12-07 08:59:38.198512 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5c 2016-12-07 08:59:38.198524 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5d 2016-12-07 08:59:38.198536 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5e 2016-12-07 08:59:38.198541 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5f 2016-12-07 08:59:38.198549 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6 2016-12-07 08:59:38.198566 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.60 2016-12-07 08:59:38.198575 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.61 2016-12-07 08:59:38.198580 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.62 2016-12-07 08:59:38.198585 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.63 2016-12-07 08:59:38.198595 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.64 2016-12-07 08:59:38.198617 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.65 2016-12-07 08:59:38.198635 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.66 2016-12-07 08:59:38.198641 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.67 2016-12-07 08:59:38.198662 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.68 2016-12-07 08:59:38.198672 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.69 2016-12-07 08:59:38.198682 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6a 2016-12-07 08:59:38.198695 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6b 2016-12-07 08:59:38.198707 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6c 2016-12-07 08:59:38.198713 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6d 2016-12-07 08:59:38.198721 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6e 2016-12-07 08:59:38.198728 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6f 2016-12-07 08:59:38.198747 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7 2016-12-07 08:59:38.198752 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.70 2016-12-07 08:59:38.198757 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.71 2016-12-07 08:59:38.198769 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.72 2016-12-07 08:59:38.198779 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.73 2016-12-07 08:59:38.198784 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.74 2016-12-07 08:59:38.198790 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.75 2016-12-07 08:59:38.198795 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.76 2016-12-07 08:59:38.198799 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.77 2016-12-07 08:59:38.198807 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.78 2016-12-07 08:59:38.198816 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.79 2016-12-07 08:59:38.198821 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7a 2016-12-07 08:59:38.198828 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7b 2016-12-07 08:59:38.198833 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7c 2016-12-07 08:59:38.198840 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7d 2016-12-07 08:59:38.198844 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7e 2016-12-07 08:59:38.198851 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7f 2016-12-07 08:59:38.198861 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.8 2016-12-07 08:59:38.198870 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.9 2016-12-07 08:59:38.198876 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.a 2016-12-07 08:59:38.198881 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.b 2016-12-07 08:59:38.198891 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.c 2016-12-07 08:59:38.198898 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.d 2016-12-07 08:59:38.198904 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.e 2016-12-07 08:59:38.198916 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.f 2016-12-07 08:59:38.198924 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.0 2016-12-07 08:59:38.198934 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1 2016-12-07 08:59:38.198939 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.10 2016-12-07 08:59:38.198947 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.11 2016-12-07 08:59:38.198957 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.12 2016-12-07 08:59:38.198962 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.13 2016-12-07 08:59:38.198976 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.14 2016-12-07 08:59:38.198986 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.15 2016-12-07 08:59:38.198991 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.16 2016-12-07 08:59:38.198996 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.17 2016-12-07 08:59:38.199007 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.18 2016-12-07 08:59:38.199012 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.19 2016-12-07 08:59:38.199017 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1a 2016-12-07 08:59:38.199025 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1b 2016-12-07 08:59:38.199040 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1c 2016-12-07 08:59:38.199050 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1d 2016-12-07 08:59:38.199058 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1e 2016-12-07 08:59:38.199066 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1f 2016-12-07 08:59:38.199071 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2 2016-12-07 08:59:38.199079 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.20 2016-12-07 08:59:38.199085 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.21 2016-12-07 08:59:38.199098 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.22 2016-12-07 08:59:38.199104 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.23 2016-12-07 08:59:38.199111 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.24 2016-12-07 08:59:38.199117 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.25 2016-12-07 08:59:38.199128 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.26 2016-12-07 08:59:38.199133 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.27 2016-12-07 08:59:38.199138 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.28 2016-12-07 08:59:38.199149 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.29 2016-12-07 08:59:38.199157 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2a 2016-12-07 08:59:38.199162 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2b 2016-12-07 08:59:38.199168 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2c 2016-12-07 08:59:38.199175 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2d 2016-12-07 08:59:38.199182 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2e 2016-12-07 08:59:38.199190 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2f 2016-12-07 08:59:38.199198 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3 2016-12-07 08:59:38.199207 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.30 2016-12-07 08:59:38.199213 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.31 2016-12-07 08:59:38.199218 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.32 2016-12-07 08:59:38.199226 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.33 2016-12-07 08:59:38.199236 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.34 2016-12-07 08:59:38.199243 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.35 2016-12-07 08:59:38.199250 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.36 2016-12-07 08:59:38.199259 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.37 2016-12-07 08:59:38.199264 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.38 2016-12-07 08:59:38.199271 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.39 2016-12-07 08:59:38.199280 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3a 2016-12-07 08:59:38.199288 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3b 2016-12-07 08:59:38.199293 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3c 2016-12-07 08:59:38.199303 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3d 2016-12-07 08:59:38.199311 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3e 2016-12-07 08:59:38.199319 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3f 2016-12-07 08:59:38.199325 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4 2016-12-07 08:59:38.199335 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.40 2016-12-07 08:59:38.199341 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.41 2016-12-07 08:59:38.199346 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.42 2016-12-07 08:59:38.199356 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.43 2016-12-07 08:59:38.199363 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.44 2016-12-07 08:59:38.199371 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.45 2016-12-07 08:59:38.199377 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.46 2016-12-07 08:59:38.199386 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.47 2016-12-07 08:59:38.199392 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.48 2016-12-07 08:59:38.199397 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.49 2016-12-07 08:59:38.199407 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4a 2016-12-07 08:59:38.199413 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4b 2016-12-07 08:59:38.199419 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4c 2016-12-07 08:59:38.199430 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4d 2016-12-07 08:59:38.199435 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4e 2016-12-07 08:59:38.199442 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4f 2016-12-07 08:59:38.199452 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5 2016-12-07 08:59:38.199461 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.50 2016-12-07 08:59:38.199472 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.51 2016-12-07 08:59:38.199479 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.52 2016-12-07 08:59:38.199486 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.53 2016-12-07 08:59:38.199491 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.54 2016-12-07 08:59:38.199502 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.55 2016-12-07 08:59:38.199514 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.56 2016-12-07 08:59:38.199520 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.57 2016-12-07 08:59:38.199527 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.58 2016-12-07 08:59:38.199535 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.59 2016-12-07 08:59:38.199541 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5a 2016-12-07 08:59:38.199548 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5b 2016-12-07 08:59:38.199553 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5c 2016-12-07 08:59:38.199562 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5d 2016-12-07 08:59:38.199567 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5e 2016-12-07 08:59:38.199579 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5f 2016-12-07 08:59:38.199585 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6 2016-12-07 08:59:38.199605 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.60 2016-12-07 08:59:38.199626 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.61 2016-12-07 08:59:38.199631 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.62 2016-12-07 08:59:38.199641 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.63 2016-12-07 08:59:38.199647 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.64 2016-12-07 08:59:38.199655 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.65 2016-12-07 08:59:38.199664 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.66 2016-12-07 08:59:38.199671 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.67 2016-12-07 08:59:38.199677 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.68 2016-12-07 08:59:38.199687 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.69 2016-12-07 08:59:38.199692 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6a 2016-12-07 08:59:38.199698 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6b 2016-12-07 08:59:38.199812 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6c 2016-12-07 08:59:38.199827 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6d 2016-12-07 08:59:38.199833 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6e 2016-12-07 08:59:38.199839 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6f 2016-12-07 08:59:38.199845 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7 2016-12-07 08:59:38.199854 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.70 2016-12-07 08:59:38.199859 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.71 2016-12-07 08:59:38.199868 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.72 2016-12-07 08:59:38.199874 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.73 2016-12-07 08:59:38.199881 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.74 2016-12-07 08:59:38.199888 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.75 2016-12-07 08:59:38.199895 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.76 2016-12-07 08:59:38.199900 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.77 2016-12-07 08:59:38.199905 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.78 2016-12-07 08:59:38.199914 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.79 2016-12-07 08:59:38.199926 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7a 2016-12-07 08:59:38.199932 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7b 2016-12-07 08:59:38.199937 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7c 2016-12-07 08:59:38.199944 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7d 2016-12-07 08:59:38.199949 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7e 2016-12-07 08:59:38.199957 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7f 2016-12-07 08:59:38.199964 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.8 2016-12-07 08:59:38.199970 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.9 2016-12-07 08:59:38.199978 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.a 2016-12-07 08:59:38.199983 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.b 2016-12-07 08:59:38.199990 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.c 2016-12-07 08:59:38.199998 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.d 2016-12-07 08:59:38.200003 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.e 2016-12-07 08:59:38.200008 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.f 2016-12-07 08:59:38.200111 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.0 2016-12-07 08:59:38.200123 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.1 2016-12-07 08:59:38.200128 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.2 2016-12-07 08:59:38.200132 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.3 2016-12-07 08:59:38.200136 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.4 2016-12-07 08:59:38.200140 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.5 2016-12-07 08:59:38.200161 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v37652 update_logger 2016-12-07 08:59:38.200401 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(mdsmap 1..438) refresh 2016-12-07 08:59:38.200407 7f8dfe4f9480 10 mon.ceph3@-1(probing).mds e0 update_from_paxos version 438, my e 0 2016-12-07 08:59:38.200504 7f8dfe4f9480 10 mon.ceph3@-1(probing).mds e0 update_from_paxos got 438 2016-12-07 08:59:38.200558 7f8dfe4f9480 4 mon.ceph3@-1(probing).mds e438 new map 2016-12-07 08:59:38.200564 7f8dfe4f9480 0 mon.ceph3@-1(probing).mds e438 print_map e438 enable_multiple, ever_enabled_multiple: 0,0 compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=file layout v2} Filesystem 'first_cephfs' (1) fs_name first_cephfs epoch 436 flags 8 created 2016-11-16 06:06:34.376616 modified 2016-11-23 11:37:26.768424 tableserver 0 root 0 session_timeout 60 session_autoclose 300 max_file_size 1099511627776 last_failure 0 last_failure_osd_epoch 68 compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=file layout v2} max_mds 1 in 0 up {0=58523} failed damaged stopped data_pools 1 metadata_pool 2 inline_data disabled 58523: 10.10.38.40:6806/13750 'ceph1' mds.0.433 up:active seq 70 Standby daemons: 39073: 10.10.38.41:6805/12990 'ceph2' mds.-1.0 up:standby seq 1 (standby for rank 0) 2016-12-07 08:59:38.200615 7f8dfe4f9480 10 mon.ceph3@-1(probing).mds e438 update_logger 2016-12-07 08:59:38.200799 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(osdmap 1..71) refresh 2016-12-07 08:59:38.200809 7f8dfe4f9480 15 mon.ceph3@-1(probing).osd e0 update_from_paxos paxos e 71, my e 0 2016-12-07 08:59:38.200905 7f8dfe4f9480 7 mon.ceph3@-1(probing).osd e0 update_from_paxos loading latest full map e71 2016-12-07 08:59:38.200975 7f8dfe4f9480 10 mon.ceph3@-1(probing).osd e71 check_subs 2016-12-07 08:59:38.200983 7f8dfe4f9480 10 mon.ceph3@-1(probing).osd e71 share_map_with_random_osd no up osd on our session map 2016-12-07 08:59:38.200985 7f8dfe4f9480 10 mon.ceph3@-1(probing).osd e71 update_logger 2016-12-07 08:59:38.200992 7f8dfe4f9480 0 mon.ceph3@-1(probing).osd e71 crush map has features 2200130813952, adjusting msgr requires 2016-12-07 08:59:38.200994 7f8dfe4f9480 0 mon.ceph3@-1(probing).osd e71 crush map has features 2200130813952, adjusting msgr requires 2016-12-07 08:59:38.200997 7f8dfe4f9480 0 mon.ceph3@-1(probing).osd e71 crush map has features 2200130813952, adjusting msgr requires 2016-12-07 08:59:38.200998 7f8dfe4f9480 0 mon.ceph3@-1(probing).osd e71 crush map has features 2200130813952, adjusting msgr requires 2016-12-07 08:59:38.201265 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(logm 50434..51109) refresh 2016-12-07 08:59:38.201274 7f8dfe4f9480 10 mon.ceph3@-1(probing).log v51109 update_from_paxos 2016-12-07 08:59:38.201277 7f8dfe4f9480 10 mon.ceph3@-1(probing).log v51109 update_from_paxos version 51109 summary v 0 2016-12-07 08:59:38.201361 7f8dfe4f9480 10 mon.ceph3@-1(probing).log v51109 update_from_paxos latest full 51108 2016-12-07 08:59:38.201457 7f8dfe4f9480 7 mon.ceph3@-1(probing).log v51109 update_from_paxos loading summary e51108 2016-12-07 08:59:38.201489 7f8dfe4f9480 7 mon.ceph3@-1(probing).log v51109 update_from_paxos loaded summary e51108 2016-12-07 08:59:38.201602 7f8dfe4f9480 7 mon.ceph3@-1(probing).log v51109 update_from_paxos applying incremental log 51109 2016-11-25 09:22:08.515897 mon.0 10.10.38.40:6789/0 44591 : audit [DBG] from='client.? 10.10.38.40:0/717519612' entity='client.admin' cmd=[{"prefix": "df"}]: dispatch 2016-12-07 08:59:38.201624 7f8dfe4f9480 20 mon.ceph3@-1(probing).log v51109 update_from_paxos logging for channel 'audit' to file '/var/log/ceph/ceph.audit.log' 2016-12-07 08:59:38.201640 7f8dfe4f9480 15 mon.ceph3@-1(probing).log v51109 update_from_paxos logging for 0 channels 2016-12-07 08:59:38.201645 7f8dfe4f9480 10 mon.ceph3@-1(probing).log v51109 check_subs 2016-12-07 08:59:38.201888 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(monmap 1..3) refresh 2016-12-07 08:59:38.202132 7f8dfe4f9480 1 mon.ceph3@-1(probing).paxosservice(auth 1..264) refresh upgraded, format 0 -> 1 2016-12-07 08:59:38.202137 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(auth 1..264) refresh 2016-12-07 08:59:38.202140 7f8dfe4f9480 10 mon.ceph3@-1(probing).auth v264 update_from_paxos 2016-12-07 08:59:38.202219 7f8dfe4f9480 10 mon.ceph3@-1(probing).auth v264 update_from_paxos version 264 keys ver 0 latest 261 2016-12-07 08:59:38.202306 7f8dfe4f9480 7 mon.ceph3@-1(probing).auth v264 update_from_paxos loading summary e 261 2016-12-07 08:59:38.202311 7f8dfe4f9480 7 mon.ceph3@-1(probing).auth v264 update_from_paxos latest length 1958 2016-12-07 08:59:38.202609 7f8dfe4f9480 10 mon.ceph3@-1(probing).auth v264 update_from_paxos key server version 261 2016-12-07 08:59:38.202793 7f8dfe4f9480 20 mon.ceph3@-1(probing).auth v264 update_from_paxos walking through version 262 len 649 2016-12-07 08:59:38.202977 7f8dfe4f9480 20 mon.ceph3@-1(probing).auth v264 update_from_paxos walking through version 263 len 649 2016-12-07 08:59:38.203153 7f8dfe4f9480 20 mon.ceph3@-1(probing).auth v264 update_from_paxos walking through version 264 len 649 2016-12-07 08:59:38.203257 7f8dfe4f9480 10 mon.ceph3@-1(probing).auth v264 update_from_paxos() last_allocated_id=104096 max_global_id=104096 format_version 1 2016-12-07 08:59:38.203263 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(pgmap 36912..37652) post_refresh 2016-12-07 08:59:38.203267 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v37652 post_paxos_update 2016-12-07 08:59:38.203281 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v37652 check_subs 2016-12-07 08:59:38.203283 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(mdsmap 1..438) post_refresh 2016-12-07 08:59:38.203284 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(osdmap 1..71) post_refresh 2016-12-07 08:59:38.203285 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(logm 50434..51109) post_refresh 2016-12-07 08:59:38.203285 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(monmap 1..3) post_refresh 2016-12-07 08:59:38.203286 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(auth 1..264) post_refresh 2016-12-07 08:59:38.203289 7f8dfe4f9480 10 mon.ceph3@-1(probing).health(0) init 2016-12-07 08:59:38.203598 7f8dfe4f9480 2 mon.ceph3@-1(probing) e3 init 2016-12-07 08:59:38.204222 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 bootstrap 2016-12-07 08:59:38.204230 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 sync_reset_requester 2016-12-07 08:59:38.204231 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 unregister_cluster_logger - not registered 2016-12-07 08:59:38.204232 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 cancel_probe_timeout (none scheduled) 2016-12-07 08:59:38.204233 7f8dfe4f9480 0 mon.ceph3@-1(probing) e3 my rank is now 0 (was -1) 2016-12-07 08:59:38.204240 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 _reset 2016-12-07 08:59:38.204241 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 cancel_probe_timeout (none scheduled) 2016-12-07 08:59:38.204242 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 timecheck_finish 2016-12-07 08:59:38.204244 7f8dfe4f9480 15 mon.ceph3@0(probing) e3 health_tick_stop 2016-12-07 08:59:38.204245 7f8dfe4f9480 15 mon.ceph3@0(probing) e3 health_interval_stop 2016-12-07 08:59:38.204247 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 scrub_event_cancel 2016-12-07 08:59:38.204248 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 scrub_reset 2016-12-07 08:59:38.204250 7f8dfe4f9480 10 mon.ceph3@0(probing).paxos(paxos recovering c 89357..90053) restart -- canceling timeouts 2016-12-07 08:59:38.204255 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(pgmap 36912..37652) restart 2016-12-07 08:59:38.204258 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(mdsmap 1..438) restart 2016-12-07 08:59:38.204259 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(osdmap 1..71) restart 2016-12-07 08:59:38.204260 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(logm 50434..51109) restart 2016-12-07 08:59:38.204261 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(monmap 1..3) restart 2016-12-07 08:59:38.204262 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(auth 1..264) restart 2016-12-07 08:59:38.204264 7f8dfe4f9480 1 mon.ceph3@0(probing) e3 win_standalone_election 2016-12-07 08:59:38.204376 7f8dfe4f9480 1 mon.ceph3@0(probing).elector(488) init, last seen epoch 488 2016-12-07 08:59:38.204383 7f8dfe4f9480 10 mon.ceph3@0(probing).elector(488) bump_epoch 488 to 489 2016-12-07 08:59:38.221449 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 join_election 2016-12-07 08:59:38.221468 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 _reset 2016-12-07 08:59:38.221470 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 cancel_probe_timeout (none scheduled) 2016-12-07 08:59:38.221472 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 timecheck_finish 2016-12-07 08:59:38.221474 7f8dfe4f9480 15 mon.ceph3@0(probing) e3 health_tick_stop 2016-12-07 08:59:38.221476 7f8dfe4f9480 15 mon.ceph3@0(probing) e3 health_interval_stop 2016-12-07 08:59:38.221477 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 scrub_event_cancel 2016-12-07 08:59:38.221479 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 scrub_reset 2016-12-07 08:59:38.221481 7f8dfe4f9480 10 mon.ceph3@0(probing).paxos(paxos recovering c 89357..90053) restart -- canceling timeouts 2016-12-07 08:59:38.221487 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(pgmap 36912..37652) restart 2016-12-07 08:59:38.221490 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(mdsmap 1..438) restart 2016-12-07 08:59:38.221492 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(osdmap 1..71) restart 2016-12-07 08:59:38.221494 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(logm 50434..51109) restart 2016-12-07 08:59:38.221496 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(monmap 1..3) restart 2016-12-07 08:59:38.221497 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(auth 1..264) restart 2016-12-07 08:59:38.221504 7f8dfe4f9480 10 mon.ceph3@0(electing) e3 win_election epoch 489 quorum 0 features 576460752032874495 2016-12-07 08:59:38.221526 7f8dfe4f9480 0 log_channel(cluster) log [INF] : mon.ceph3@0 won leader election with quorum 0 2016-12-07 08:59:38.221529 7f8dfe4f9480 10 log_client _send_to_monlog to self 2016-12-07 08:59:38.221530 7f8dfe4f9480 10 log_client log_queue is 1 last_log 1 sent 0 num 1 unsent 1 sending 1 2016-12-07 08:59:38.221533 7f8dfe4f9480 10 log_client will send 2016-12-07 08:59:38.221527 mon.0 10.10.38.42:6789/0 1 : cluster [INF] mon.ceph3@0 won leader election with quorum 0 2016-12-07 08:59:38.221562 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(monmap 1..3) election_finished 2016-12-07 08:59:38.221564 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(monmap 1..3) _active 2016-12-07 08:59:38.221565 7f8dfe4f9480 7 mon.ceph3@0(leader).paxosservice(monmap 1..3) _active creating new pending 2016-12-07 08:59:38.221570 7f8dfe4f9480 10 mon.ceph3@0(leader).monmap v3 create_pending monmap epoch 4 2016-12-07 08:59:38.221589 7f8dfe4f9480 0 log_channel(cluster) log [INF] : monmap e3: 1 mons at {ceph3=10.10.38.42:6789/0} 2016-12-07 08:59:38.221594 7f8dfe4f9480 10 log_client _send_to_monlog to self 2016-12-07 08:59:38.221595 7f8dfe4f9480 10 log_client log_queue is 2 last_log 2 sent 1 num 2 unsent 1 sending 1 2016-12-07 08:59:38.221597 7f8dfe4f9480 10 log_client will send 2016-12-07 08:59:38.221593 mon.0 10.10.38.42:6789/0 2 : cluster [INF] monmap e3: 1 mons at {ceph3=10.10.38.42:6789/0} 2016-12-07 08:59:38.221640 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(pgmap 36912..37652) election_finished 2016-12-07 08:59:38.221645 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(pgmap 36912..37652) _active 2016-12-07 08:59:38.221647 7f8dfe4f9480 7 mon.ceph3@0(leader).paxosservice(pgmap 36912..37652) _active creating new pending 2016-12-07 08:59:38.221652 7f8dfe4f9480 10 mon.ceph3@0(leader).pg v37652 create_pending v 37653 2016-12-07 08:59:38.221657 7f8dfe4f9480 10 mon.ceph3@0(leader).pg v37652 check_osd_map already seen 71 >= 71 2016-12-07 08:59:38.221659 7f8dfe4f9480 10 mon.ceph3@0(leader).pg v37652 update_logger 2016-12-07 08:59:38.221677 7f8dfe4f9480 0 log_channel(cluster) log [INF] : pgmap v37652: 320 pgs: 320 active+clean; 35394 kB data, 1171 MB used, 214 GB / 215 GB avail 2016-12-07 08:59:38.221682 7f8dfe4f9480 10 log_client _send_to_monlog to self 2016-12-07 08:59:38.221684 7f8dfe4f9480 10 log_client log_queue is 3 last_log 3 sent 2 num 3 unsent 1 sending 1 2016-12-07 08:59:38.221686 7f8dfe4f9480 10 log_client will send 2016-12-07 08:59:38.221682 mon.0 10.10.38.42:6789/0 3 : cluster [INF] pgmap v37652: 320 pgs: 320 active+clean; 35394 kB data, 1171 MB used, 214 GB / 215 GB avail 2016-12-07 08:59:38.221696 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(mdsmap 1..438) election_finished 2016-12-07 08:59:38.221697 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(mdsmap 1..438) _active 2016-12-07 08:59:38.221698 7f8dfe4f9480 7 mon.ceph3@0(leader).paxosservice(mdsmap 1..438) _active creating new pending 2016-12-07 08:59:38.221715 7f8dfe4f9480 10 mon.ceph3@0(leader).mds e438 create_pending e439 2016-12-07 08:59:38.221719 7f8dfe4f9480 10 mon.ceph3@0(leader).mds e438 e438: 1/1/1 up {0=ceph1=up:active}, 1 up:standby 2016-12-07 08:59:38.221743 7f8dfe4f9480 20 mon.ceph3@0(leader).mds e438 gid 39073 is standby and following nobody 2016-12-07 08:59:38.221750 7f8dfe4f9480 10 mon.ceph3@0(leader).mds e438 setting to follow mds rank 0 2016-12-07 08:59:38.224093 7f8dfe4f9480 -1 mon/MDSMonitor.cc: In function 'bool MDSMonitor::maybe_promote_standby(std::shared_ptr)' thread 7f8dfe4f9480 time 2016-12-07 08:59:38.221755 mon/MDSMonitor.cc: 2797: FAILED assert(info.state == MDSMap::STATE_STANDBY) ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7f8dfe0389fb] 2: (MDSMonitor::maybe_promote_standby(std::shared_ptr)+0x976) [0x7f8dfdd79016] 3: (MDSMonitor::tick()+0x397) [0x7f8dfdd7cfa7] 4: (MDSMonitor::on_active()+0x15) [0x7f8dfdd730a5] 5: (PaxosService::_active()+0x1df) [0x7f8dfdcff2ef] 6: (PaxosService::election_finished()+0x67) [0x7f8dfdcffa47] 7: (Monitor::win_election(unsigned int, std::set, std::allocator >&, unsigned long, MonCommand const*, int, std::set, std::allocator > const*)+0x236) [0x7f8dfdcc1336] 8: (Monitor::win_standalone_election()+0x158) [0x7f8dfdcc1738] 9: (Monitor::bootstrap()+0xa03) [0x7f8dfdcc2283] 10: (Monitor::init()+0x190) [0x7f8dfdcc2590] 11: (main()+0x24ca) [0x7f8dfdc38a4a] 12: (__libc_start_main()+0xf5) [0x7f8dfb1fcec5] 13: (()+0x2609ca) [0x7f8dfdc8a9ca] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -517> 2016-12-07 08:59:38.159620 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command perfcounters_dump hook 0x7f8e08c7d710 -516> 2016-12-07 08:59:38.159646 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command 1 hook 0x7f8e08c7d710 -515> 2016-12-07 08:59:38.159650 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command perf dump hook 0x7f8e08c7d710 -514> 2016-12-07 08:59:38.159654 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command perfcounters_schema hook 0x7f8e08c7d710 -513> 2016-12-07 08:59:38.159657 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command 2 hook 0x7f8e08c7d710 -512> 2016-12-07 08:59:38.159664 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command perf schema hook 0x7f8e08c7d710 -511> 2016-12-07 08:59:38.159667 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command perf reset hook 0x7f8e08c7d710 -510> 2016-12-07 08:59:38.159671 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command config show hook 0x7f8e08c7d710 -509> 2016-12-07 08:59:38.159676 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command config set hook 0x7f8e08c7d710 -508> 2016-12-07 08:59:38.159678 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command config get hook 0x7f8e08c7d710 -507> 2016-12-07 08:59:38.159679 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command config diff hook 0x7f8e08c7d710 -506> 2016-12-07 08:59:38.159683 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command log flush hook 0x7f8e08c7d710 -505> 2016-12-07 08:59:38.159685 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command log dump hook 0x7f8e08c7d710 -504> 2016-12-07 08:59:38.159689 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command log reopen hook 0x7f8e08c7d710 -503> 2016-12-07 08:59:38.164863 7f8dfe4f9480 0 set uid:gid to 64045:64045 (ceph:ceph) -502> 2016-12-07 08:59:38.164885 7f8dfe4f9480 0 ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b), process ceph-mon, pid 3695 -501> 2016-12-07 08:59:38.164955 7f8dfe4f9480 0 pidfile_write: ignore empty --pid-file -500> 2016-12-07 08:59:38.166333 7f8dfe4f9480 5 asok(0x7f8e08c7a480) init /var/run/ceph/ceph-mon.ceph3.asok -499> 2016-12-07 08:59:38.166343 7f8dfe4f9480 5 asok(0x7f8e08c7a480) bind_and_listen /var/run/ceph/ceph-mon.ceph3.asok -498> 2016-12-07 08:59:38.166359 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command 0 hook 0x7f8e08cd2430 -497> 2016-12-07 08:59:38.166363 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command version hook 0x7f8e08cd2430 -496> 2016-12-07 08:59:38.166366 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command git_version hook 0x7f8e08cd2430 -495> 2016-12-07 08:59:38.166372 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command help hook 0x7f8e08c87230 -494> 2016-12-07 08:59:38.166375 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command get_command_descriptions hook 0x7f8e08cd3270 -493> 2016-12-07 08:59:38.166436 7f8df89db700 5 asok(0x7f8e08c7a480) entry start -492> 2016-12-07 08:59:38.173417 7f8dfe4f9480 10 load: jerasure load: lrc load: isa -491> 2016-12-07 08:59:38.173757 7f8dfe4f9480 1 leveldb: Recovering log #8837 -490> 2016-12-07 08:59:38.173794 7f8dfe4f9480 1 leveldb: Level-0 table #8839: started -489> 2016-12-07 08:59:38.179928 7f8dfe4f9480 1 leveldb: Level-0 table #8839: 142 bytes OK -488> 2016-12-07 08:59:38.193276 7f8dfe4f9480 1 leveldb: Delete type=2 #8838 -487> 2016-12-07 08:59:38.193681 7f8dfe4f9480 1 leveldb: Delete type=3 #8835 -486> 2016-12-07 08:59:38.193713 7f8dfe4f9480 1 leveldb: Delete type=0 #8837 -485> 2016-12-07 08:59:38.194394 7f8df349c700 1 leveldb: Compacting 68@0 + 0@1 files -484> 2016-12-07 08:59:38.194626 7f8dfe4f9480 10 obtain_monmap -483> 2016-12-07 08:59:38.194977 7f8dfe4f9480 10 obtain_monmap read last committed monmap ver 3 -482> 2016-12-07 08:59:38.195031 7f8dfe4f9480 0 starting mon.ceph3 rank 0 at 10.10.38.42:6789/0 mon_data /var/lib/ceph/mon/ceph-ceph3 fsid 7fddfc10-5e7d-44c6-ac0b-fdd129e4219b -481> 2016-12-07 08:59:38.195077 7f8dfe4f9480 1 -- 10.10.38.42:6789/0 learned my addr 10.10.38.42:6789/0 -480> 2016-12-07 08:59:38.195084 7f8dfe4f9480 1 accepter.accepter.bind my_inst.addr is 10.10.38.42:6789/0 need_addr=0 -479> 2016-12-07 08:59:38.195780 7f8dfe4f9480 5 adding auth protocol: cephx -478> 2016-12-07 08:59:38.195786 7f8dfe4f9480 5 adding auth protocol: cephx -477> 2016-12-07 08:59:38.195807 7f8dfe4f9480 20 log_channel(cluster) update_config log_to_monitors {default=true} log_to_syslog {default=false} log_channels {audit=local0,default=daemon} log_prios {default=info} -476> 2016-12-07 08:59:38.195816 7f8dfe4f9480 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) -475> 2016-12-07 08:59:38.195819 7f8dfe4f9480 20 log_channel(audit) update_config log_to_monitors {default=true} log_to_syslog {default=false} log_channels {audit=local0,default=daemon} log_prios {default=info} -474> 2016-12-07 08:59:38.195821 7f8dfe4f9480 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) -473> 2016-12-07 08:59:38.195923 7f8dfe4f9480 1 mon.ceph3@-1(probing) e3 preinit fsid 7fddfc10-5e7d-44c6-ac0b-fdd129e4219b -472> 2016-12-07 08:59:38.196067 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 check_fsid cluster_uuid contains '7fddfc10-5e7d-44c6-ac0b-fdd129e4219b' -471> 2016-12-07 08:59:38.196154 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 features compat={},rocompat={},incompat={1=initial feature set (~v.18),3=single paxos with k/v store (v0.?),4=support erasure code pools,5=new-style osdmap encoding,6=support isa/lrc erasure code,7=support shec erasure code} -470> 2016-12-07 08:59:38.196164 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 apply_compatset_features_to_quorum_requirements required_features 9025616074506240 -469> 2016-12-07 08:59:38.196167 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 required_features 9025616074506240 -468> 2016-12-07 08:59:38.196241 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 has_ever_joined = 1 -467> 2016-12-07 08:59:38.196472 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 sync_last_committed_floor 0 -466> 2016-12-07 08:59:38.196479 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 init_paxos -465> 2016-12-07 08:59:38.196693 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxos(paxos recovering c 89357..90053) init last_pn: 0 accepted_pn: 20401 last_committed: 90053 first_committed: 89357 -464> 2016-12-07 08:59:38.196707 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxos(paxos recovering c 89357..90053) init -463> 2016-12-07 08:59:38.196842 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 refresh_from_paxos -462> 2016-12-07 08:59:38.197075 7f8dfe4f9480 1 mon.ceph3@-1(probing).paxosservice(pgmap 36912..37652) refresh upgraded, format 0 -> 1 -461> 2016-12-07 08:59:38.197086 7f8dfe4f9480 1 mon.ceph3@-1(probing).pg v0 on_upgrade discarding in-core PGMap -460> 2016-12-07 08:59:38.197100 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(pgmap 36912..37652) refresh -459> 2016-12-07 08:59:38.197103 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v0 update_from_paxos v0, read_full -458> 2016-12-07 08:59:38.197106 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v0 read_pgmap_meta -457> 2016-12-07 08:59:38.197425 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.0 -456> 2016-12-07 08:59:38.197440 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1 -455> 2016-12-07 08:59:38.197446 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.10 -454> 2016-12-07 08:59:38.197451 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.11 -453> 2016-12-07 08:59:38.197455 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.12 -452> 2016-12-07 08:59:38.197462 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.13 -451> 2016-12-07 08:59:38.197466 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.14 -450> 2016-12-07 08:59:38.197471 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.15 -449> 2016-12-07 08:59:38.197475 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.16 -448> 2016-12-07 08:59:38.197480 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.17 -447> 2016-12-07 08:59:38.197489 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.18 -446> 2016-12-07 08:59:38.197497 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.19 -445> 2016-12-07 08:59:38.197504 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1a -444> 2016-12-07 08:59:38.197513 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1b -443> 2016-12-07 08:59:38.197518 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1c -442> 2016-12-07 08:59:38.197523 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1d -441> 2016-12-07 08:59:38.197527 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1e -440> 2016-12-07 08:59:38.197534 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.1f -439> 2016-12-07 08:59:38.197539 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2 -438> 2016-12-07 08:59:38.197548 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.20 -437> 2016-12-07 08:59:38.197557 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.21 -436> 2016-12-07 08:59:38.197562 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.22 -435> 2016-12-07 08:59:38.197567 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.23 -434> 2016-12-07 08:59:38.197577 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.24 -433> 2016-12-07 08:59:38.197584 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.25 -432> 2016-12-07 08:59:38.197589 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.26 -431> 2016-12-07 08:59:38.197598 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.27 -430> 2016-12-07 08:59:38.197606 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.28 -429> 2016-12-07 08:59:38.197611 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.29 -428> 2016-12-07 08:59:38.197632 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2a -427> 2016-12-07 08:59:38.197639 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2b -426> 2016-12-07 08:59:38.197660 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2c -425> 2016-12-07 08:59:38.197670 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2d -424> 2016-12-07 08:59:38.197682 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2e -423> 2016-12-07 08:59:38.197693 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.2f -422> 2016-12-07 08:59:38.197703 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3 -421> 2016-12-07 08:59:38.197708 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.30 -420> 2016-12-07 08:59:38.197719 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.31 -419> 2016-12-07 08:59:38.197728 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.32 -418> 2016-12-07 08:59:38.197733 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.33 -417> 2016-12-07 08:59:38.197738 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.34 -416> 2016-12-07 08:59:38.197747 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.35 -415> 2016-12-07 08:59:38.197752 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.36 -414> 2016-12-07 08:59:38.197760 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.37 -413> 2016-12-07 08:59:38.197765 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.38 -412> 2016-12-07 08:59:38.197778 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.39 -411> 2016-12-07 08:59:38.197795 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3a -410> 2016-12-07 08:59:38.197800 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3b -409> 2016-12-07 08:59:38.197810 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3c -408> 2016-12-07 08:59:38.197815 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3d -407> 2016-12-07 08:59:38.197820 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3e -406> 2016-12-07 08:59:38.197824 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.3f -405> 2016-12-07 08:59:38.197829 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.4 -404> 2016-12-07 08:59:38.197836 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.5 -403> 2016-12-07 08:59:38.197841 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.6 -402> 2016-12-07 08:59:38.197849 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.7 -401> 2016-12-07 08:59:38.197858 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.8 -400> 2016-12-07 08:59:38.197863 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.9 -399> 2016-12-07 08:59:38.197868 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.a -398> 2016-12-07 08:59:38.197872 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.b -397> 2016-12-07 08:59:38.197881 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.c -396> 2016-12-07 08:59:38.197886 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.d -395> 2016-12-07 08:59:38.197893 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.e -394> 2016-12-07 08:59:38.197903 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 0.f -393> 2016-12-07 08:59:38.197909 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.0 -392> 2016-12-07 08:59:38.197914 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1 -391> 2016-12-07 08:59:38.197923 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.10 -390> 2016-12-07 08:59:38.197928 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.11 -389> 2016-12-07 08:59:38.197933 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.12 -388> 2016-12-07 08:59:38.197941 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.13 -387> 2016-12-07 08:59:38.197948 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.14 -386> 2016-12-07 08:59:38.197956 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.15 -385> 2016-12-07 08:59:38.197961 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.16 -384> 2016-12-07 08:59:38.197966 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.17 -383> 2016-12-07 08:59:38.197975 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.18 -382> 2016-12-07 08:59:38.197982 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.19 -381> 2016-12-07 08:59:38.197987 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1a -380> 2016-12-07 08:59:38.197995 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1b -379> 2016-12-07 08:59:38.198004 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1c -378> 2016-12-07 08:59:38.198009 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1d -377> 2016-12-07 08:59:38.198014 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1e -376> 2016-12-07 08:59:38.198024 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.1f -375> 2016-12-07 08:59:38.198032 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2 -374> 2016-12-07 08:59:38.198037 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.20 -373> 2016-12-07 08:59:38.198048 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.21 -372> 2016-12-07 08:59:38.198055 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.22 -371> 2016-12-07 08:59:38.198064 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.23 -370> 2016-12-07 08:59:38.198069 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.24 -369> 2016-12-07 08:59:38.198074 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.25 -368> 2016-12-07 08:59:38.198081 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.26 -367> 2016-12-07 08:59:38.198086 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.27 -366> 2016-12-07 08:59:38.198092 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.28 -365> 2016-12-07 08:59:38.198103 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.29 -364> 2016-12-07 08:59:38.198108 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2a -363> 2016-12-07 08:59:38.198116 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2b -362> 2016-12-07 08:59:38.198121 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2c -361> 2016-12-07 08:59:38.198144 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2d -360> 2016-12-07 08:59:38.198154 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2e -359> 2016-12-07 08:59:38.198165 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.2f -358> 2016-12-07 08:59:38.198172 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3 -357> 2016-12-07 08:59:38.198180 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.30 -356> 2016-12-07 08:59:38.198185 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.31 -355> 2016-12-07 08:59:38.198190 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.32 -354> 2016-12-07 08:59:38.198199 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.33 -353> 2016-12-07 08:59:38.198204 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.34 -352> 2016-12-07 08:59:38.198218 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.35 -351> 2016-12-07 08:59:38.198223 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.36 -350> 2016-12-07 08:59:38.198232 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.37 -349> 2016-12-07 08:59:38.198239 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.38 -348> 2016-12-07 08:59:38.198244 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.39 -347> 2016-12-07 08:59:38.198255 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3a -346> 2016-12-07 08:59:38.198261 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3b -345> 2016-12-07 08:59:38.198270 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3c -344> 2016-12-07 08:59:38.198277 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3d -343> 2016-12-07 08:59:38.198284 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3e -342> 2016-12-07 08:59:38.198292 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.3f -341> 2016-12-07 08:59:38.198298 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4 -340> 2016-12-07 08:59:38.198302 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.40 -339> 2016-12-07 08:59:38.198311 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.41 -338> 2016-12-07 08:59:38.198319 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.42 -337> 2016-12-07 08:59:38.198326 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.43 -336> 2016-12-07 08:59:38.198336 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.44 -335> 2016-12-07 08:59:38.198341 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.45 -334> 2016-12-07 08:59:38.198346 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.46 -333> 2016-12-07 08:59:38.198355 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.47 -332> 2016-12-07 08:59:38.198362 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.48 -331> 2016-12-07 08:59:38.198366 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.49 -330> 2016-12-07 08:59:38.198374 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4a -329> 2016-12-07 08:59:38.198383 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4b -328> 2016-12-07 08:59:38.198389 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4c -327> 2016-12-07 08:59:38.198394 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4d -326> 2016-12-07 08:59:38.198401 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4e -325> 2016-12-07 08:59:38.198406 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.4f -324> 2016-12-07 08:59:38.198415 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5 -323> 2016-12-07 08:59:38.198425 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.50 -322> 2016-12-07 08:59:38.198435 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.51 -321> 2016-12-07 08:59:38.198441 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.52 -320> 2016-12-07 08:59:38.198446 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.53 -319> 2016-12-07 08:59:38.198456 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.54 -318> 2016-12-07 08:59:38.198460 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.55 -317> 2016-12-07 08:59:38.198467 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.56 -316> 2016-12-07 08:59:38.198477 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.57 -315> 2016-12-07 08:59:38.198482 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.58 -314> 2016-12-07 08:59:38.198487 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.59 -313> 2016-12-07 08:59:38.198500 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5a -312> 2016-12-07 08:59:38.198505 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5b -311> 2016-12-07 08:59:38.198512 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5c -310> 2016-12-07 08:59:38.198524 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5d -309> 2016-12-07 08:59:38.198536 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5e -308> 2016-12-07 08:59:38.198541 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.5f -307> 2016-12-07 08:59:38.198549 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6 -306> 2016-12-07 08:59:38.198566 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.60 -305> 2016-12-07 08:59:38.198575 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.61 -304> 2016-12-07 08:59:38.198580 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.62 -303> 2016-12-07 08:59:38.198585 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.63 -302> 2016-12-07 08:59:38.198595 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.64 -301> 2016-12-07 08:59:38.198617 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.65 -300> 2016-12-07 08:59:38.198635 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.66 -299> 2016-12-07 08:59:38.198641 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.67 -298> 2016-12-07 08:59:38.198662 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.68 -297> 2016-12-07 08:59:38.198672 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.69 -296> 2016-12-07 08:59:38.198682 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6a -295> 2016-12-07 08:59:38.198695 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6b -294> 2016-12-07 08:59:38.198707 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6c -293> 2016-12-07 08:59:38.198713 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6d -292> 2016-12-07 08:59:38.198721 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6e -291> 2016-12-07 08:59:38.198728 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.6f -290> 2016-12-07 08:59:38.198747 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7 -289> 2016-12-07 08:59:38.198752 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.70 -288> 2016-12-07 08:59:38.198757 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.71 -287> 2016-12-07 08:59:38.198769 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.72 -286> 2016-12-07 08:59:38.198779 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.73 -285> 2016-12-07 08:59:38.198784 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.74 -284> 2016-12-07 08:59:38.198790 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.75 -283> 2016-12-07 08:59:38.198795 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.76 -282> 2016-12-07 08:59:38.198799 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.77 -281> 2016-12-07 08:59:38.198807 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.78 -280> 2016-12-07 08:59:38.198816 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.79 -279> 2016-12-07 08:59:38.198821 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7a -278> 2016-12-07 08:59:38.198828 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7b -277> 2016-12-07 08:59:38.198833 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7c -276> 2016-12-07 08:59:38.198840 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7d -275> 2016-12-07 08:59:38.198844 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7e -274> 2016-12-07 08:59:38.198851 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.7f -273> 2016-12-07 08:59:38.198861 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.8 -272> 2016-12-07 08:59:38.198870 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.9 -271> 2016-12-07 08:59:38.198876 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.a -270> 2016-12-07 08:59:38.198881 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.b -269> 2016-12-07 08:59:38.198891 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.c -268> 2016-12-07 08:59:38.198898 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.d -267> 2016-12-07 08:59:38.198904 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.e -266> 2016-12-07 08:59:38.198916 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 1.f -265> 2016-12-07 08:59:38.198924 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.0 -264> 2016-12-07 08:59:38.198934 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1 -263> 2016-12-07 08:59:38.198939 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.10 -262> 2016-12-07 08:59:38.198947 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.11 -261> 2016-12-07 08:59:38.198957 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.12 -260> 2016-12-07 08:59:38.198962 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.13 -259> 2016-12-07 08:59:38.198976 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.14 -258> 2016-12-07 08:59:38.198986 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.15 -257> 2016-12-07 08:59:38.198991 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.16 -256> 2016-12-07 08:59:38.198996 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.17 -255> 2016-12-07 08:59:38.199007 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.18 -254> 2016-12-07 08:59:38.199012 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.19 -253> 2016-12-07 08:59:38.199017 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1a -252> 2016-12-07 08:59:38.199025 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1b -251> 2016-12-07 08:59:38.199040 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1c -250> 2016-12-07 08:59:38.199050 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1d -249> 2016-12-07 08:59:38.199058 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1e -248> 2016-12-07 08:59:38.199066 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.1f -247> 2016-12-07 08:59:38.199071 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2 -246> 2016-12-07 08:59:38.199079 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.20 -245> 2016-12-07 08:59:38.199085 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.21 -244> 2016-12-07 08:59:38.199098 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.22 -243> 2016-12-07 08:59:38.199104 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.23 -242> 2016-12-07 08:59:38.199111 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.24 -241> 2016-12-07 08:59:38.199117 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.25 -240> 2016-12-07 08:59:38.199128 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.26 -239> 2016-12-07 08:59:38.199133 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.27 -238> 2016-12-07 08:59:38.199138 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.28 -237> 2016-12-07 08:59:38.199149 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.29 -236> 2016-12-07 08:59:38.199157 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2a -235> 2016-12-07 08:59:38.199162 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2b -234> 2016-12-07 08:59:38.199168 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2c -233> 2016-12-07 08:59:38.199175 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2d -232> 2016-12-07 08:59:38.199182 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2e -231> 2016-12-07 08:59:38.199190 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.2f -230> 2016-12-07 08:59:38.199198 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3 -229> 2016-12-07 08:59:38.199207 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.30 -228> 2016-12-07 08:59:38.199213 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.31 -227> 2016-12-07 08:59:38.199218 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.32 -226> 2016-12-07 08:59:38.199226 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.33 -225> 2016-12-07 08:59:38.199236 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.34 -224> 2016-12-07 08:59:38.199243 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.35 -223> 2016-12-07 08:59:38.199250 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.36 -222> 2016-12-07 08:59:38.199259 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.37 -221> 2016-12-07 08:59:38.199264 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.38 -220> 2016-12-07 08:59:38.199271 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.39 -219> 2016-12-07 08:59:38.199280 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3a -218> 2016-12-07 08:59:38.199288 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3b -217> 2016-12-07 08:59:38.199293 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3c -216> 2016-12-07 08:59:38.199303 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3d -215> 2016-12-07 08:59:38.199311 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3e -214> 2016-12-07 08:59:38.199319 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.3f -213> 2016-12-07 08:59:38.199325 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4 -212> 2016-12-07 08:59:38.199335 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.40 -211> 2016-12-07 08:59:38.199341 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.41 -210> 2016-12-07 08:59:38.199346 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.42 -209> 2016-12-07 08:59:38.199356 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.43 -208> 2016-12-07 08:59:38.199363 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.44 -207> 2016-12-07 08:59:38.199371 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.45 -206> 2016-12-07 08:59:38.199377 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.46 -205> 2016-12-07 08:59:38.199386 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.47 -204> 2016-12-07 08:59:38.199392 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.48 -203> 2016-12-07 08:59:38.199397 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.49 -202> 2016-12-07 08:59:38.199407 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4a -201> 2016-12-07 08:59:38.199413 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4b -200> 2016-12-07 08:59:38.199419 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4c -199> 2016-12-07 08:59:38.199430 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4d -198> 2016-12-07 08:59:38.199435 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4e -197> 2016-12-07 08:59:38.199442 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.4f -196> 2016-12-07 08:59:38.199452 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5 -195> 2016-12-07 08:59:38.199461 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.50 -194> 2016-12-07 08:59:38.199472 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.51 -193> 2016-12-07 08:59:38.199479 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.52 -192> 2016-12-07 08:59:38.199486 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.53 -191> 2016-12-07 08:59:38.199491 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.54 -190> 2016-12-07 08:59:38.199502 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.55 -189> 2016-12-07 08:59:38.199514 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.56 -188> 2016-12-07 08:59:38.199520 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.57 -187> 2016-12-07 08:59:38.199527 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.58 -186> 2016-12-07 08:59:38.199535 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.59 -185> 2016-12-07 08:59:38.199541 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5a -184> 2016-12-07 08:59:38.199548 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5b -183> 2016-12-07 08:59:38.199553 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5c -182> 2016-12-07 08:59:38.199562 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5d -181> 2016-12-07 08:59:38.199567 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5e -180> 2016-12-07 08:59:38.199579 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.5f -179> 2016-12-07 08:59:38.199585 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6 -178> 2016-12-07 08:59:38.199605 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.60 -177> 2016-12-07 08:59:38.199626 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.61 -176> 2016-12-07 08:59:38.199631 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.62 -175> 2016-12-07 08:59:38.199641 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.63 -174> 2016-12-07 08:59:38.199647 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.64 -173> 2016-12-07 08:59:38.199655 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.65 -172> 2016-12-07 08:59:38.199664 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.66 -171> 2016-12-07 08:59:38.199671 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.67 -170> 2016-12-07 08:59:38.199677 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.68 -169> 2016-12-07 08:59:38.199687 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.69 -168> 2016-12-07 08:59:38.199692 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6a -167> 2016-12-07 08:59:38.199698 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6b -166> 2016-12-07 08:59:38.199812 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6c -165> 2016-12-07 08:59:38.199827 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6d -164> 2016-12-07 08:59:38.199833 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6e -163> 2016-12-07 08:59:38.199839 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.6f -162> 2016-12-07 08:59:38.199845 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7 -161> 2016-12-07 08:59:38.199854 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.70 -160> 2016-12-07 08:59:38.199859 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.71 -159> 2016-12-07 08:59:38.199868 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.72 -158> 2016-12-07 08:59:38.199874 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.73 -157> 2016-12-07 08:59:38.199881 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.74 -156> 2016-12-07 08:59:38.199888 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.75 -155> 2016-12-07 08:59:38.199895 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.76 -154> 2016-12-07 08:59:38.199900 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.77 -153> 2016-12-07 08:59:38.199905 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.78 -152> 2016-12-07 08:59:38.199914 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.79 -151> 2016-12-07 08:59:38.199926 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7a -150> 2016-12-07 08:59:38.199932 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7b -149> 2016-12-07 08:59:38.199937 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7c -148> 2016-12-07 08:59:38.199944 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7d -147> 2016-12-07 08:59:38.199949 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7e -146> 2016-12-07 08:59:38.199957 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.7f -145> 2016-12-07 08:59:38.199964 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.8 -144> 2016-12-07 08:59:38.199970 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.9 -143> 2016-12-07 08:59:38.199978 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.a -142> 2016-12-07 08:59:38.199983 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.b -141> 2016-12-07 08:59:38.199990 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.c -140> 2016-12-07 08:59:38.199998 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.d -139> 2016-12-07 08:59:38.200003 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.e -138> 2016-12-07 08:59:38.200008 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got 2.f -137> 2016-12-07 08:59:38.200111 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.0 -136> 2016-12-07 08:59:38.200123 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.1 -135> 2016-12-07 08:59:38.200128 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.2 -134> 2016-12-07 08:59:38.200132 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.3 -133> 2016-12-07 08:59:38.200136 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.4 -132> 2016-12-07 08:59:38.200140 7f8dfe4f9480 20 mon.ceph3@-1(probing).pg v37652 got osd.5 -131> 2016-12-07 08:59:38.200161 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v37652 update_logger -130> 2016-12-07 08:59:38.200401 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(mdsmap 1..438) refresh -129> 2016-12-07 08:59:38.200407 7f8dfe4f9480 10 mon.ceph3@-1(probing).mds e0 update_from_paxos version 438, my e 0 -128> 2016-12-07 08:59:38.200504 7f8dfe4f9480 10 mon.ceph3@-1(probing).mds e0 update_from_paxos got 438 -127> 2016-12-07 08:59:38.200558 7f8dfe4f9480 4 mon.ceph3@-1(probing).mds e438 new map -126> 2016-12-07 08:59:38.200564 7f8dfe4f9480 0 mon.ceph3@-1(probing).mds e438 print_map e438 enable_multiple, ever_enabled_multiple: 0,0 compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=file layout v2} Filesystem 'first_cephfs' (1) fs_name first_cephfs epoch 436 flags 8 created 2016-11-16 06:06:34.376616 modified 2016-11-23 11:37:26.768424 tableserver 0 root 0 session_timeout 60 session_autoclose 300 max_file_size 1099511627776 last_failure 0 last_failure_osd_epoch 68 compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=file layout v2} max_mds 1 in 0 up {0=58523} failed damaged stopped data_pools 1 metadata_pool 2 inline_data disabled 58523: 10.10.38.40:6806/13750 'ceph1' mds.0.433 up:active seq 70 Standby daemons: 39073: 10.10.38.41:6805/12990 'ceph2' mds.-1.0 up:standby seq 1 (standby for rank 0) -125> 2016-12-07 08:59:38.200615 7f8dfe4f9480 10 mon.ceph3@-1(probing).mds e438 update_logger -124> 2016-12-07 08:59:38.200799 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(osdmap 1..71) refresh -123> 2016-12-07 08:59:38.200809 7f8dfe4f9480 15 mon.ceph3@-1(probing).osd e0 update_from_paxos paxos e 71, my e 0 -122> 2016-12-07 08:59:38.200905 7f8dfe4f9480 7 mon.ceph3@-1(probing).osd e0 update_from_paxos loading latest full map e71 -121> 2016-12-07 08:59:38.200975 7f8dfe4f9480 10 mon.ceph3@-1(probing).osd e71 check_subs -120> 2016-12-07 08:59:38.200983 7f8dfe4f9480 10 mon.ceph3@-1(probing).osd e71 share_map_with_random_osd no up osd on our session map -119> 2016-12-07 08:59:38.200985 7f8dfe4f9480 10 mon.ceph3@-1(probing).osd e71 update_logger -118> 2016-12-07 08:59:38.200992 7f8dfe4f9480 0 mon.ceph3@-1(probing).osd e71 crush map has features 2200130813952, adjusting msgr requires -117> 2016-12-07 08:59:38.200994 7f8dfe4f9480 0 mon.ceph3@-1(probing).osd e71 crush map has features 2200130813952, adjusting msgr requires -116> 2016-12-07 08:59:38.200997 7f8dfe4f9480 0 mon.ceph3@-1(probing).osd e71 crush map has features 2200130813952, adjusting msgr requires -115> 2016-12-07 08:59:38.200998 7f8dfe4f9480 0 mon.ceph3@-1(probing).osd e71 crush map has features 2200130813952, adjusting msgr requires -114> 2016-12-07 08:59:38.201265 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(logm 50434..51109) refresh -113> 2016-12-07 08:59:38.201274 7f8dfe4f9480 10 mon.ceph3@-1(probing).log v51109 update_from_paxos -112> 2016-12-07 08:59:38.201277 7f8dfe4f9480 10 mon.ceph3@-1(probing).log v51109 update_from_paxos version 51109 summary v 0 -111> 2016-12-07 08:59:38.201361 7f8dfe4f9480 10 mon.ceph3@-1(probing).log v51109 update_from_paxos latest full 51108 -110> 2016-12-07 08:59:38.201457 7f8dfe4f9480 7 mon.ceph3@-1(probing).log v51109 update_from_paxos loading summary e51108 -109> 2016-12-07 08:59:38.201489 7f8dfe4f9480 7 mon.ceph3@-1(probing).log v51109 update_from_paxos loaded summary e51108 -108> 2016-12-07 08:59:38.201602 7f8dfe4f9480 7 mon.ceph3@-1(probing).log v51109 update_from_paxos applying incremental log 51109 2016-11-25 09:22:08.515897 mon.0 10.10.38.40:6789/0 44591 : audit [DBG] from='client.? 10.10.38.40:0/717519612' entity='client.admin' cmd=[{"prefix": "df"}]: dispatch -107> 2016-12-07 08:59:38.201624 7f8dfe4f9480 20 mon.ceph3@-1(probing).log v51109 update_from_paxos logging for channel 'audit' to file '/var/log/ceph/ceph.audit.log' -106> 2016-12-07 08:59:38.201640 7f8dfe4f9480 15 mon.ceph3@-1(probing).log v51109 update_from_paxos logging for 0 channels -105> 2016-12-07 08:59:38.201645 7f8dfe4f9480 10 mon.ceph3@-1(probing).log v51109 check_subs -104> 2016-12-07 08:59:38.201888 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(monmap 1..3) refresh -103> 2016-12-07 08:59:38.202132 7f8dfe4f9480 1 mon.ceph3@-1(probing).paxosservice(auth 1..264) refresh upgraded, format 0 -> 1 -102> 2016-12-07 08:59:38.202137 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(auth 1..264) refresh -101> 2016-12-07 08:59:38.202140 7f8dfe4f9480 10 mon.ceph3@-1(probing).auth v264 update_from_paxos -100> 2016-12-07 08:59:38.202219 7f8dfe4f9480 10 mon.ceph3@-1(probing).auth v264 update_from_paxos version 264 keys ver 0 latest 261 -99> 2016-12-07 08:59:38.202306 7f8dfe4f9480 7 mon.ceph3@-1(probing).auth v264 update_from_paxos loading summary e 261 -98> 2016-12-07 08:59:38.202311 7f8dfe4f9480 7 mon.ceph3@-1(probing).auth v264 update_from_paxos latest length 1958 -97> 2016-12-07 08:59:38.202609 7f8dfe4f9480 10 mon.ceph3@-1(probing).auth v264 update_from_paxos key server version 261 -96> 2016-12-07 08:59:38.202793 7f8dfe4f9480 20 mon.ceph3@-1(probing).auth v264 update_from_paxos walking through version 262 len 649 -95> 2016-12-07 08:59:38.202977 7f8dfe4f9480 20 mon.ceph3@-1(probing).auth v264 update_from_paxos walking through version 263 len 649 -94> 2016-12-07 08:59:38.203153 7f8dfe4f9480 20 mon.ceph3@-1(probing).auth v264 update_from_paxos walking through version 264 len 649 -93> 2016-12-07 08:59:38.203257 7f8dfe4f9480 10 mon.ceph3@-1(probing).auth v264 update_from_paxos() last_allocated_id=104096 max_global_id=104096 format_version 1 -92> 2016-12-07 08:59:38.203263 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(pgmap 36912..37652) post_refresh -91> 2016-12-07 08:59:38.203267 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v37652 post_paxos_update -90> 2016-12-07 08:59:38.203281 7f8dfe4f9480 10 mon.ceph3@-1(probing).pg v37652 check_subs -89> 2016-12-07 08:59:38.203283 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(mdsmap 1..438) post_refresh -88> 2016-12-07 08:59:38.203284 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(osdmap 1..71) post_refresh -87> 2016-12-07 08:59:38.203285 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(logm 50434..51109) post_refresh -86> 2016-12-07 08:59:38.203285 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(monmap 1..3) post_refresh -85> 2016-12-07 08:59:38.203286 7f8dfe4f9480 10 mon.ceph3@-1(probing).paxosservice(auth 1..264) post_refresh -84> 2016-12-07 08:59:38.203289 7f8dfe4f9480 10 mon.ceph3@-1(probing).health(0) init -83> 2016-12-07 08:59:38.203427 7f8dfe4f9480 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mon/ceph-ceph3/keyring -82> 2016-12-07 08:59:38.203448 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command mon_status hook 0x7f8e08d5e660 -81> 2016-12-07 08:59:38.203453 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command quorum_status hook 0x7f8e08d5e660 -80> 2016-12-07 08:59:38.203456 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command sync_force hook 0x7f8e08d5e660 -79> 2016-12-07 08:59:38.203459 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command add_bootstrap_peer_hint hook 0x7f8e08d5e660 -78> 2016-12-07 08:59:38.203462 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command quorum enter hook 0x7f8e08d5e660 -77> 2016-12-07 08:59:38.203466 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command quorum exit hook 0x7f8e08d5e660 -76> 2016-12-07 08:59:38.203469 7f8dfe4f9480 5 asok(0x7f8e08c7a480) register_command ops hook 0x7f8e08d5e660 -75> 2016-12-07 08:59:38.203492 7f8dfe4f9480 1 -- 10.10.38.42:6789/0 messenger.start -74> 2016-12-07 08:59:38.203598 7f8dfe4f9480 2 mon.ceph3@-1(probing) e3 init -73> 2016-12-07 08:59:38.204155 7f8dfe4f9480 1 accepter.accepter.start -72> 2016-12-07 08:59:38.204222 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 bootstrap -71> 2016-12-07 08:59:38.204230 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 sync_reset_requester -70> 2016-12-07 08:59:38.204231 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 unregister_cluster_logger - not registered -69> 2016-12-07 08:59:38.204232 7f8dfe4f9480 10 mon.ceph3@-1(probing) e3 cancel_probe_timeout (none scheduled) -68> 2016-12-07 08:59:38.204233 7f8dfe4f9480 0 mon.ceph3@-1(probing) e3 my rank is now 0 (was -1) -67> 2016-12-07 08:59:38.204235 7f8dfe4f9480 1 -- 10.10.38.42:6789/0 mark_down_all -66> 2016-12-07 08:59:38.204240 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 _reset -65> 2016-12-07 08:59:38.204241 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 cancel_probe_timeout (none scheduled) -64> 2016-12-07 08:59:38.204242 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 timecheck_finish -63> 2016-12-07 08:59:38.204244 7f8dfe4f9480 15 mon.ceph3@0(probing) e3 health_tick_stop -62> 2016-12-07 08:59:38.204245 7f8dfe4f9480 15 mon.ceph3@0(probing) e3 health_interval_stop -61> 2016-12-07 08:59:38.204247 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 scrub_event_cancel -60> 2016-12-07 08:59:38.204248 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 scrub_reset -59> 2016-12-07 08:59:38.204250 7f8dfe4f9480 10 mon.ceph3@0(probing).paxos(paxos recovering c 89357..90053) restart -- canceling timeouts -58> 2016-12-07 08:59:38.204255 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(pgmap 36912..37652) restart -57> 2016-12-07 08:59:38.204258 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(mdsmap 1..438) restart -56> 2016-12-07 08:59:38.204259 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(osdmap 1..71) restart -55> 2016-12-07 08:59:38.204260 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(logm 50434..51109) restart -54> 2016-12-07 08:59:38.204261 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(monmap 1..3) restart -53> 2016-12-07 08:59:38.204262 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(auth 1..264) restart -52> 2016-12-07 08:59:38.204264 7f8dfe4f9480 1 mon.ceph3@0(probing) e3 win_standalone_election -51> 2016-12-07 08:59:38.204376 7f8dfe4f9480 1 mon.ceph3@0(probing).elector(488) init, last seen epoch 488 -50> 2016-12-07 08:59:38.204383 7f8dfe4f9480 10 mon.ceph3@0(probing).elector(488) bump_epoch 488 to 489 -49> 2016-12-07 08:59:38.221449 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 join_election -48> 2016-12-07 08:59:38.221468 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 _reset -47> 2016-12-07 08:59:38.221470 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 cancel_probe_timeout (none scheduled) -46> 2016-12-07 08:59:38.221472 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 timecheck_finish -45> 2016-12-07 08:59:38.221474 7f8dfe4f9480 15 mon.ceph3@0(probing) e3 health_tick_stop -44> 2016-12-07 08:59:38.221476 7f8dfe4f9480 15 mon.ceph3@0(probing) e3 health_interval_stop -43> 2016-12-07 08:59:38.221477 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 scrub_event_cancel -42> 2016-12-07 08:59:38.221479 7f8dfe4f9480 10 mon.ceph3@0(probing) e3 scrub_reset -41> 2016-12-07 08:59:38.221481 7f8dfe4f9480 10 mon.ceph3@0(probing).paxos(paxos recovering c 89357..90053) restart -- canceling timeouts -40> 2016-12-07 08:59:38.221487 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(pgmap 36912..37652) restart -39> 2016-12-07 08:59:38.221490 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(mdsmap 1..438) restart -38> 2016-12-07 08:59:38.221492 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(osdmap 1..71) restart -37> 2016-12-07 08:59:38.221494 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(logm 50434..51109) restart -36> 2016-12-07 08:59:38.221496 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(monmap 1..3) restart -35> 2016-12-07 08:59:38.221497 7f8dfe4f9480 10 mon.ceph3@0(probing).paxosservice(auth 1..264) restart -34> 2016-12-07 08:59:38.221504 7f8dfe4f9480 10 mon.ceph3@0(electing) e3 win_election epoch 489 quorum 0 features 576460752032874495 -33> 2016-12-07 08:59:38.221526 7f8dfe4f9480 0 log_channel(cluster) log [INF] : mon.ceph3@0 won leader election with quorum 0 -32> 2016-12-07 08:59:38.221529 7f8dfe4f9480 10 log_client _send_to_monlog to self -31> 2016-12-07 08:59:38.221530 7f8dfe4f9480 10 log_client log_queue is 1 last_log 1 sent 0 num 1 unsent 1 sending 1 -30> 2016-12-07 08:59:38.221533 7f8dfe4f9480 10 log_client will send 2016-12-07 08:59:38.221527 mon.0 10.10.38.42:6789/0 1 : cluster [INF] mon.ceph3@0 won leader election with quorum 0 -29> 2016-12-07 08:59:38.221544 7f8dfe4f9480 1 -- 10.10.38.42:6789/0 --> 10.10.38.42:6789/0 -- log(1 entries from seq 1 at 2016-12-07 08:59:38.221527) v1 -- ?+0 0x7f8e08d8d730 con 0x7f8e08ced390 -28> 2016-12-07 08:59:38.221562 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(monmap 1..3) election_finished -27> 2016-12-07 08:59:38.221564 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(monmap 1..3) _active -26> 2016-12-07 08:59:38.221565 7f8dfe4f9480 7 mon.ceph3@0(leader).paxosservice(monmap 1..3) _active creating new pending -25> 2016-12-07 08:59:38.221570 7f8dfe4f9480 10 mon.ceph3@0(leader).monmap v3 create_pending monmap epoch 4 -24> 2016-12-07 08:59:38.221589 7f8dfe4f9480 0 log_channel(cluster) log [INF] : monmap e3: 1 mons at {ceph3=10.10.38.42:6789/0} -23> 2016-12-07 08:59:38.221594 7f8dfe4f9480 10 log_client _send_to_monlog to self -22> 2016-12-07 08:59:38.221595 7f8dfe4f9480 10 log_client log_queue is 2 last_log 2 sent 1 num 2 unsent 1 sending 1 -21> 2016-12-07 08:59:38.221597 7f8dfe4f9480 10 log_client will send 2016-12-07 08:59:38.221593 mon.0 10.10.38.42:6789/0 2 : cluster [INF] monmap e3: 1 mons at {ceph3=10.10.38.42:6789/0} -20> 2016-12-07 08:59:38.221602 7f8dfe4f9480 1 -- 10.10.38.42:6789/0 --> 10.10.38.42:6789/0 -- log(1 entries from seq 2 at 2016-12-07 08:59:38.221593) v1 -- ?+0 0x7f8e08d8ee60 con 0x7f8e08ced390 -19> 2016-12-07 08:59:38.221640 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(pgmap 36912..37652) election_finished -18> 2016-12-07 08:59:38.221645 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(pgmap 36912..37652) _active -17> 2016-12-07 08:59:38.221647 7f8dfe4f9480 7 mon.ceph3@0(leader).paxosservice(pgmap 36912..37652) _active creating new pending -16> 2016-12-07 08:59:38.221652 7f8dfe4f9480 10 mon.ceph3@0(leader).pg v37652 create_pending v 37653 -15> 2016-12-07 08:59:38.221657 7f8dfe4f9480 10 mon.ceph3@0(leader).pg v37652 check_osd_map already seen 71 >= 71 -14> 2016-12-07 08:59:38.221659 7f8dfe4f9480 10 mon.ceph3@0(leader).pg v37652 update_logger -13> 2016-12-07 08:59:38.221677 7f8dfe4f9480 0 log_channel(cluster) log [INF] : pgmap v37652: 320 pgs: 320 active+clean; 35394 kB data, 1171 MB used, 214 GB / 215 GB avail -12> 2016-12-07 08:59:38.221682 7f8dfe4f9480 10 log_client _send_to_monlog to self -11> 2016-12-07 08:59:38.221684 7f8dfe4f9480 10 log_client log_queue is 3 last_log 3 sent 2 num 3 unsent 1 sending 1 -10> 2016-12-07 08:59:38.221686 7f8dfe4f9480 10 log_client will send 2016-12-07 08:59:38.221682 mon.0 10.10.38.42:6789/0 3 : cluster [INF] pgmap v37652: 320 pgs: 320 active+clean; 35394 kB data, 1171 MB used, 214 GB / 215 GB avail -9> 2016-12-07 08:59:38.221691 7f8dfe4f9480 1 -- 10.10.38.42:6789/0 --> 10.10.38.42:6789/0 -- log(1 entries from seq 3 at 2016-12-07 08:59:38.221682) v1 -- ?+0 0x7f8e08d8ff30 con 0x7f8e08ced390 -8> 2016-12-07 08:59:38.221696 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(mdsmap 1..438) election_finished -7> 2016-12-07 08:59:38.221697 7f8dfe4f9480 10 mon.ceph3@0(leader).paxosservice(mdsmap 1..438) _active -6> 2016-12-07 08:59:38.221698 7f8dfe4f9480 7 mon.ceph3@0(leader).paxosservice(mdsmap 1..438) _active creating new pending -5> 2016-12-07 08:59:38.221696 7f8de7fff700 1 -- 10.10.38.42:6789/0 <== mon.0 10.10.38.42:6789/0 0 ==== log(1 entries from seq 1 at 2016-12-07 08:59:38.221527) v1 ==== 0+0+0 (0 0 0) 0x7f8e08d8d730 con 0x7f8e08ced390 -4> 2016-12-07 08:59:38.221715 7f8dfe4f9480 10 mon.ceph3@0(leader).mds e438 create_pending e439 -3> 2016-12-07 08:59:38.221719 7f8dfe4f9480 10 mon.ceph3@0(leader).mds e438 e438: 1/1/1 up {0=ceph1=up:active}, 1 up:standby -2> 2016-12-07 08:59:38.221743 7f8dfe4f9480 20 mon.ceph3@0(leader).mds e438 gid 39073 is standby and following nobody -1> 2016-12-07 08:59:38.221750 7f8dfe4f9480 10 mon.ceph3@0(leader).mds e438 setting to follow mds rank 0 0> 2016-12-07 08:59:38.224093 7f8dfe4f9480 -1 mon/MDSMonitor.cc: In function 'bool MDSMonitor::maybe_promote_standby(std::shared_ptr)' thread 7f8dfe4f9480 time 2016-12-07 08:59:38.221755 mon/MDSMonitor.cc: 2797: FAILED assert(info.state == MDSMap::STATE_STANDBY) ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7f8dfe0389fb] 2: (MDSMonitor::maybe_promote_standby(std::shared_ptr)+0x976) [0x7f8dfdd79016] 3: (MDSMonitor::tick()+0x397) [0x7f8dfdd7cfa7] 4: (MDSMonitor::on_active()+0x15) [0x7f8dfdd730a5] 5: (PaxosService::_active()+0x1df) [0x7f8dfdcff2ef] 6: (PaxosService::election_finished()+0x67) [0x7f8dfdcffa47] 7: (Monitor::win_election(unsigned int, std::set, std::allocator >&, unsigned long, MonCommand const*, int, std::set, std::allocator > const*)+0x236) [0x7f8dfdcc1336] 8: (Monitor::win_standalone_election()+0x158) [0x7f8dfdcc1738] 9: (Monitor::bootstrap()+0xa03) [0x7f8dfdcc2283] 10: (Monitor::init()+0x190) [0x7f8dfdcc2590] 11: (main()+0x24ca) [0x7f8dfdc38a4a] 12: (__libc_start_main()+0xf5) [0x7f8dfb1fcec5] 13: (()+0x2609ca) [0x7f8dfdc8a9ca] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 20/20 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 20/20 mon 20/20 monc 20/20 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 newstore 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 1/ 5 kinetic 1/ 5 fuse -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.ceph3.log --- end dump of recent events --- 2016-12-07 08:59:38.225562 7f8df349c700 1 leveldb: Generated table #8841: 321 keys, 2151000 bytes 2016-12-07 08:59:38.229234 7f8dfe4f9480 -1 *** Caught signal (Aborted) ** in thread 7f8dfe4f9480 thread_name:ceph-mon ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b) 1: (()+0x4f4f32) [0x7f8dfdf1ef32] 2: (()+0x10340) [0x7f8dfcf8b340] 3: (gsignal()+0x39) [0x7f8dfb211cc9] 4: (abort()+0x148) [0x7f8dfb2150d8] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x265) [0x7f8dfe038bd5] 6: (MDSMonitor::maybe_promote_standby(std::shared_ptr)+0x976) [0x7f8dfdd79016] 7: (MDSMonitor::tick()+0x397) [0x7f8dfdd7cfa7] 8: (MDSMonitor::on_active()+0x15) [0x7f8dfdd730a5] 9: (PaxosService::_active()+0x1df) [0x7f8dfdcff2ef] 10: (PaxosService::election_finished()+0x67) [0x7f8dfdcffa47] 11: (Monitor::win_election(unsigned int, std::set, std::allocator >&, unsigned long, MonCommand const*, int, std::set, std::allocator > const*)+0x236) [0x7f8dfdcc1336] 12: (Monitor::win_standalone_election()+0x158) [0x7f8dfdcc1738] 13: (Monitor::bootstrap()+0xa03) [0x7f8dfdcc2283] 14: (Monitor::init()+0x190) [0x7f8dfdcc2590] 15: (main()+0x24ca) [0x7f8dfdc38a4a] 16: (__libc_start_main()+0xf5) [0x7f8dfb1fcec5] 17: (()+0x2609ca) [0x7f8dfdc8a9ca] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -2> 2016-12-07 08:59:38.225562 7f8df349c700 1 leveldb: Generated table #8841: 321 keys, 2151000 bytes -1> 2016-12-07 08:59:38.225580 7f8df89db700 5 asok(0x7f8e08c7a480) AdminSocket: request 'get_command_descriptions' '' to 0x7f8e08cd3270 returned 2165 bytes 0> 2016-12-07 08:59:38.229234 7f8dfe4f9480 -1 *** Caught signal (Aborted) ** in thread 7f8dfe4f9480 thread_name:ceph-mon ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b) 1: (()+0x4f4f32) [0x7f8dfdf1ef32] 2: (()+0x10340) [0x7f8dfcf8b340] 3: (gsignal()+0x39) [0x7f8dfb211cc9] 4: (abort()+0x148) [0x7f8dfb2150d8] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x265) [0x7f8dfe038bd5] 6: (MDSMonitor::maybe_promote_standby(std::shared_ptr)+0x976) [0x7f8dfdd79016] 7: (MDSMonitor::tick()+0x397) [0x7f8dfdd7cfa7] 8: (MDSMonitor::on_active()+0x15) [0x7f8dfdd730a5] 9: (PaxosService::_active()+0x1df) [0x7f8dfdcff2ef] 10: (PaxosService::election_finished()+0x67) [0x7f8dfdcffa47] 11: (Monitor::win_election(unsigned int, std::set, std::allocator >&, unsigned long, MonCommand const*, int, std::set, std::allocator > const*)+0x236) [0x7f8dfdcc1336] 12: (Monitor::win_standalone_election()+0x158) [0x7f8dfdcc1738] 13: (Monitor::bootstrap()+0xa03) [0x7f8dfdcc2283] 14: (Monitor::init()+0x190) [0x7f8dfdcc2590] 15: (main()+0x24ca) [0x7f8dfdc38a4a] 16: (__libc_start_main()+0xf5) [0x7f8dfb1fcec5] 17: (()+0x2609ca) [0x7f8dfdc8a9ca] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 20/20 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 20/20 mon 20/20 monc 20/20 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 newstore 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 1/ 5 kinetic 1/ 5 fuse -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.ceph3.log --- end dump of recent events ---