Project

General

Profile

Bug #21142 » osd-2.log

Josef Zelenka, 01/11/2018 09:43 AM

 
-65> 2018-01-11 10:18:28.455085 7f2d8f420e00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
-64> 2018-01-11 10:18:28.455086 7f2d8f420e00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
-63> 2018-01-11 10:18:28.455087 7f2d8f420e00 4 rocksdb: Options.table_properties_collectors:
-62> 2018-01-11 10:18:28.455088 7f2d8f420e00 4 rocksdb: Options.inplace_update_support: 0
-61> 2018-01-11 10:18:28.455089 7f2d8f420e00 4 rocksdb: Options.inplace_update_num_locks: 10000
-60> 2018-01-11 10:18:28.455091 7f2d8f420e00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
-59> 2018-01-11 10:18:28.455093 7f2d8f420e00 4 rocksdb: Options.memtable_huge_page_size: 0
-58> 2018-01-11 10:18:28.455094 7f2d8f420e00 4 rocksdb: Options.bloom_locality: 0
-57> 2018-01-11 10:18:28.455106 7f2d8f420e00 4 rocksdb: Options.max_successive_merges: 0
-56> 2018-01-11 10:18:28.455108 7f2d8f420e00 4 rocksdb: Options.optimize_filters_for_hits: 0
-55> 2018-01-11 10:18:28.455109 7f2d8f420e00 4 rocksdb: Options.paranoid_file_checks: 0
-54> 2018-01-11 10:18:28.455110 7f2d8f420e00 4 rocksdb: Options.force_consistency_checks: 0
-53> 2018-01-11 10:18:28.455111 7f2d8f420e00 4 rocksdb: Options.report_bg_io_stats: 0
-52> 2018-01-11 10:18:32.895753 7f2d8f420e00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2859] Recovered from manifest file:db/MANIFEST-016210 succeeded,manifest_file_number is 16210, next_file_number is 16212, last_sequence is 341921103, log_number is 0,prev_log_number is 0,max_column_family is 0

-51> 2018-01-11 10:18:32.895777 7f2d8f420e00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2867] Column family [default] (ID 0), log number is 16204

-50> 2018-01-11 10:18:32.896795 7f2d8f420e00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662312896782, "job": 1, "event": "recovery_started", "log_files": [16211]}
-49> 2018-01-11 10:18:32.896808 7f2d8f420e00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:482] Recovering log #16211 mode 0
-48> 2018-01-11 10:18:32.928028 7f2d8f420e00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:815] [default] [WriteLevel0TableForRecovery] Level-0 table #16212: started
-47> 2018-01-11 10:18:32.937528 7f2d8f420e00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662312937510, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 16212, "file_size": 373756, "table_properties": {"data_size": 355562, "index_size": 2855, "filter_size": 14363, "raw_key_size": 233484, "raw_average_key_size": 42, "raw_value_size": 161972, "raw_average_value_size": 29, "num_data_blocks": 88, "num_entries": 5431, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
-46> 2018-01-11 10:18:32.937549 7f2d8f420e00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:847] [default] [WriteLevel0TableForRecovery] Level-0 table #16212: 373756 bytes OK
-45> 2018-01-11 10:18:32.937904 7f2d8f420e00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2395] Creating manifest 16213

-44> 2018-01-11 10:18:32.940367 7f2d8f420e00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662312940362, "job": 1, "event": "recovery_finished"}
-43> 2018-01-11 10:18:32.940822 7f2d8f420e00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_files.cc:307] [JOB 2] Delete db//MANIFEST-016210 type=3 #16210 -- OK

-42> 2018-01-11 10:18:32.940846 7f2d8f420e00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_files.cc:307] [JOB 2] Delete db//016211.log type=0 #16211 -- OK

-41> 2018-01-11 10:18:32.943915 7f2d8f420e00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:1063] DB pointer 0x559f74793a00
-40> 2018-01-11 10:18:32.943969 7f2d8f420e00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
-39> 2018-01-11 10:18:32.991001 7f2d8f420e00 1 freelist init
-38> 2018-01-11 10:18:33.055687 7f2d8f420e00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc opening allocation metadata
-37> 2018-01-11 10:18:35.634385 7f2d88418700 1 do_command 'perf dump' '
-36> 2018-01-11 10:18:35.634955 7f2d88418700 1 do_command 'perf dump' 'result is 17482 bytes
-35> 2018-01-11 10:18:42.739078 7f2d8f420e00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc loaded 318 G in 658679 extents
-34> 2018-01-11 10:18:42.857862 7f2d8f420e00 0 <cls> /build/ceph-12.2.1/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
-33> 2018-01-11 10:18:42.860999 7f2d8f420e00 0 <cls> /build/ceph-12.2.1/src/cls/hello/cls_hello.cc:296: loading cls_hello
-32> 2018-01-11 10:18:42.861042 7f2d8f420e00 0 _get_class not permitted to load kvs
-31> 2018-01-11 10:18:42.861282 7f2d8f420e00 0 _get_class not permitted to load lua
-30> 2018-01-11 10:18:42.863223 7f2d8f420e00 0 _get_class not permitted to load sdk
-29> 2018-01-11 10:18:42.891762 7f2d8f420e00 0 osd.2 15344 crush map has features 2200130813952, adjusting msgr requires for clients
-28> 2018-01-11 10:18:42.891790 7f2d8f420e00 0 osd.2 15344 crush map has features 2200130813952 was 8705, adjusting msgr requires for mons
-27> 2018-01-11 10:18:42.891799 7f2d8f420e00 0 osd.2 15344 crush map has features 720578140510109696, adjusting msgr requires for osds
-26> 2018-01-11 10:18:45.617022 7f2d88418700 1 do_command 'perf dump' '
-25> 2018-01-11 10:18:45.617599 7f2d88418700 1 do_command 'perf dump' 'result is 22053 bytes
-24> 2018-01-11 10:18:55.614218 7f2d88418700 1 do_command 'perf dump' '
-23> 2018-01-11 10:18:55.614847 7f2d88418700 1 do_command 'perf dump' 'result is 22053 bytes
-22> 2018-01-11 10:18:59.092306 7f2d8f420e00 0 osd.2 15344 load_pgs
-21> 2018-01-11 10:19:05.649266 7f2d88418700 1 do_command 'perf dump' '
-20> 2018-01-11 10:19:05.650106 7f2d88418700 1 do_command 'perf dump' 'result is 22122 bytes
-19> 2018-01-11 10:19:15.615286 7f2d88418700 1 do_command 'perf dump' '
-18> 2018-01-11 10:19:15.615845 7f2d88418700 1 do_command 'perf dump' 'result is 22146 bytes
-17> 2018-01-11 10:19:25.604131 7f2d88418700 1 do_command 'perf dump' '
-16> 2018-01-11 10:19:25.604831 7f2d88418700 1 do_command 'perf dump' 'result is 22149 bytes
-15> 2018-01-11 10:19:35.748401 7f2d88418700 1 do_command 'perf dump' '
-14> 2018-01-11 10:19:35.749014 7f2d88418700 1 do_command 'perf dump' 'result is 22149 bytes
-13> 2018-01-11 10:19:45.592690 7f2d88418700 1 do_command 'perf dump' '
-12> 2018-01-11 10:19:45.593107 7f2d88418700 1 do_command 'perf dump' 'result is 22149 bytes
-11> 2018-01-11 10:19:55.663018 7f2d88418700 1 do_command 'perf dump' '
-10> 2018-01-11 10:19:55.663371 7f2d88418700 1 do_command 'perf dump' 'result is 22151 bytes
-9> 2018-01-11 10:20:05.709910 7f2d88418700 1 do_command 'perf dump' '
-8> 2018-01-11 10:20:05.710256 7f2d88418700 1 do_command 'perf dump' 'result is 22152 bytes
-7> 2018-01-11 10:20:15.663581 7f2d88418700 1 do_command 'perf dump' '
-6> 2018-01-11 10:20:15.664246 7f2d88418700 1 do_command 'perf dump' 'result is 22152 bytes
-5> 2018-01-11 10:20:25.693720 7f2d88418700 1 do_command 'perf dump' '
-4> 2018-01-11 10:20:25.694053 7f2d88418700 1 do_command 'perf dump' 'result is 22152 bytes
-3> 2018-01-11 10:20:35.753869 7f2d88418700 1 do_command 'perf dump' '
-2> 2018-01-11 10:20:35.754213 7f2d88418700 1 do_command 'perf dump' 'result is 22153 bytes
-1> 2018-01-11 10:20:38.413520 7f2d8f420e00 0 osd.2 15344 load_pgs opened 5429 pgs
0> 2018-01-11 10:20:43.967917 7f2d8f420e00 -1 /build/ceph-12.2.1/src/osd/osd_types.cc: In function 'virtual void pi_compact_rep::add_interval(bool, const PastIntervals::pg_interval_t&)' thread 7f2d8f420e00 time 2018-01-11 10:20:43.965124
/build/ceph-12.2.1/src/osd/osd_types.cc: 3205: FAILED assert(interval.last > last)

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x559f6abd23f2]
2: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x559f6a85bb75]
3: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x559f6a839376]
4: (OSD::build_past_intervals_parallel()+0xe71) [0x559f6a5eef01]
5: (OSD::load_pgs()+0x14fb) [0x559f6a5f192b]
6: (OSD::init()+0x2217) [0x559f6a60bb37]
7: (main()+0x2eb8) [0x559f6a51c1f8]
8: (__libc_start_main()+0xf0) [0x7f2d8c891830]
9: (_start()+0x29) [0x559f6a5a7a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 0 lockdep
0/ 0 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 0 buffer
0/ 0 timer
0/ 0 filer
0/ 1 striper
0/ 0 objecter
0/ 0 rados
0/ 0 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 0 journaler
0/ 0 objectcacher
0/ 0 client
0/ 0 osd
0/ 0 optracker
0/ 0 objclass
0/ 0 filestore
0/ 0 journal
0/ 0 ms
0/ 0 mon
0/ 0 monc
0/ 0 paxos
0/ 0 tp
0/ 0 auth
1/ 5 crypto
0/ 0 finisher
1/ 5 heartbeatmap
0/ 0 perfcounter
0/ 0 rgw
1/10 civetweb
1/ 5 javaclient
0/ 0 asok
0/ 0 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2018-01-11 10:20:43.971431 7f2d8f420e00 -1 *** Caught signal (Aborted) **
in thread 7f2d8f420e00 thread_name:ceph-osd

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (()+0xa5a634) [0x559f6ab8f634]
2: (()+0x11390) [0x7f2d8d90b390]
3: (gsignal()+0x38) [0x7f2d8c8a6428]
4: (abort()+0x16a) [0x7f2d8c8a802a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x559f6abd257e]
6: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x559f6a85bb75]
7: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x559f6a839376]
8: (OSD::build_past_intervals_parallel()+0xe71) [0x559f6a5eef01]
9: (OSD::load_pgs()+0x14fb) [0x559f6a5f192b]
10: (OSD::init()+0x2217) [0x559f6a60bb37]
11: (main()+0x2eb8) [0x559f6a51c1f8]
12: (__libc_start_main()+0xf0) [0x7f2d8c891830]
13: (_start()+0x29) [0x559f6a5a7a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
0> 2018-01-11 10:20:43.971431 7f2d8f420e00 -1 *** Caught signal (Aborted) **
in thread 7f2d8f420e00 thread_name:ceph-osd

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (()+0xa5a634) [0x559f6ab8f634]
2: (()+0x11390) [0x7f2d8d90b390]
3: (gsignal()+0x38) [0x7f2d8c8a6428]
4: (abort()+0x16a) [0x7f2d8c8a802a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x559f6abd257e]
6: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x559f6a85bb75]
7: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x559f6a839376]
8: (OSD::build_past_intervals_parallel()+0xe71) [0x559f6a5eef01]
9: (OSD::load_pgs()+0x14fb) [0x559f6a5f192b]
10: (OSD::init()+0x2217) [0x559f6a60bb37]
11: (main()+0x2eb8) [0x559f6a51c1f8]
12: (__libc_start_main()+0xf0) [0x7f2d8c891830]
13: (_start()+0x29) [0x559f6a5a7a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 0 lockdep
0/ 0 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 0 buffer
0/ 0 timer
0/ 0 filer
0/ 1 striper
0/ 0 objecter
0/ 0 rados
0/ 0 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 0 journaler
0/ 0 objectcacher
0/ 0 client
0/ 0 osd
0/ 0 optracker
0/ 0 objclass
0/ 0 filestore
0/ 0 journal
0/ 0 ms
0/ 0 mon
0/ 0 monc
0/ 0 paxos
0/ 0 tp
0/ 0 auth
1/ 5 crypto
0/ 0 finisher
1/ 5 heartbeatmap
0/ 0 perfcounter
0/ 0 rgw
1/10 civetweb
1/ 5 javaclient
0/ 0 asok
0/ 0 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2018-01-11 10:21:04.820311 7f146d17ae00 0 set uid:gid to 64045:64045 (ceph:ceph)
2018-01-11 10:21:04.820336 7f146d17ae00 0 ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable), process (unknown), pid 1935869
2018-01-11 10:21:04.827145 7f146d17ae00 0 pidfile_write: ignore empty --pid-file
2018-01-11 10:21:04.838000 7f146d17ae00 0 load: jerasure load: lrc load: isa
2018-01-11 10:21:04.838088 7f146d17ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:21:04.838099 7f146d17ae00 1 bdev(0x55fabdc36900 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:21:04.838398 7f146d17ae00 1 bdev(0x55fabdc36900 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:21:04.838838 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
2018-01-11 10:21:04.838870 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
2018-01-11 10:21:04.838878 7f146d17ae00 1 bdev(0x55fabdc36900 /var/lib/ceph/osd/ceph-2/block) close
2018-01-11 10:21:05.119739 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _mount path /var/lib/ceph/osd/ceph-2
2018-01-11 10:21:05.119792 7f146d17ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:21:05.119798 7f146d17ae00 1 bdev(0x55fabdc36d80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:21:05.120121 7f146d17ae00 1 bdev(0x55fabdc36d80 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:21:05.120597 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
2018-01-11 10:21:05.120618 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
2018-01-11 10:21:05.120695 7f146d17ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block.db type kernel
2018-01-11 10:21:05.120706 7f146d17ae00 1 bdev(0x55fabdc36fc0 /var/lib/ceph/osd/ceph-2/block.db) open path /var/lib/ceph/osd/ceph-2/block.db
2018-01-11 10:21:05.120918 7f146d17ae00 1 bdev(0x55fabdc36fc0 /var/lib/ceph/osd/ceph-2/block.db) open size 29999759360 (0x6fc200000, 28610 MB) block_size 4096 (4096 B) non-rotational
2018-01-11 10:21:05.120927 7f146d17ae00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block.db size 28610 MB
2018-01-11 10:21:05.121223 7f146d17ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:21:05.121233 7f146d17ae00 1 bdev(0x55fabdc37200 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:21:05.121468 7f146d17ae00 1 bdev(0x55fabdc37200 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:21:05.121480 7f146d17ae00 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-2/block size 931 GB
2018-01-11 10:21:05.121521 7f146d17ae00 1 bluefs mount
2018-01-11 10:21:05.202584 7f146d17ae00 0 set rocksdb option compaction_readahead_size = 2097152
2018-01-11 10:21:05.202601 7f146d17ae00 0 set rocksdb option compression = kNoCompression
2018-01-11 10:21:05.202608 7f146d17ae00 0 set rocksdb option max_write_buffer_number = 4
2018-01-11 10:21:05.202613 7f146d17ae00 0 set rocksdb option min_write_buffer_number_to_merge = 1
2018-01-11 10:21:05.202617 7f146d17ae00 0 set rocksdb option recycle_log_file_num = 4
2018-01-11 10:21:05.202622 7f146d17ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2018-01-11 10:21:05.202627 7f146d17ae00 0 set rocksdb option write_buffer_size = 268435456
2018-01-11 10:21:05.202680 7f146d17ae00 0 set rocksdb option compaction_readahead_size = 2097152
2018-01-11 10:21:05.202686 7f146d17ae00 0 set rocksdb option compression = kNoCompression
2018-01-11 10:21:05.202690 7f146d17ae00 0 set rocksdb option max_write_buffer_number = 4
2018-01-11 10:21:05.202694 7f146d17ae00 0 set rocksdb option min_write_buffer_number_to_merge = 1
2018-01-11 10:21:05.202698 7f146d17ae00 0 set rocksdb option recycle_log_file_num = 4
2018-01-11 10:21:05.202701 7f146d17ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2018-01-11 10:21:05.202705 7f146d17ae00 0 set rocksdb option write_buffer_size = 268435456
2018-01-11 10:21:05.202889 7f146d17ae00 4 rocksdb: RocksDB version: 5.4.0

2018-01-11 10:21:05.202904 7f146d17ae00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
2018-01-11 10:21:05.202906 7f146d17ae00 4 rocksdb: Compile date Sep 26 2017
2018-01-11 10:21:05.202908 7f146d17ae00 4 rocksdb: DB SUMMARY

2018-01-11 10:21:05.202931 7f146d17ae00 4 rocksdb: CURRENT file: CURRENT

2018-01-11 10:21:05.202933 7f146d17ae00 4 rocksdb: IDENTITY file: IDENTITY

2018-01-11 10:21:05.202935 7f146d17ae00 4 rocksdb: MANIFEST file: MANIFEST-016213 size: 69462 Bytes

2018-01-11 10:21:05.202939 7f146d17ae00 4 rocksdb: SST files in db dir, Total Num: 7, files: 015138.sst 015139.sst 015140.sst 016206.sst 016207.sst 016209.sst 016212.sst

2018-01-11 10:21:05.203198 7f146d17ae00 4 rocksdb: SST files in db.slow dir, Total Num: 378, files: 007717.sst 007748.sst 008161.sst 008669.sst 008670.sst 009336.sst 009514.sst 009515.sst 009516.sst

2018-01-11 10:21:05.203204 7f146d17ae00 4 rocksdb: Write Ahead Log file in db: 016214.log size: 618187 ;

2018-01-11 10:21:05.203207 7f146d17ae00 4 rocksdb: Options.error_if_exists: 0
2018-01-11 10:21:05.203210 7f146d17ae00 4 rocksdb: Options.create_if_missing: 0
2018-01-11 10:21:05.203211 7f146d17ae00 4 rocksdb: Options.paranoid_checks: 1
2018-01-11 10:21:05.203212 7f146d17ae00 4 rocksdb: Options.env: 0x55fabdf8bfc0
2018-01-11 10:21:05.203213 7f146d17ae00 4 rocksdb: Options.info_log: 0x55fabe1a5460
2018-01-11 10:21:05.203214 7f146d17ae00 4 rocksdb: Options.max_open_files: -1
2018-01-11 10:21:05.203215 7f146d17ae00 4 rocksdb: Options.max_file_opening_threads: 16
2018-01-11 10:21:05.203216 7f146d17ae00 4 rocksdb: Options.use_fsync: 0
2018-01-11 10:21:05.203217 7f146d17ae00 4 rocksdb: Options.max_log_file_size: 0
2018-01-11 10:21:05.203219 7f146d17ae00 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
2018-01-11 10:21:05.203220 7f146d17ae00 4 rocksdb: Options.log_file_time_to_roll: 0
2018-01-11 10:21:05.203221 7f146d17ae00 4 rocksdb: Options.keep_log_file_num: 1000
2018-01-11 10:21:05.203222 7f146d17ae00 4 rocksdb: Options.recycle_log_file_num: 4
2018-01-11 10:21:05.203223 7f146d17ae00 4 rocksdb: Options.allow_fallocate: 1
2018-01-11 10:21:05.203224 7f146d17ae00 4 rocksdb: Options.allow_mmap_reads: 0
2018-01-11 10:21:05.203225 7f146d17ae00 4 rocksdb: Options.allow_mmap_writes: 0
2018-01-11 10:21:05.203226 7f146d17ae00 4 rocksdb: Options.use_direct_reads: 0
2018-01-11 10:21:05.203227 7f146d17ae00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
2018-01-11 10:21:05.203228 7f146d17ae00 4 rocksdb: Options.create_missing_column_families: 0
2018-01-11 10:21:05.203229 7f146d17ae00 4 rocksdb: Options.db_log_dir:
2018-01-11 10:21:05.203230 7f146d17ae00 4 rocksdb: Options.wal_dir: db
2018-01-11 10:21:05.203231 7f146d17ae00 4 rocksdb: Options.table_cache_numshardbits: 6
2018-01-11 10:21:05.203233 7f146d17ae00 4 rocksdb: Options.max_subcompactions: 1
2018-01-11 10:21:05.203234 7f146d17ae00 4 rocksdb: Options.max_background_flushes: 1
2018-01-11 10:21:05.203235 7f146d17ae00 4 rocksdb: Options.WAL_ttl_seconds: 0
2018-01-11 10:21:05.203236 7f146d17ae00 4 rocksdb: Options.WAL_size_limit_MB: 0
2018-01-11 10:21:05.203237 7f146d17ae00 4 rocksdb: Options.manifest_preallocation_size: 4194304
2018-01-11 10:21:05.203239 7f146d17ae00 4 rocksdb: Options.is_fd_close_on_exec: 1
2018-01-11 10:21:05.203240 7f146d17ae00 4 rocksdb: Options.advise_random_on_open: 1
2018-01-11 10:21:05.203242 7f146d17ae00 4 rocksdb: Options.db_write_buffer_size: 0
2018-01-11 10:21:05.203243 7f146d17ae00 4 rocksdb: Options.access_hint_on_compaction_start: 1
2018-01-11 10:21:05.203244 7f146d17ae00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
2018-01-11 10:21:05.203246 7f146d17ae00 4 rocksdb: Options.compaction_readahead_size: 2097152
2018-01-11 10:21:05.203247 7f146d17ae00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
2018-01-11 10:21:05.203248 7f146d17ae00 4 rocksdb: Options.writable_file_max_buffer_size: 0
2018-01-11 10:21:05.203249 7f146d17ae00 4 rocksdb: Options.use_adaptive_mutex: 0
2018-01-11 10:21:05.203250 7f146d17ae00 4 rocksdb: Options.rate_limiter: (nil)
2018-01-11 10:21:05.203251 7f146d17ae00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
2018-01-11 10:21:05.203252 7f146d17ae00 4 rocksdb: Options.bytes_per_sync: 0
2018-01-11 10:21:05.203253 7f146d17ae00 4 rocksdb: Options.wal_bytes_per_sync: 0
2018-01-11 10:21:05.203255 7f146d17ae00 4 rocksdb: Options.wal_recovery_mode: 0
2018-01-11 10:21:05.203256 7f146d17ae00 4 rocksdb: Options.enable_thread_tracking: 0
2018-01-11 10:21:05.203257 7f146d17ae00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
2018-01-11 10:21:05.203258 7f146d17ae00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
2018-01-11 10:21:05.203259 7f146d17ae00 4 rocksdb: Options.write_thread_max_yield_usec: 100
2018-01-11 10:21:05.203260 7f146d17ae00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
2018-01-11 10:21:05.203261 7f146d17ae00 4 rocksdb: Options.row_cache: None
2018-01-11 10:21:05.203262 7f146d17ae00 4 rocksdb: Options.wal_filter: None
2018-01-11 10:21:05.203264 7f146d17ae00 4 rocksdb: Options.avoid_flush_during_recovery: 0
2018-01-11 10:21:05.203265 7f146d17ae00 4 rocksdb: Options.base_background_compactions: 1
2018-01-11 10:21:05.203266 7f146d17ae00 4 rocksdb: Options.max_background_compactions: 1
2018-01-11 10:21:05.203267 7f146d17ae00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
2018-01-11 10:21:05.203268 7f146d17ae00 4 rocksdb: Options.delayed_write_rate : 16777216
2018-01-11 10:21:05.203269 7f146d17ae00 4 rocksdb: Options.max_total_wal_size: 0
2018-01-11 10:21:05.203270 7f146d17ae00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
2018-01-11 10:21:05.203280 7f146d17ae00 4 rocksdb: Options.stats_dump_period_sec: 600
2018-01-11 10:21:05.203282 7f146d17ae00 4 rocksdb: Compression algorithms supported:
2018-01-11 10:21:05.203283 7f146d17ae00 4 rocksdb: Snappy supported: 0
2018-01-11 10:21:05.203284 7f146d17ae00 4 rocksdb: Zlib supported: 0
2018-01-11 10:21:05.203285 7f146d17ae00 4 rocksdb: Bzip supported: 0
2018-01-11 10:21:05.203286 7f146d17ae00 4 rocksdb: LZ4 supported: 0
2018-01-11 10:21:05.203287 7f146d17ae00 4 rocksdb: ZSTD supported: 0
2018-01-11 10:21:05.203288 7f146d17ae00 4 rocksdb: Fast CRC32 supported: 0
2018-01-11 10:21:05.203457 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2609] Recovering from manifest file: MANIFEST-016213

2018-01-11 10:21:05.203513 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/column_family.cc:407] --------------- Options for column family [default]:

2018-01-11 10:21:05.203522 7f146d17ae00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
2018-01-11 10:21:05.203528 7f146d17ae00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
2018-01-11 10:21:05.203530 7f146d17ae00 4 rocksdb: Options.compaction_filter: None
2018-01-11 10:21:05.203531 7f146d17ae00 4 rocksdb: Options.compaction_filter_factory: None
2018-01-11 10:21:05.203532 7f146d17ae00 4 rocksdb: Options.memtable_factory: SkipListFactory
2018-01-11 10:21:05.203533 7f146d17ae00 4 rocksdb: Options.table_factory: BlockBasedTable
2018-01-11 10:21:05.203556 7f146d17ae00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fabdbc6170)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 1
pin_l0_filter_and_index_blocks_in_cache: 1
index_type: 0
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x55fabdeb8e70
block_cache_name: LRUCache
block_cache_options:
capacity : 536870912
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
format_version: 2

2018-01-11 10:21:05.203565 7f146d17ae00 4 rocksdb: Options.write_buffer_size: 268435456
2018-01-11 10:21:05.203566 7f146d17ae00 4 rocksdb: Options.max_write_buffer_number: 4
2018-01-11 10:21:05.203568 7f146d17ae00 4 rocksdb: Options.compression: NoCompression
2018-01-11 10:21:05.203569 7f146d17ae00 4 rocksdb: Options.bottommost_compression: Disabled
2018-01-11 10:21:05.203571 7f146d17ae00 4 rocksdb: Options.prefix_extractor: nullptr
2018-01-11 10:21:05.203572 7f146d17ae00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
2018-01-11 10:21:05.203573 7f146d17ae00 4 rocksdb: Options.num_levels: 7
2018-01-11 10:21:05.203574 7f146d17ae00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
2018-01-11 10:21:05.203575 7f146d17ae00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
2018-01-11 10:21:05.203576 7f146d17ae00 4 rocksdb: Options.compression_opts.window_bits: -14
2018-01-11 10:21:05.203577 7f146d17ae00 4 rocksdb: Options.compression_opts.level: -1
2018-01-11 10:21:05.203578 7f146d17ae00 4 rocksdb: Options.compression_opts.strategy: 0
2018-01-11 10:21:05.203579 7f146d17ae00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
2018-01-11 10:21:05.203581 7f146d17ae00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
2018-01-11 10:21:05.203582 7f146d17ae00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
2018-01-11 10:21:05.203583 7f146d17ae00 4 rocksdb: Options.level0_stop_writes_trigger: 36
2018-01-11 10:21:05.203584 7f146d17ae00 4 rocksdb: Options.target_file_size_base: 67108864
2018-01-11 10:21:05.203586 7f146d17ae00 4 rocksdb: Options.target_file_size_multiplier: 1
2018-01-11 10:21:05.203587 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
2018-01-11 10:21:05.203588 7f146d17ae00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
2018-01-11 10:21:05.203589 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
2018-01-11 10:21:05.203593 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
2018-01-11 10:21:05.203594 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
2018-01-11 10:21:05.203595 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
2018-01-11 10:21:05.203596 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
2018-01-11 10:21:05.203597 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
2018-01-11 10:21:05.203598 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
2018-01-11 10:21:05.203599 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
2018-01-11 10:21:05.203601 7f146d17ae00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
2018-01-11 10:21:05.203602 7f146d17ae00 4 rocksdb: Options.max_compaction_bytes: 1677721600
2018-01-11 10:21:05.203603 7f146d17ae00 4 rocksdb: Options.arena_block_size: 33554432
2018-01-11 10:21:05.203604 7f146d17ae00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
2018-01-11 10:21:05.203605 7f146d17ae00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
2018-01-11 10:21:05.203606 7f146d17ae00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
2018-01-11 10:21:05.203607 7f146d17ae00 4 rocksdb: Options.disable_auto_compactions: 0
2018-01-11 10:21:05.203609 7f146d17ae00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
2018-01-11 10:21:05.203611 7f146d17ae00 4 rocksdb: Options.compaction_pri: kByCompensatedSize
2018-01-11 10:21:05.203612 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
2018-01-11 10:21:05.203613 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
2018-01-11 10:21:05.203614 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
2018-01-11 10:21:05.203615 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
2018-01-11 10:21:05.203616 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
2018-01-11 10:21:05.203618 7f146d17ae00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
2018-01-11 10:21:05.203629 7f146d17ae00 4 rocksdb: Options.table_properties_collectors:
2018-01-11 10:21:05.203631 7f146d17ae00 4 rocksdb: Options.inplace_update_support: 0
2018-01-11 10:21:05.203632 7f146d17ae00 4 rocksdb: Options.inplace_update_num_locks: 10000
2018-01-11 10:21:05.203633 7f146d17ae00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
2018-01-11 10:21:05.203636 7f146d17ae00 4 rocksdb: Options.memtable_huge_page_size: 0
2018-01-11 10:21:05.203637 7f146d17ae00 4 rocksdb: Options.bloom_locality: 0
2018-01-11 10:21:05.203639 7f146d17ae00 4 rocksdb: Options.max_successive_merges: 0
2018-01-11 10:21:05.203640 7f146d17ae00 4 rocksdb: Options.optimize_filters_for_hits: 0
2018-01-11 10:21:05.203641 7f146d17ae00 4 rocksdb: Options.paranoid_file_checks: 0
2018-01-11 10:21:05.203642 7f146d17ae00 4 rocksdb: Options.force_consistency_checks: 0
2018-01-11 10:21:05.203644 7f146d17ae00 4 rocksdb: Options.report_bg_io_stats: 0
2018-01-11 10:21:09.608311 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2859] Recovered from manifest file:db/MANIFEST-016213 succeeded,manifest_file_number is 16213, next_file_number is 16215, last_sequence is 341926534, log_number is 0,prev_log_number is 0,max_column_family is 0

2018-01-11 10:21:09.608343 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2867] Column family [default] (ID 0), log number is 16212

2018-01-11 10:21:09.609478 7f146d17ae00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662469609462, "job": 1, "event": "recovery_started", "log_files": [16214]}
2018-01-11 10:21:09.609490 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:482] Recovering log #16214 mode 0
2018-01-11 10:21:09.642412 7f146d17ae00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662469642393, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 16215, "file_size": 373756, "table_properties": {"data_size": 355562, "index_size": 2855, "filter_size": 14363, "raw_key_size": 233484, "raw_average_key_size": 42, "raw_value_size": 161972, "raw_average_value_size": 29, "num_data_blocks": 88, "num_entries": 5431, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-01-11 10:21:09.642710 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2395] Creating manifest 16216

2018-01-11 10:21:09.644944 7f146d17ae00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662469644933, "job": 1, "event": "recovery_finished"}
2018-01-11 10:21:09.648917 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:1063] DB pointer 0x55fabdec7a00
2018-01-11 10:21:09.648988 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
2018-01-11 10:21:09.703883 7f146d17ae00 1 freelist init
2018-01-11 10:21:09.768744 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc opening allocation metadata
2018-01-11 10:21:19.465952 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc loaded 318 G in 658679 extents
2018-01-11 10:21:19.587443 7f146d17ae00 0 <cls> /build/ceph-12.2.1/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
2018-01-11 10:21:19.589388 7f146d17ae00 0 <cls> /build/ceph-12.2.1/src/cls/hello/cls_hello.cc:296: loading cls_hello
2018-01-11 10:21:19.589405 7f146d17ae00 0 _get_class not permitted to load kvs
2018-01-11 10:21:19.589538 7f146d17ae00 0 _get_class not permitted to load lua
2018-01-11 10:21:19.590489 7f146d17ae00 0 _get_class not permitted to load sdk
2018-01-11 10:21:19.622870 7f146d17ae00 0 osd.2 15344 crush map has features 2200130813952, adjusting msgr requires for clients
2018-01-11 10:21:19.622896 7f146d17ae00 0 osd.2 15344 crush map has features 2200130813952 was 8705, adjusting msgr requires for mons
2018-01-11 10:21:19.622905 7f146d17ae00 0 osd.2 15344 crush map has features 720578140510109696, adjusting msgr requires for osds
2018-01-11 10:21:35.904730 7f146d17ae00 0 osd.2 15344 load_pgs
2018-01-11 10:23:15.632872 7f146d17ae00 0 osd.2 15344 load_pgs opened 5429 pgs
2018-01-11 10:23:21.141023 7f146d17ae00 -1 /build/ceph-12.2.1/src/osd/osd_types.cc: In function 'virtual void pi_compact_rep::add_interval(bool, const PastIntervals::pg_interval_t&)' thread 7f146d17ae00 time 2018-01-11 10:23:21.136434
/build/ceph-12.2.1/src/osd/osd_types.cc: 3205: FAILED assert(interval.last > last)

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x55fab317c3f2]
2: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x55fab2e05b75]
3: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x55fab2de3376]
4: (OSD::build_past_intervals_parallel()+0xe71) [0x55fab2b98f01]
5: (OSD::load_pgs()+0x14fb) [0x55fab2b9b92b]
6: (OSD::init()+0x2217) [0x55fab2bb5b37]
7: (main()+0x2eb8) [0x55fab2ac61f8]
8: (__libc_start_main()+0xf0) [0x7f146a5eb830]
9: (_start()+0x29) [0x55fab2b51a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-248> 2018-01-11 10:21:04.808494 7f146d17ae00 5 asok(0x55fabdc172c0) register_command perfcounters_dump hook 0x55fabdbca1b0
-247> 2018-01-11 10:21:04.808519 7f146d17ae00 5 asok(0x55fabdc172c0) register_command 1 hook 0x55fabdbca1b0
-246> 2018-01-11 10:21:04.808525 7f146d17ae00 5 asok(0x55fabdc172c0) register_command perf dump hook 0x55fabdbca1b0
-245> 2018-01-11 10:21:04.808555 7f146d17ae00 5 asok(0x55fabdc172c0) register_command perfcounters_schema hook 0x55fabdbca1b0
-244> 2018-01-11 10:21:04.808564 7f146d17ae00 5 asok(0x55fabdc172c0) register_command perf histogram dump hook 0x55fabdbca1b0
-243> 2018-01-11 10:21:04.808568 7f146d17ae00 5 asok(0x55fabdc172c0) register_command 2 hook 0x55fabdbca1b0
-242> 2018-01-11 10:21:04.808575 7f146d17ae00 5 asok(0x55fabdc172c0) register_command perf schema hook 0x55fabdbca1b0
-241> 2018-01-11 10:21:04.808580 7f146d17ae00 5 asok(0x55fabdc172c0) register_command perf histogram schema hook 0x55fabdbca1b0
-240> 2018-01-11 10:21:04.808588 7f146d17ae00 5 asok(0x55fabdc172c0) register_command perf reset hook 0x55fabdbca1b0
-239> 2018-01-11 10:21:04.808594 7f146d17ae00 5 asok(0x55fabdc172c0) register_command config show hook 0x55fabdbca1b0
-238> 2018-01-11 10:21:04.808601 7f146d17ae00 5 asok(0x55fabdc172c0) register_command config help hook 0x55fabdbca1b0
-237> 2018-01-11 10:21:04.808609 7f146d17ae00 5 asok(0x55fabdc172c0) register_command config set hook 0x55fabdbca1b0
-236> 2018-01-11 10:21:04.808617 7f146d17ae00 5 asok(0x55fabdc172c0) register_command config get hook 0x55fabdbca1b0
-235> 2018-01-11 10:21:04.808622 7f146d17ae00 5 asok(0x55fabdc172c0) register_command config diff hook 0x55fabdbca1b0
-234> 2018-01-11 10:21:04.808630 7f146d17ae00 5 asok(0x55fabdc172c0) register_command config diff get hook 0x55fabdbca1b0
-233> 2018-01-11 10:21:04.808637 7f146d17ae00 5 asok(0x55fabdc172c0) register_command log flush hook 0x55fabdbca1b0
-232> 2018-01-11 10:21:04.808645 7f146d17ae00 5 asok(0x55fabdc172c0) register_command log dump hook 0x55fabdbca1b0
-231> 2018-01-11 10:21:04.808652 7f146d17ae00 5 asok(0x55fabdc172c0) register_command log reopen hook 0x55fabdbca1b0
-230> 2018-01-11 10:21:04.808670 7f146d17ae00 5 asok(0x55fabdc172c0) register_command dump_mempools hook 0x55fabdeb2b68
-229> 2018-01-11 10:21:04.820311 7f146d17ae00 0 set uid:gid to 64045:64045 (ceph:ceph)
-228> 2018-01-11 10:21:04.820336 7f146d17ae00 0 ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable), process (unknown), pid 1935869
-227> 2018-01-11 10:21:04.827145 7f146d17ae00 0 pidfile_write: ignore empty --pid-file
-226> 2018-01-11 10:21:04.838000 7f146d17ae00 0 load: jerasure load: lrc load: isa
-225> 2018-01-11 10:21:04.838088 7f146d17ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-224> 2018-01-11 10:21:04.838099 7f146d17ae00 1 bdev(0x55fabdc36900 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-223> 2018-01-11 10:21:04.838398 7f146d17ae00 1 bdev(0x55fabdc36900 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
-222> 2018-01-11 10:21:04.838838 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
-221> 2018-01-11 10:21:04.838870 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
-220> 2018-01-11 10:21:04.838878 7f146d17ae00 1 bdev(0x55fabdc36900 /var/lib/ceph/osd/ceph-2/block) close
-219> 2018-01-11 10:21:05.119739 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _mount path /var/lib/ceph/osd/ceph-2
-218> 2018-01-11 10:21:05.119792 7f146d17ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-217> 2018-01-11 10:21:05.119798 7f146d17ae00 1 bdev(0x55fabdc36d80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-216> 2018-01-11 10:21:05.120121 7f146d17ae00 1 bdev(0x55fabdc36d80 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
-215> 2018-01-11 10:21:05.120597 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
-214> 2018-01-11 10:21:05.120618 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
-213> 2018-01-11 10:21:05.120695 7f146d17ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block.db type kernel
-212> 2018-01-11 10:21:05.120706 7f146d17ae00 1 bdev(0x55fabdc36fc0 /var/lib/ceph/osd/ceph-2/block.db) open path /var/lib/ceph/osd/ceph-2/block.db
-211> 2018-01-11 10:21:05.120918 7f146d17ae00 1 bdev(0x55fabdc36fc0 /var/lib/ceph/osd/ceph-2/block.db) open size 29999759360 (0x6fc200000, 28610 MB) block_size 4096 (4096 B) non-rotational
-210> 2018-01-11 10:21:05.120927 7f146d17ae00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block.db size 28610 MB
-209> 2018-01-11 10:21:05.121223 7f146d17ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-208> 2018-01-11 10:21:05.121233 7f146d17ae00 1 bdev(0x55fabdc37200 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-207> 2018-01-11 10:21:05.121468 7f146d17ae00 1 bdev(0x55fabdc37200 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
-206> 2018-01-11 10:21:05.121480 7f146d17ae00 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-2/block size 931 GB
-205> 2018-01-11 10:21:05.121521 7f146d17ae00 1 bluefs mount
-204> 2018-01-11 10:21:05.202584 7f146d17ae00 0 set rocksdb option compaction_readahead_size = 2097152
-203> 2018-01-11 10:21:05.202601 7f146d17ae00 0 set rocksdb option compression = kNoCompression
-202> 2018-01-11 10:21:05.202608 7f146d17ae00 0 set rocksdb option max_write_buffer_number = 4
-201> 2018-01-11 10:21:05.202613 7f146d17ae00 0 set rocksdb option min_write_buffer_number_to_merge = 1
-200> 2018-01-11 10:21:05.202617 7f146d17ae00 0 set rocksdb option recycle_log_file_num = 4
-199> 2018-01-11 10:21:05.202622 7f146d17ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-198> 2018-01-11 10:21:05.202627 7f146d17ae00 0 set rocksdb option write_buffer_size = 268435456
-197> 2018-01-11 10:21:05.202680 7f146d17ae00 0 set rocksdb option compaction_readahead_size = 2097152
-196> 2018-01-11 10:21:05.202686 7f146d17ae00 0 set rocksdb option compression = kNoCompression
-195> 2018-01-11 10:21:05.202690 7f146d17ae00 0 set rocksdb option max_write_buffer_number = 4
-194> 2018-01-11 10:21:05.202694 7f146d17ae00 0 set rocksdb option min_write_buffer_number_to_merge = 1
-193> 2018-01-11 10:21:05.202698 7f146d17ae00 0 set rocksdb option recycle_log_file_num = 4
-192> 2018-01-11 10:21:05.202701 7f146d17ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-191> 2018-01-11 10:21:05.202705 7f146d17ae00 0 set rocksdb option write_buffer_size = 268435456
-190> 2018-01-11 10:21:05.202889 7f146d17ae00 4 rocksdb: RocksDB version: 5.4.0

-189> 2018-01-11 10:21:05.202904 7f146d17ae00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
-188> 2018-01-11 10:21:05.202906 7f146d17ae00 4 rocksdb: Compile date Sep 26 2017
-187> 2018-01-11 10:21:05.202908 7f146d17ae00 4 rocksdb: DB SUMMARY

-186> 2018-01-11 10:21:05.202931 7f146d17ae00 4 rocksdb: CURRENT file: CURRENT

-185> 2018-01-11 10:21:05.202933 7f146d17ae00 4 rocksdb: IDENTITY file: IDENTITY

-184> 2018-01-11 10:21:05.202935 7f146d17ae00 4 rocksdb: MANIFEST file: MANIFEST-016213 size: 69462 Bytes

-183> 2018-01-11 10:21:05.202939 7f146d17ae00 4 rocksdb: SST files in db dir, Total Num: 7, files: 015138.sst 015139.sst 015140.sst 016206.sst 016207.sst 016209.sst 016212.sst

-182> 2018-01-11 10:21:05.203198 7f146d17ae00 4 rocksdb: SST files in db.slow dir, Total Num: 378, files: 007717.sst 007748.sst 008161.sst 008669.sst 008670.sst 009336.sst 009514.sst 009515.sst 009516.sst

-181> 2018-01-11 10:21:05.203204 7f146d17ae00 4 rocksdb: Write Ahead Log file in db: 016214.log size: 618187 ;

-180> 2018-01-11 10:21:05.203207 7f146d17ae00 4 rocksdb: Options.error_if_exists: 0
-179> 2018-01-11 10:21:05.203210 7f146d17ae00 4 rocksdb: Options.create_if_missing: 0
-178> 2018-01-11 10:21:05.203211 7f146d17ae00 4 rocksdb: Options.paranoid_checks: 1
-177> 2018-01-11 10:21:05.203212 7f146d17ae00 4 rocksdb: Options.env: 0x55fabdf8bfc0
-176> 2018-01-11 10:21:05.203213 7f146d17ae00 4 rocksdb: Options.info_log: 0x55fabe1a5460
-175> 2018-01-11 10:21:05.203214 7f146d17ae00 4 rocksdb: Options.max_open_files: -1
-174> 2018-01-11 10:21:05.203215 7f146d17ae00 4 rocksdb: Options.max_file_opening_threads: 16
-173> 2018-01-11 10:21:05.203216 7f146d17ae00 4 rocksdb: Options.use_fsync: 0
-172> 2018-01-11 10:21:05.203217 7f146d17ae00 4 rocksdb: Options.max_log_file_size: 0
-171> 2018-01-11 10:21:05.203219 7f146d17ae00 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
-170> 2018-01-11 10:21:05.203220 7f146d17ae00 4 rocksdb: Options.log_file_time_to_roll: 0
-169> 2018-01-11 10:21:05.203221 7f146d17ae00 4 rocksdb: Options.keep_log_file_num: 1000
-168> 2018-01-11 10:21:05.203222 7f146d17ae00 4 rocksdb: Options.recycle_log_file_num: 4
-167> 2018-01-11 10:21:05.203223 7f146d17ae00 4 rocksdb: Options.allow_fallocate: 1
-166> 2018-01-11 10:21:05.203224 7f146d17ae00 4 rocksdb: Options.allow_mmap_reads: 0
-165> 2018-01-11 10:21:05.203225 7f146d17ae00 4 rocksdb: Options.allow_mmap_writes: 0
-164> 2018-01-11 10:21:05.203226 7f146d17ae00 4 rocksdb: Options.use_direct_reads: 0
-163> 2018-01-11 10:21:05.203227 7f146d17ae00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
-162> 2018-01-11 10:21:05.203228 7f146d17ae00 4 rocksdb: Options.create_missing_column_families: 0
-161> 2018-01-11 10:21:05.203229 7f146d17ae00 4 rocksdb: Options.db_log_dir:
-160> 2018-01-11 10:21:05.203230 7f146d17ae00 4 rocksdb: Options.wal_dir: db
-159> 2018-01-11 10:21:05.203231 7f146d17ae00 4 rocksdb: Options.table_cache_numshardbits: 6
-158> 2018-01-11 10:21:05.203233 7f146d17ae00 4 rocksdb: Options.max_subcompactions: 1
-157> 2018-01-11 10:21:05.203234 7f146d17ae00 4 rocksdb: Options.max_background_flushes: 1
-156> 2018-01-11 10:21:05.203235 7f146d17ae00 4 rocksdb: Options.WAL_ttl_seconds: 0
-155> 2018-01-11 10:21:05.203236 7f146d17ae00 4 rocksdb: Options.WAL_size_limit_MB: 0
-154> 2018-01-11 10:21:05.203237 7f146d17ae00 4 rocksdb: Options.manifest_preallocation_size: 4194304
-153> 2018-01-11 10:21:05.203239 7f146d17ae00 4 rocksdb: Options.is_fd_close_on_exec: 1
-152> 2018-01-11 10:21:05.203240 7f146d17ae00 4 rocksdb: Options.advise_random_on_open: 1
-151> 2018-01-11 10:21:05.203242 7f146d17ae00 4 rocksdb: Options.db_write_buffer_size: 0
-150> 2018-01-11 10:21:05.203243 7f146d17ae00 4 rocksdb: Options.access_hint_on_compaction_start: 1
-149> 2018-01-11 10:21:05.203244 7f146d17ae00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
-148> 2018-01-11 10:21:05.203246 7f146d17ae00 4 rocksdb: Options.compaction_readahead_size: 2097152
-147> 2018-01-11 10:21:05.203247 7f146d17ae00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
-146> 2018-01-11 10:21:05.203248 7f146d17ae00 4 rocksdb: Options.writable_file_max_buffer_size: 0
-145> 2018-01-11 10:21:05.203249 7f146d17ae00 4 rocksdb: Options.use_adaptive_mutex: 0
-144> 2018-01-11 10:21:05.203250 7f146d17ae00 4 rocksdb: Options.rate_limiter: (nil)
-143> 2018-01-11 10:21:05.203251 7f146d17ae00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
-142> 2018-01-11 10:21:05.203252 7f146d17ae00 4 rocksdb: Options.bytes_per_sync: 0
-141> 2018-01-11 10:21:05.203253 7f146d17ae00 4 rocksdb: Options.wal_bytes_per_sync: 0
-140> 2018-01-11 10:21:05.203255 7f146d17ae00 4 rocksdb: Options.wal_recovery_mode: 0
-139> 2018-01-11 10:21:05.203256 7f146d17ae00 4 rocksdb: Options.enable_thread_tracking: 0
-138> 2018-01-11 10:21:05.203257 7f146d17ae00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
-137> 2018-01-11 10:21:05.203258 7f146d17ae00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
-136> 2018-01-11 10:21:05.203259 7f146d17ae00 4 rocksdb: Options.write_thread_max_yield_usec: 100
-135> 2018-01-11 10:21:05.203260 7f146d17ae00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
-134> 2018-01-11 10:21:05.203261 7f146d17ae00 4 rocksdb: Options.row_cache: None
-133> 2018-01-11 10:21:05.203262 7f146d17ae00 4 rocksdb: Options.wal_filter: None
-132> 2018-01-11 10:21:05.203264 7f146d17ae00 4 rocksdb: Options.avoid_flush_during_recovery: 0
-131> 2018-01-11 10:21:05.203265 7f146d17ae00 4 rocksdb: Options.base_background_compactions: 1
-130> 2018-01-11 10:21:05.203266 7f146d17ae00 4 rocksdb: Options.max_background_compactions: 1
-129> 2018-01-11 10:21:05.203267 7f146d17ae00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
-128> 2018-01-11 10:21:05.203268 7f146d17ae00 4 rocksdb: Options.delayed_write_rate : 16777216
-127> 2018-01-11 10:21:05.203269 7f146d17ae00 4 rocksdb: Options.max_total_wal_size: 0
-126> 2018-01-11 10:21:05.203270 7f146d17ae00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
-125> 2018-01-11 10:21:05.203280 7f146d17ae00 4 rocksdb: Options.stats_dump_period_sec: 600
-124> 2018-01-11 10:21:05.203282 7f146d17ae00 4 rocksdb: Compression algorithms supported:
-123> 2018-01-11 10:21:05.203283 7f146d17ae00 4 rocksdb: Snappy supported: 0
-122> 2018-01-11 10:21:05.203284 7f146d17ae00 4 rocksdb: Zlib supported: 0
-121> 2018-01-11 10:21:05.203285 7f146d17ae00 4 rocksdb: Bzip supported: 0
-120> 2018-01-11 10:21:05.203286 7f146d17ae00 4 rocksdb: LZ4 supported: 0
-119> 2018-01-11 10:21:05.203287 7f146d17ae00 4 rocksdb: ZSTD supported: 0
-118> 2018-01-11 10:21:05.203288 7f146d17ae00 4 rocksdb: Fast CRC32 supported: 0
-117> 2018-01-11 10:21:05.203457 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2609] Recovering from manifest file: MANIFEST-016213

-116> 2018-01-11 10:21:05.203513 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/column_family.cc:407] --------------- Options for column family [default]:

-115> 2018-01-11 10:21:05.203522 7f146d17ae00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
-114> 2018-01-11 10:21:05.203528 7f146d17ae00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
-113> 2018-01-11 10:21:05.203530 7f146d17ae00 4 rocksdb: Options.compaction_filter: None
-112> 2018-01-11 10:21:05.203531 7f146d17ae00 4 rocksdb: Options.compaction_filter_factory: None
-111> 2018-01-11 10:21:05.203532 7f146d17ae00 4 rocksdb: Options.memtable_factory: SkipListFactory
-110> 2018-01-11 10:21:05.203533 7f146d17ae00 4 rocksdb: Options.table_factory: BlockBasedTable
-109> 2018-01-11 10:21:05.203556 7f146d17ae00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fabdbc6170)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 1
pin_l0_filter_and_index_blocks_in_cache: 1
index_type: 0
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x55fabdeb8e70
block_cache_name: LRUCache
block_cache_options:
capacity : 536870912
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
format_version: 2

-108> 2018-01-11 10:21:05.203565 7f146d17ae00 4 rocksdb: Options.write_buffer_size: 268435456
-107> 2018-01-11 10:21:05.203566 7f146d17ae00 4 rocksdb: Options.max_write_buffer_number: 4
-106> 2018-01-11 10:21:05.203568 7f146d17ae00 4 rocksdb: Options.compression: NoCompression
-105> 2018-01-11 10:21:05.203569 7f146d17ae00 4 rocksdb: Options.bottommost_compression: Disabled
-104> 2018-01-11 10:21:05.203571 7f146d17ae00 4 rocksdb: Options.prefix_extractor: nullptr
-103> 2018-01-11 10:21:05.203572 7f146d17ae00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
-102> 2018-01-11 10:21:05.203573 7f146d17ae00 4 rocksdb: Options.num_levels: 7
-101> 2018-01-11 10:21:05.203574 7f146d17ae00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
-100> 2018-01-11 10:21:05.203575 7f146d17ae00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
-99> 2018-01-11 10:21:05.203576 7f146d17ae00 4 rocksdb: Options.compression_opts.window_bits: -14
-98> 2018-01-11 10:21:05.203577 7f146d17ae00 4 rocksdb: Options.compression_opts.level: -1
-97> 2018-01-11 10:21:05.203578 7f146d17ae00 4 rocksdb: Options.compression_opts.strategy: 0
-96> 2018-01-11 10:21:05.203579 7f146d17ae00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
-95> 2018-01-11 10:21:05.203581 7f146d17ae00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
-94> 2018-01-11 10:21:05.203582 7f146d17ae00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
-93> 2018-01-11 10:21:05.203583 7f146d17ae00 4 rocksdb: Options.level0_stop_writes_trigger: 36
-92> 2018-01-11 10:21:05.203584 7f146d17ae00 4 rocksdb: Options.target_file_size_base: 67108864
-91> 2018-01-11 10:21:05.203586 7f146d17ae00 4 rocksdb: Options.target_file_size_multiplier: 1
-90> 2018-01-11 10:21:05.203587 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
-89> 2018-01-11 10:21:05.203588 7f146d17ae00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
-88> 2018-01-11 10:21:05.203589 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
-87> 2018-01-11 10:21:05.203593 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
-86> 2018-01-11 10:21:05.203594 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
-85> 2018-01-11 10:21:05.203595 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
-84> 2018-01-11 10:21:05.203596 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
-83> 2018-01-11 10:21:05.203597 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
-82> 2018-01-11 10:21:05.203598 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
-81> 2018-01-11 10:21:05.203599 7f146d17ae00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
-80> 2018-01-11 10:21:05.203601 7f146d17ae00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
-79> 2018-01-11 10:21:05.203602 7f146d17ae00 4 rocksdb: Options.max_compaction_bytes: 1677721600
-78> 2018-01-11 10:21:05.203603 7f146d17ae00 4 rocksdb: Options.arena_block_size: 33554432
-77> 2018-01-11 10:21:05.203604 7f146d17ae00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
-76> 2018-01-11 10:21:05.203605 7f146d17ae00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
-75> 2018-01-11 10:21:05.203606 7f146d17ae00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
-74> 2018-01-11 10:21:05.203607 7f146d17ae00 4 rocksdb: Options.disable_auto_compactions: 0
-73> 2018-01-11 10:21:05.203609 7f146d17ae00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
-72> 2018-01-11 10:21:05.203611 7f146d17ae00 4 rocksdb: Options.compaction_pri: kByCompensatedSize
-71> 2018-01-11 10:21:05.203612 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
-70> 2018-01-11 10:21:05.203613 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
-69> 2018-01-11 10:21:05.203614 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
-68> 2018-01-11 10:21:05.203615 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
-67> 2018-01-11 10:21:05.203616 7f146d17ae00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
-66> 2018-01-11 10:21:05.203618 7f146d17ae00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
-65> 2018-01-11 10:21:05.203629 7f146d17ae00 4 rocksdb: Options.table_properties_collectors:
-64> 2018-01-11 10:21:05.203631 7f146d17ae00 4 rocksdb: Options.inplace_update_support: 0
-63> 2018-01-11 10:21:05.203632 7f146d17ae00 4 rocksdb: Options.inplace_update_num_locks: 10000
-62> 2018-01-11 10:21:05.203633 7f146d17ae00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
-61> 2018-01-11 10:21:05.203636 7f146d17ae00 4 rocksdb: Options.memtable_huge_page_size: 0
-60> 2018-01-11 10:21:05.203637 7f146d17ae00 4 rocksdb: Options.bloom_locality: 0
-59> 2018-01-11 10:21:05.203639 7f146d17ae00 4 rocksdb: Options.max_successive_merges: 0
-58> 2018-01-11 10:21:05.203640 7f146d17ae00 4 rocksdb: Options.optimize_filters_for_hits: 0
-57> 2018-01-11 10:21:05.203641 7f146d17ae00 4 rocksdb: Options.paranoid_file_checks: 0
-56> 2018-01-11 10:21:05.203642 7f146d17ae00 4 rocksdb: Options.force_consistency_checks: 0
-55> 2018-01-11 10:21:05.203644 7f146d17ae00 4 rocksdb: Options.report_bg_io_stats: 0
-54> 2018-01-11 10:21:05.643661 7f1466172700 1 do_command 'perf dump' '
-53> 2018-01-11 10:21:05.644045 7f1466172700 1 do_command 'perf dump' 'result is 16218 bytes
-52> 2018-01-11 10:21:09.608311 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2859] Recovered from manifest file:db/MANIFEST-016213 succeeded,manifest_file_number is 16213, next_file_number is 16215, last_sequence is 341926534, log_number is 0,prev_log_number is 0,max_column_family is 0

-51> 2018-01-11 10:21:09.608343 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2867] Column family [default] (ID 0), log number is 16212

-50> 2018-01-11 10:21:09.609478 7f146d17ae00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662469609462, "job": 1, "event": "recovery_started", "log_files": [16214]}
-49> 2018-01-11 10:21:09.609490 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:482] Recovering log #16214 mode 0
-48> 2018-01-11 10:21:09.635142 7f146d17ae00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:815] [default] [WriteLevel0TableForRecovery] Level-0 table #16215: started
-47> 2018-01-11 10:21:09.642412 7f146d17ae00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662469642393, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 16215, "file_size": 373756, "table_properties": {"data_size": 355562, "index_size": 2855, "filter_size": 14363, "raw_key_size": 233484, "raw_average_key_size": 42, "raw_value_size": 161972, "raw_average_value_size": 29, "num_data_blocks": 88, "num_entries": 5431, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
-46> 2018-01-11 10:21:09.642428 7f146d17ae00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:847] [default] [WriteLevel0TableForRecovery] Level-0 table #16215: 373756 bytes OK
-45> 2018-01-11 10:21:09.642710 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2395] Creating manifest 16216

-44> 2018-01-11 10:21:09.644944 7f146d17ae00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662469644933, "job": 1, "event": "recovery_finished"}
-43> 2018-01-11 10:21:09.645558 7f146d17ae00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_files.cc:307] [JOB 2] Delete db//MANIFEST-016213 type=3 #16213 -- OK

-42> 2018-01-11 10:21:09.645582 7f146d17ae00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_files.cc:307] [JOB 2] Delete db//016214.log type=0 #16214 -- OK

-41> 2018-01-11 10:21:09.648917 7f146d17ae00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:1063] DB pointer 0x55fabdec7a00
-40> 2018-01-11 10:21:09.648988 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
-39> 2018-01-11 10:21:09.703883 7f146d17ae00 1 freelist init
-38> 2018-01-11 10:21:09.768744 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc opening allocation metadata
-37> 2018-01-11 10:21:15.552377 7f1466172700 1 do_command 'perf dump' '
-36> 2018-01-11 10:21:15.552995 7f1466172700 1 do_command 'perf dump' 'result is 17482 bytes
-35> 2018-01-11 10:21:19.465952 7f146d17ae00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc loaded 318 G in 658679 extents
-34> 2018-01-11 10:21:19.587443 7f146d17ae00 0 <cls> /build/ceph-12.2.1/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
-33> 2018-01-11 10:21:19.589388 7f146d17ae00 0 <cls> /build/ceph-12.2.1/src/cls/hello/cls_hello.cc:296: loading cls_hello
-32> 2018-01-11 10:21:19.589405 7f146d17ae00 0 _get_class not permitted to load kvs
-31> 2018-01-11 10:21:19.589538 7f146d17ae00 0 _get_class not permitted to load lua
-30> 2018-01-11 10:21:19.590489 7f146d17ae00 0 _get_class not permitted to load sdk
-29> 2018-01-11 10:21:19.622870 7f146d17ae00 0 osd.2 15344 crush map has features 2200130813952, adjusting msgr requires for clients
-28> 2018-01-11 10:21:19.622896 7f146d17ae00 0 osd.2 15344 crush map has features 2200130813952 was 8705, adjusting msgr requires for mons
-27> 2018-01-11 10:21:19.622905 7f146d17ae00 0 osd.2 15344 crush map has features 720578140510109696, adjusting msgr requires for osds
-26> 2018-01-11 10:21:25.641090 7f1466172700 1 do_command 'perf dump' '
-25> 2018-01-11 10:21:25.641764 7f1466172700 1 do_command 'perf dump' 'result is 22053 bytes
-24> 2018-01-11 10:21:35.558129 7f1466172700 1 do_command 'perf dump' '
-23> 2018-01-11 10:21:35.558472 7f1466172700 1 do_command 'perf dump' 'result is 22053 bytes
-22> 2018-01-11 10:21:35.904730 7f146d17ae00 0 osd.2 15344 load_pgs
-21> 2018-01-11 10:21:45.600132 7f1466172700 1 do_command 'perf dump' '
-20> 2018-01-11 10:21:45.600786 7f1466172700 1 do_command 'perf dump' 'result is 22123 bytes
-19> 2018-01-11 10:21:55.629297 7f1466172700 1 do_command 'perf dump' '
-18> 2018-01-11 10:21:55.630052 7f1466172700 1 do_command 'perf dump' 'result is 22146 bytes
-17> 2018-01-11 10:22:05.634929 7f1466172700 1 do_command 'perf dump' '
-16> 2018-01-11 10:22:05.635413 7f1466172700 1 do_command 'perf dump' 'result is 22149 bytes
-15> 2018-01-11 10:22:15.596505 7f1466172700 1 do_command 'perf dump' '
-14> 2018-01-11 10:22:15.596891 7f1466172700 1 do_command 'perf dump' 'result is 22149 bytes
-13> 2018-01-11 10:22:25.673513 7f1466172700 1 do_command 'perf dump' '
-12> 2018-01-11 10:22:25.673848 7f1466172700 1 do_command 'perf dump' 'result is 22150 bytes
-11> 2018-01-11 10:22:35.747126 7f1466172700 1 do_command 'perf dump' '
-10> 2018-01-11 10:22:35.747510 7f1466172700 1 do_command 'perf dump' 'result is 22151 bytes
-9> 2018-01-11 10:22:45.657393 7f1466172700 1 do_command 'perf dump' '
-8> 2018-01-11 10:22:45.657742 7f1466172700 1 do_command 'perf dump' 'result is 22152 bytes
-7> 2018-01-11 10:22:55.642623 7f1466172700 1 do_command 'perf dump' '
-6> 2018-01-11 10:22:55.642982 7f1466172700 1 do_command 'perf dump' 'result is 22152 bytes
-5> 2018-01-11 10:23:05.722727 7f1466172700 1 do_command 'perf dump' '
-4> 2018-01-11 10:23:05.723129 7f1466172700 1 do_command 'perf dump' 'result is 22152 bytes
-3> 2018-01-11 10:23:15.632872 7f146d17ae00 0 osd.2 15344 load_pgs opened 5429 pgs
-2> 2018-01-11 10:23:15.783190 7f1466172700 1 do_command 'perf dump' '
-1> 2018-01-11 10:23:15.783546 7f1466172700 1 do_command 'perf dump' 'result is 22154 bytes
0> 2018-01-11 10:23:21.141023 7f146d17ae00 -1 /build/ceph-12.2.1/src/osd/osd_types.cc: In function 'virtual void pi_compact_rep::add_interval(bool, const PastIntervals::pg_interval_t&)' thread 7f146d17ae00 time 2018-01-11 10:23:21.136434
/build/ceph-12.2.1/src/osd/osd_types.cc: 3205: FAILED assert(interval.last > last)

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x55fab317c3f2]
2: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x55fab2e05b75]
3: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x55fab2de3376]
4: (OSD::build_past_intervals_parallel()+0xe71) [0x55fab2b98f01]
5: (OSD::load_pgs()+0x14fb) [0x55fab2b9b92b]
6: (OSD::init()+0x2217) [0x55fab2bb5b37]
7: (main()+0x2eb8) [0x55fab2ac61f8]
8: (__libc_start_main()+0xf0) [0x7f146a5eb830]
9: (_start()+0x29) [0x55fab2b51a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 0 lockdep
0/ 0 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 0 buffer
0/ 0 timer
0/ 0 filer
0/ 1 striper
0/ 0 objecter
0/ 0 rados
0/ 0 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 0 journaler
0/ 0 objectcacher
0/ 0 client
0/ 0 osd
0/ 0 optracker
0/ 0 objclass
0/ 0 filestore
0/ 0 journal
0/ 0 ms
0/ 0 mon
0/ 0 monc
0/ 0 paxos
0/ 0 tp
0/ 0 auth
1/ 5 crypto
0/ 0 finisher
1/ 5 heartbeatmap
0/ 0 perfcounter
0/ 0 rgw
1/10 civetweb
1/ 5 javaclient
0/ 0 asok
0/ 0 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2018-01-11 10:23:21.146507 7f146d17ae00 -1 *** Caught signal (Aborted) **
in thread 7f146d17ae00 thread_name:ceph-osd

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (()+0xa5a634) [0x55fab3139634]
2: (()+0x11390) [0x7f146b665390]
3: (gsignal()+0x38) [0x7f146a600428]
4: (abort()+0x16a) [0x7f146a60202a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x55fab317c57e]
6: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x55fab2e05b75]
7: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x55fab2de3376]
8: (OSD::build_past_intervals_parallel()+0xe71) [0x55fab2b98f01]
9: (OSD::load_pgs()+0x14fb) [0x55fab2b9b92b]
10: (OSD::init()+0x2217) [0x55fab2bb5b37]
11: (main()+0x2eb8) [0x55fab2ac61f8]
12: (__libc_start_main()+0xf0) [0x7f146a5eb830]
13: (_start()+0x29) [0x55fab2b51a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
0> 2018-01-11 10:23:21.146507 7f146d17ae00 -1 *** Caught signal (Aborted) **
in thread 7f146d17ae00 thread_name:ceph-osd

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (()+0xa5a634) [0x55fab3139634]
2: (()+0x11390) [0x7f146b665390]
3: (gsignal()+0x38) [0x7f146a600428]
4: (abort()+0x16a) [0x7f146a60202a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x55fab317c57e]
6: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x55fab2e05b75]
7: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x55fab2de3376]
8: (OSD::build_past_intervals_parallel()+0xe71) [0x55fab2b98f01]
9: (OSD::load_pgs()+0x14fb) [0x55fab2b9b92b]
10: (OSD::init()+0x2217) [0x55fab2bb5b37]
11: (main()+0x2eb8) [0x55fab2ac61f8]
12: (__libc_start_main()+0xf0) [0x7f146a5eb830]
13: (_start()+0x29) [0x55fab2b51a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 0 lockdep
0/ 0 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 0 buffer
0/ 0 timer
0/ 0 filer
0/ 1 striper
0/ 0 objecter
0/ 0 rados
0/ 0 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 0 journaler
0/ 0 objectcacher
0/ 0 client
0/ 0 osd
0/ 0 optracker
0/ 0 objclass
0/ 0 filestore
0/ 0 journal
0/ 0 ms
0/ 0 mon
0/ 0 monc
0/ 0 paxos
0/ 0 tp
0/ 0 auth
1/ 5 crypto
0/ 0 finisher
1/ 5 heartbeatmap
0/ 0 perfcounter
0/ 0 rgw
1/10 civetweb
1/ 5 javaclient
0/ 0 asok
0/ 0 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2018-01-11 10:23:41.783228 7f133607be00 0 set uid:gid to 64045:64045 (ceph:ceph)
2018-01-11 10:23:41.783264 7f133607be00 0 ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable), process (unknown), pid 1941516
2018-01-11 10:23:41.792755 7f133607be00 0 pidfile_write: ignore empty --pid-file
2018-01-11 10:23:41.807145 7f133607be00 0 load: jerasure load: lrc load: isa
2018-01-11 10:23:41.807271 7f133607be00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:23:41.807288 7f133607be00 1 bdev(0x55723f356900 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:23:41.807675 7f133607be00 1 bdev(0x55723f356900 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:23:41.808272 7f133607be00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
2018-01-11 10:23:41.808317 7f133607be00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
2018-01-11 10:23:41.808328 7f133607be00 1 bdev(0x55723f356900 /var/lib/ceph/osd/ceph-2/block) close
2018-01-11 10:23:42.095548 7f133607be00 1 bluestore(/var/lib/ceph/osd/ceph-2) _mount path /var/lib/ceph/osd/ceph-2
2018-01-11 10:23:42.095604 7f133607be00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:23:42.095610 7f133607be00 1 bdev(0x55723f356d80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:23:42.095887 7f133607be00 1 bdev(0x55723f356d80 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:23:42.096371 7f133607be00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
2018-01-11 10:23:42.096395 7f133607be00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
2018-01-11 10:23:42.096489 7f133607be00 1 bdev create path /var/lib/ceph/osd/ceph-2/block.db type kernel
2018-01-11 10:23:42.096499 7f133607be00 1 bdev(0x55723f356fc0 /var/lib/ceph/osd/ceph-2/block.db) open path /var/lib/ceph/osd/ceph-2/block.db
2018-01-11 10:23:42.096767 7f133607be00 1 bdev(0x55723f356fc0 /var/lib/ceph/osd/ceph-2/block.db) open size 29999759360 (0x6fc200000, 28610 MB) block_size 4096 (4096 B) non-rotational
2018-01-11 10:23:42.096779 7f133607be00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block.db size 28610 MB
2018-01-11 10:23:42.097026 7f133607be00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:23:42.097035 7f133607be00 1 bdev(0x55723f357200 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:23:42.097263 7f133607be00 1 bdev(0x55723f357200 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:23:42.097275 7f133607be00 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-2/block size 931 GB
2018-01-11 10:23:42.097311 7f133607be00 1 bluefs mount
2018-01-11 10:23:42.173961 7f133607be00 0 set rocksdb option compaction_readahead_size = 2097152
2018-01-11 10:23:42.173975 7f133607be00 0 set rocksdb option compression = kNoCompression
2018-01-11 10:23:42.173981 7f133607be00 0 set rocksdb option max_write_buffer_number = 4
2018-01-11 10:23:42.173996 7f133607be00 0 set rocksdb option min_write_buffer_number_to_merge = 1
2018-01-11 10:23:42.174012 7f133607be00 0 set rocksdb option recycle_log_file_num = 4
2018-01-11 10:23:42.174016 7f133607be00 0 set rocksdb option writable_file_max_buffer_size = 0
2018-01-11 10:23:42.174021 7f133607be00 0 set rocksdb option write_buffer_size = 268435456
2018-01-11 10:23:42.174046 7f133607be00 0 set rocksdb option compaction_readahead_size = 2097152
2018-01-11 10:23:42.174051 7f133607be00 0 set rocksdb option compression = kNoCompression
2018-01-11 10:23:42.174056 7f133607be00 0 set rocksdb option max_write_buffer_number = 4
2018-01-11 10:23:42.174078 7f133607be00 0 set rocksdb option min_write_buffer_number_to_merge = 1
2018-01-11 10:23:42.174091 7f133607be00 0 set rocksdb option recycle_log_file_num = 4
2018-01-11 10:23:42.174094 7f133607be00 0 set rocksdb option writable_file_max_buffer_size = 0
2018-01-11 10:23:42.174097 7f133607be00 0 set rocksdb option write_buffer_size = 268435456
2018-01-11 10:23:42.174249 7f133607be00 4 rocksdb: RocksDB version: 5.4.0

2018-01-11 10:23:42.174258 7f133607be00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
2018-01-11 10:23:42.174260 7f133607be00 4 rocksdb: Compile date Sep 26 2017
2018-01-11 10:23:42.174273 7f133607be00 4 rocksdb: DB SUMMARY

2018-01-11 10:23:42.174292 7f133607be00 4 rocksdb: CURRENT file: CURRENT

2018-01-11 10:23:42.174294 7f133607be00 4 rocksdb: IDENTITY file: IDENTITY

2018-01-11 10:23:42.174296 7f133607be00 4 rocksdb: MANIFEST file: MANIFEST-016216 size: 69541 Bytes

2018-01-11 10:23:42.174299 7f133607be00 4 rocksdb: SST files in db dir, Total Num: 8, files: 015138.sst 015139.sst 015140.sst 016206.sst 016207.sst 016209.sst 016212.sst 016215.sst

2018-01-11 10:23:42.174524 7f133607be00 4 rocksdb: SST files in db.slow dir, Total Num: 378, files: 007717.sst 007748.sst 008161.sst 008669.sst 008670.sst 009336.sst 009514.sst 009515.sst 009516.sst

2018-01-11 10:23:42.174530 7f133607be00 4 rocksdb: Write Ahead Log file in db: 016217.log size: 618187 ;

2018-01-11 10:23:42.174532 7f133607be00 4 rocksdb: Options.error_if_exists: 0
2018-01-11 10:23:42.174534 7f133607be00 4 rocksdb: Options.create_if_missing: 0
2018-01-11 10:23:42.174534 7f133607be00 4 rocksdb: Options.paranoid_checks: 1
2018-01-11 10:23:42.174535 7f133607be00 4 rocksdb: Options.env: 0x55723f6abf80
2018-01-11 10:23:42.174537 7f133607be00 4 rocksdb: Options.info_log: 0x55723f8c5480
2018-01-11 10:23:42.174538 7f133607be00 4 rocksdb: Options.max_open_files: -1
2018-01-11 10:23:42.174540 7f133607be00 4 rocksdb: Options.max_file_opening_threads: 16
2018-01-11 10:23:42.174541 7f133607be00 4 rocksdb: Options.use_fsync: 0
2018-01-11 10:23:42.174542 7f133607be00 4 rocksdb: Options.max_log_file_size: 0
2018-01-11 10:23:42.174542 7f133607be00 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
2018-01-11 10:23:42.174544 7f133607be00 4 rocksdb: Options.log_file_time_to_roll: 0
2018-01-11 10:23:42.174544 7f133607be00 4 rocksdb: Options.keep_log_file_num: 1000
2018-01-11 10:23:42.174545 7f133607be00 4 rocksdb: Options.recycle_log_file_num: 4
2018-01-11 10:23:42.174546 7f133607be00 4 rocksdb: Options.allow_fallocate: 1
2018-01-11 10:23:42.174547 7f133607be00 4 rocksdb: Options.allow_mmap_reads: 0
2018-01-11 10:23:42.174548 7f133607be00 4 rocksdb: Options.allow_mmap_writes: 0
2018-01-11 10:23:42.174549 7f133607be00 4 rocksdb: Options.use_direct_reads: 0
2018-01-11 10:23:42.174550 7f133607be00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
2018-01-11 10:23:42.174550 7f133607be00 4 rocksdb: Options.create_missing_column_families: 0
2018-01-11 10:23:42.174551 7f133607be00 4 rocksdb: Options.db_log_dir:
2018-01-11 10:23:42.174552 7f133607be00 4 rocksdb: Options.wal_dir: db
2018-01-11 10:23:42.174553 7f133607be00 4 rocksdb: Options.table_cache_numshardbits: 6
2018-01-11 10:23:42.174553 7f133607be00 4 rocksdb: Options.max_subcompactions: 1
2018-01-11 10:23:42.174554 7f133607be00 4 rocksdb: Options.max_background_flushes: 1
2018-01-11 10:23:42.174555 7f133607be00 4 rocksdb: Options.WAL_ttl_seconds: 0
2018-01-11 10:23:42.174556 7f133607be00 4 rocksdb: Options.WAL_size_limit_MB: 0
2018-01-11 10:23:42.174557 7f133607be00 4 rocksdb: Options.manifest_preallocation_size: 4194304
2018-01-11 10:23:42.174558 7f133607be00 4 rocksdb: Options.is_fd_close_on_exec: 1
2018-01-11 10:23:42.174559 7f133607be00 4 rocksdb: Options.advise_random_on_open: 1
2018-01-11 10:23:42.174559 7f133607be00 4 rocksdb: Options.db_write_buffer_size: 0
2018-01-11 10:23:42.174560 7f133607be00 4 rocksdb: Options.access_hint_on_compaction_start: 1
2018-01-11 10:23:42.174561 7f133607be00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
2018-01-11 10:23:42.174562 7f133607be00 4 rocksdb: Options.compaction_readahead_size: 2097152
2018-01-11 10:23:42.174563 7f133607be00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
2018-01-11 10:23:42.174563 7f133607be00 4 rocksdb: Options.writable_file_max_buffer_size: 0
2018-01-11 10:23:42.174564 7f133607be00 4 rocksdb: Options.use_adaptive_mutex: 0
2018-01-11 10:23:42.174565 7f133607be00 4 rocksdb: Options.rate_limiter: (nil)
2018-01-11 10:23:42.174566 7f133607be00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
2018-01-11 10:23:42.174567 7f133607be00 4 rocksdb: Options.bytes_per_sync: 0
2018-01-11 10:23:42.174568 7f133607be00 4 rocksdb: Options.wal_bytes_per_sync: 0
2018-01-11 10:23:42.174569 7f133607be00 4 rocksdb: Options.wal_recovery_mode: 0
2018-01-11 10:23:42.174569 7f133607be00 4 rocksdb: Options.enable_thread_tracking: 0
2018-01-11 10:23:42.174570 7f133607be00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
2018-01-11 10:23:42.174571 7f133607be00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
2018-01-11 10:23:42.174572 7f133607be00 4 rocksdb: Options.write_thread_max_yield_usec: 100
2018-01-11 10:23:42.174573 7f133607be00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
2018-01-11 10:23:42.174573 7f133607be00 4 rocksdb: Options.row_cache: None
2018-01-11 10:23:42.174575 7f133607be00 4 rocksdb: Options.wal_filter: None
2018-01-11 10:23:42.174576 7f133607be00 4 rocksdb: Options.avoid_flush_during_recovery: 0
2018-01-11 10:23:42.174577 7f133607be00 4 rocksdb: Options.base_background_compactions: 1
2018-01-11 10:23:42.174577 7f133607be00 4 rocksdb: Options.max_background_compactions: 1
2018-01-11 10:23:42.174578 7f133607be00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
2018-01-11 10:23:42.174579 7f133607be00 4 rocksdb: Options.delayed_write_rate : 16777216
2018-01-11 10:23:42.174580 7f133607be00 4 rocksdb: Options.max_total_wal_size: 0
2018-01-11 10:23:42.174580 7f133607be00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
2018-01-11 10:23:42.174581 7f133607be00 4 rocksdb: Options.stats_dump_period_sec: 600
2018-01-11 10:23:42.174582 7f133607be00 4 rocksdb: Compression algorithms supported:
2018-01-11 10:23:42.174591 7f133607be00 4 rocksdb: Snappy supported: 0
2018-01-11 10:23:42.174592 7f133607be00 4 rocksdb: Zlib supported: 0
2018-01-11 10:23:42.174593 7f133607be00 4 rocksdb: Bzip supported: 0
2018-01-11 10:23:42.174594 7f133607be00 4 rocksdb: LZ4 supported: 0
2018-01-11 10:23:42.174594 7f133607be00 4 rocksdb: ZSTD supported: 0
2018-01-11 10:23:42.174595 7f133607be00 4 rocksdb: Fast CRC32 supported: 0
2018-01-11 10:23:42.174801 7f133607be00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2609] Recovering from manifest file: MANIFEST-016216

2018-01-11 10:23:42.174857 7f133607be00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/column_family.cc:407] --------------- Options for column family [default]:

2018-01-11 10:23:42.174863 7f133607be00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
2018-01-11 10:23:42.174868 7f133607be00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
2018-01-11 10:23:42.174869 7f133607be00 4 rocksdb: Options.compaction_filter: None
2018-01-11 10:23:42.174870 7f133607be00 4 rocksdb: Options.compaction_filter_factory: None
2018-01-11 10:23:42.174871 7f133607be00 4 rocksdb: Options.memtable_factory: SkipListFactory
2018-01-11 10:23:42.174872 7f133607be00 4 rocksdb: Options.table_factory: BlockBasedTable
2018-01-11 10:23:42.174890 7f133607be00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55723f2e6170)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 1
pin_l0_filter_and_index_blocks_in_cache: 1
index_type: 0
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x55723f5d8e70
block_cache_name: LRUCache
block_cache_options:
capacity : 536870912
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
format_version: 2

2018-01-11 10:23:42.174894 7f133607be00 4 rocksdb: Options.write_buffer_size: 268435456
2018-01-11 10:23:42.174895 7f133607be00 4 rocksdb: Options.max_write_buffer_number: 4
2018-01-11 10:23:42.174896 7f133607be00 4 rocksdb: Options.compression: NoCompression
2018-01-11 10:23:42.174898 7f133607be00 4 rocksdb: Options.bottommost_compression: Disabled
2018-01-11 10:23:42.174899 7f133607be00 4 rocksdb: Options.prefix_extractor: nullptr
2018-01-11 10:23:42.174900 7f133607be00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
2018-01-11 10:23:42.174902 7f133607be00 4 rocksdb: Options.num_levels: 7
2018-01-11 10:23:42.174903 7f133607be00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
2018-01-11 10:23:42.174904 7f133607be00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
2018-01-11 10:23:42.174904 7f133607be00 4 rocksdb: Options.compression_opts.window_bits: -14
2018-01-11 10:23:42.174906 7f133607be00 4 rocksdb: Options.compression_opts.level: -1
2018-01-11 10:23:42.174906 7f133607be00 4 rocksdb: Options.compression_opts.strategy: 0
2018-01-11 10:23:42.174907 7f133607be00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
2018-01-11 10:23:42.174908 7f133607be00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
2018-01-11 10:23:42.174909 7f133607be00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
2018-01-11 10:23:42.174910 7f133607be00 4 rocksdb: Options.level0_stop_writes_trigger: 36
2018-01-11 10:23:42.174911 7f133607be00 4 rocksdb: Options.target_file_size_base: 67108864
2018-01-11 10:23:42.174912 7f133607be00 4 rocksdb: Options.target_file_size_multiplier: 1
2018-01-11 10:23:42.174913 7f133607be00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
2018-01-11 10:23:42.174914 7f133607be00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
2018-01-11 10:23:42.174916 7f133607be00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
2018-01-11 10:23:42.174919 7f133607be00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
2018-01-11 10:23:42.174920 7f133607be00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
2018-01-11 10:23:42.174921 7f133607be00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
2018-01-11 10:23:42.174922 7f133607be00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
2018-01-11 10:23:42.174923 7f133607be00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
2018-01-11 10:23:42.174923 7f133607be00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
2018-01-11 10:23:42.174924 7f133607be00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
2018-01-11 10:23:42.174925 7f133607be00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
2018-01-11 10:23:42.174925 7f133607be00 4 rocksdb: Options.max_compaction_bytes: 1677721600
2018-01-11 10:23:42.174926 7f133607be00 4 rocksdb: Options.arena_block_size: 33554432
2018-01-11 10:23:42.174935 7f133607be00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
2018-01-11 10:23:42.174936 7f133607be00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
2018-01-11 10:23:42.174937 7f133607be00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
2018-01-11 10:23:42.174938 7f133607be00 4 rocksdb: Options.disable_auto_compactions: 0
2018-01-11 10:23:42.174939 7f133607be00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
2018-01-11 10:23:42.174941 7f133607be00 4 rocksdb: Options.compaction_pri: kByCompensatedSize
2018-01-11 10:23:42.174941 7f133607be00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
2018-01-11 10:23:42.174942 7f133607be00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
2018-01-11 10:23:42.174943 7f133607be00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
2018-01-11 10:23:42.174944 7f133607be00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
2018-01-11 10:23:42.174944 7f133607be00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
2018-01-11 10:23:42.174946 7f133607be00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
2018-01-11 10:23:42.174947 7f133607be00 4 rocksdb: Options.table_properties_collectors:
2018-01-11 10:23:42.174948 7f133607be00 4 rocksdb: Options.inplace_update_support: 0
2018-01-11 10:23:42.174948 7f133607be00 4 rocksdb: Options.inplace_update_num_locks: 10000
2018-01-11 10:23:42.174949 7f133607be00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
2018-01-11 10:23:42.174950 7f133607be00 4 rocksdb: Options.memtable_huge_page_size: 0
2018-01-11 10:23:42.174951 7f133607be00 4 rocksdb: Options.bloom_locality: 0
2018-01-11 10:23:42.174952 7f133607be00 4 rocksdb: Options.max_successive_merges: 0
2018-01-11 10:23:42.174952 7f133607be00 4 rocksdb: Options.optimize_filters_for_hits: 0
2018-01-11 10:23:42.174953 7f133607be00 4 rocksdb: Options.paranoid_file_checks: 0
2018-01-11 10:23:42.174954 7f133607be00 4 rocksdb: Options.force_consistency_checks: 0
2018-01-11 10:23:42.174955 7f133607be00 4 rocksdb: Options.report_bg_io_stats: 0
2018-01-11 10:23:46.504016 7f133607be00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2859] Recovered from manifest file:db/MANIFEST-016216 succeeded,manifest_file_number is 16216, next_file_number is 16218, last_sequence is 341931965, log_number is 0,prev_log_number is 0,max_column_family is 0

2018-01-11 10:23:46.504048 7f133607be00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2867] Column family [default] (ID 0), log number is 16215

2018-01-11 10:23:46.505146 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662626505131, "job": 1, "event": "recovery_started", "log_files": [16217]}
2018-01-11 10:23:46.505159 7f133607be00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:482] Recovering log #16217 mode 0
2018-01-11 10:23:46.540048 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662626540033, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 16218, "file_size": 373756, "table_properties": {"data_size": 355562, "index_size": 2855, "filter_size": 14363, "raw_key_size": 233484, "raw_average_key_size": 42, "raw_value_size": 161972, "raw_average_value_size": 29, "num_data_blocks": 88, "num_entries": 5431, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-01-11 10:23:46.540321 7f133607be00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2395] Creating manifest 16219

2018-01-11 10:23:46.541731 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662626541729, "job": 1, "event": "recovery_finished"}
2018-01-11 10:23:46.543835 7f133607be00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:1063] DB pointer 0x55723f5e7a00
2018-01-11 10:23:46.543873 7f133607be00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
2018-01-11 10:23:46.544679 7f1322812700 3 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl.cc:447] ------- DUMPING STATS -------
2018-01-11 10:23:46.544693 7f1322812700 3 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl.cc:448]
** DB Stats **
Uptime(secs): 4.4 total, 4.4 interval
Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s
Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s
Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s
Interval stall: 00:00:0.000 H:M:S, 0.0 percent

** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
----------------------------------------------------------------------------------------------------------------------------------------------------------
L0 4/0 1.43 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.1 0 1 0.008 0 0
L1 5/0 211.65 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
L2 46/0 2.45 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
L3 332/0 20.04 GB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Sum 387/0 22.69 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.1 0 1 0.008 0 0
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.1 0 1 0.008 0 0
Uptime(secs): 4.4 total, 4.4 interval
Flush(GB): cumulative 0.000, interval 0.000
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 0.00 GB write, 0.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Interval compaction: 0.00 GB write, 0.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** File Read Latency Histogram By Level [default] **

** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
----------------------------------------------------------------------------------------------------------------------------------------------------------
L0 4/0 1.43 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.1 0 1 0.008 0 0
L1 5/0 211.65 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
L2 46/0 2.45 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
L3 332/0 20.04 GB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Sum 387/0 22.69 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.1 0 1 0.008 0 0
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Uptime(secs): 4.4 total, 4.4 interval
Flush(GB): cumulative 0.000, interval 0.000
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 0.00 GB write, 0.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** File Read Latency Histogram By Level [default] **

2018-01-11 10:23:46.544816 7f1322812700 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/compaction_job.cc:1403] [default] [JOB 3] Compacting 4@0 + 2@1 files to L1, score 1.00
2018-01-11 10:23:46.544833 7f1322812700 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/compaction_job.cc:1407] [default] Compaction start summary: Base version 2 Base level 0, inputs: [16218(364KB) 16215(364KB) 16212(364KB) 16209(364KB)], [16206(8200KB) 16207(7669KB)]

2018-01-11 10:23:46.544864 7f1322812700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662626544840, "job": 3, "event": "compaction_started", "files_L0": [16218, 16215, 16212, 16209], "files_L1": [16206, 16207], "score": 1, "input_data_size": 17745854}
2018-01-11 10:23:46.591713 7f133607be00 1 freelist init
2018-01-11 10:23:46.648381 7f133607be00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc opening allocation metadata
2018-01-11 10:23:46.674574 7f1322812700 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 3] Generated table #16223: 9436 keys, 8397458 bytes
2018-01-11 10:23:46.674712 7f1322812700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662626674690, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 16223, "file_size": 8397458, "table_properties": {"data_size": 8298571, "index_size": 55745, "filter_size": 42158, "raw_key_size": 664873, "raw_average_key_size": 70, "raw_value_size": 7672844, "raw_average_value_size": 813, "num_data_blocks": 1589, "num_entries": 9436, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "18", "kMergeOperands": "0"}}
2018-01-11 10:23:46.905148 7f1322812700 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 3] Generated table #16224: 165144 keys, 7853372 bytes
2018-01-11 10:23:46.905184 7f1322812700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662626905170, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 16224, "file_size": 7853372, "table_properties": {"data_size": 7366335, "index_size": 56825, "filter_size": 429222, "raw_key_size": 5476264, "raw_average_key_size": 33, "raw_value_size": 2615059, "raw_average_value_size": 15, "num_data_blocks": 1794, "num_entries": 165144, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "29605", "kMergeOperands": "134752"}}
2018-01-11 10:23:46.905887 7f1322812700 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/compaction_job.cc:1173] [default] [JOB 3] Compacted 4@0 + 2@1 files to L1 => 16250830 bytes
2018-01-11 10:23:46.906874 7f1322812700 4 rocksdb: (Original Log Time 2018/01/11-10:23:46.906800) [/build/ceph-12.2.1/src/rocksdb/db/compaction_job.cc:621] [default] compacted to: base level 1 max bytes base 268435456 files[0 5 46 332 0 0 0] max score 0.99, MB/sec: 49.2 rd, 45.0 wr, level 1, files in(4, 2) out(2) MB in(1.4, 15.5) out(15.5), read-write-amplify(22.7) write-amplify(10.9) OK, records in: 196304, records dropped: 21724

2018-01-11 10:23:46.906884 7f1322812700 4 rocksdb: (Original Log Time 2018/01/11-10:23:46.906827) EVENT_LOG_v1 {"time_micros": 1515662626906814, "job": 3, "event": "compaction_finished", "compaction_time_micros": 361004, "output_level": 1, "num_output_files": 2, "total_output_size": 16250830, "num_input_records": 196304, "num_output_records": 174580, "num_subcompactions": 1, "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 5, 46, 332, 0, 0, 0]}
2018-01-11 10:23:56.759550 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662636759545, "job": 0, "event": "table_file_deletion", "file_number": 16218}
2018-01-11 10:23:56.759564 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662636759562, "job": 0, "event": "table_file_deletion", "file_number": 16215}
2018-01-11 10:23:56.759636 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662636759628, "job": 0, "event": "table_file_deletion", "file_number": 16212}
2018-01-11 10:23:56.759689 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662636759687, "job": 0, "event": "table_file_deletion", "file_number": 16209}
2018-01-11 10:23:56.759701 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662636759700, "job": 0, "event": "table_file_deletion", "file_number": 16207}
2018-01-11 10:23:56.759714 7f133607be00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662636759712, "job": 0, "event": "table_file_deletion", "file_number": 16206}
2018-01-11 10:23:56.759776 7f133607be00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc loaded 318 G in 658679 extents
2018-01-11 10:23:56.883805 7f133607be00 0 <cls> /build/ceph-12.2.1/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
2018-01-11 10:23:56.886633 7f133607be00 0 <cls> /build/ceph-12.2.1/src/cls/hello/cls_hello.cc:296: loading cls_hello
2018-01-11 10:23:56.886671 7f133607be00 0 _get_class not permitted to load kvs
2018-01-11 10:23:56.886820 7f133607be00 0 _get_class not permitted to load lua
2018-01-11 10:23:56.887888 7f133607be00 0 _get_class not permitted to load sdk
2018-01-11 10:23:56.917113 7f133607be00 0 osd.2 15344 crush map has features 2200130813952, adjusting msgr requires for clients
2018-01-11 10:23:56.917132 7f133607be00 0 osd.2 15344 crush map has features 2200130813952 was 8705, adjusting msgr requires for mons
2018-01-11 10:23:56.917137 7f133607be00 0 osd.2 15344 crush map has features 720578140510109696, adjusting msgr requires for osds
2018-01-11 10:24:13.476205 7f133607be00 0 osd.2 15344 load_pgs
2018-01-11 10:25:01.929742 7fb2c1f0ee00 0 ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable), process (unknown), pid 1944339
2018-01-11 10:25:01.938339 7fb2c1f0ee00 0 pidfile_write: ignore empty --pid-file
2018-01-11 10:25:01.954505 7fb2c1f0ee00 0 load: jerasure load: lrc load: isa
2018-01-11 10:25:01.954795 7fb2c1f0ee00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:25:01.954828 7fb2c1f0ee00 1 bdev(0x556e11df4b40 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:25:01.955845 7fb2c1f0ee00 1 bdev(0x556e11df4b40 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:25:01.956376 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
2018-01-11 10:25:01.956418 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
2018-01-11 10:25:01.956426 7fb2c1f0ee00 1 bdev(0x556e11df4b40 /var/lib/ceph/osd/ceph-2/block) close
2018-01-11 10:25:02.248000 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _mount path /var/lib/ceph/osd/ceph-2
2018-01-11 10:25:02.248255 7fb2c1f0ee00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:25:02.248281 7fb2c1f0ee00 1 bdev(0x556e11df4d80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:25:02.248947 7fb2c1f0ee00 1 bdev(0x556e11df4d80 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:25:02.249758 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
2018-01-11 10:25:02.249801 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
2018-01-11 10:25:02.250034 7fb2c1f0ee00 1 bdev create path /var/lib/ceph/osd/ceph-2/block.db type kernel
2018-01-11 10:25:02.250068 7fb2c1f0ee00 1 bdev(0x556e11df4fc0 /var/lib/ceph/osd/ceph-2/block.db) open path /var/lib/ceph/osd/ceph-2/block.db
2018-01-11 10:25:02.250741 7fb2c1f0ee00 1 bdev(0x556e11df4fc0 /var/lib/ceph/osd/ceph-2/block.db) open size 29999759360 (0x6fc200000, 28610 MB) block_size 4096 (4096 B) non-rotational
2018-01-11 10:25:02.250775 7fb2c1f0ee00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block.db size 28610 MB
2018-01-11 10:25:02.252438 7fb2c1f0ee00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2018-01-11 10:25:02.252471 7fb2c1f0ee00 1 bdev(0x556e11df5200 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2018-01-11 10:25:02.253123 7fb2c1f0ee00 1 bdev(0x556e11df5200 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
2018-01-11 10:25:02.253156 7fb2c1f0ee00 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-2/block size 931 GB
2018-01-11 10:25:02.253231 7fb2c1f0ee00 1 bluefs mount
2018-01-11 10:25:02.285589 7fb2c1f0ee00 0 set rocksdb option compaction_readahead_size = 2097152
2018-01-11 10:25:02.285660 7fb2c1f0ee00 0 set rocksdb option compression = kNoCompression
2018-01-11 10:25:02.285666 7fb2c1f0ee00 0 set rocksdb option max_write_buffer_number = 4
2018-01-11 10:25:02.285670 7fb2c1f0ee00 0 set rocksdb option min_write_buffer_number_to_merge = 1
2018-01-11 10:25:02.285673 7fb2c1f0ee00 0 set rocksdb option recycle_log_file_num = 4
2018-01-11 10:25:02.285675 7fb2c1f0ee00 0 set rocksdb option writable_file_max_buffer_size = 0
2018-01-11 10:25:02.285679 7fb2c1f0ee00 0 set rocksdb option write_buffer_size = 268435456
2018-01-11 10:25:02.285699 7fb2c1f0ee00 0 set rocksdb option compaction_readahead_size = 2097152
2018-01-11 10:25:02.285702 7fb2c1f0ee00 0 set rocksdb option compression = kNoCompression
2018-01-11 10:25:02.285706 7fb2c1f0ee00 0 set rocksdb option max_write_buffer_number = 4
2018-01-11 10:25:02.285708 7fb2c1f0ee00 0 set rocksdb option min_write_buffer_number_to_merge = 1
2018-01-11 10:25:02.285711 7fb2c1f0ee00 0 set rocksdb option recycle_log_file_num = 4
2018-01-11 10:25:02.285714 7fb2c1f0ee00 0 set rocksdb option writable_file_max_buffer_size = 0
2018-01-11 10:25:02.285717 7fb2c1f0ee00 0 set rocksdb option write_buffer_size = 268435456
2018-01-11 10:25:02.285827 7fb2c1f0ee00 4 rocksdb: RocksDB version: 5.4.0

2018-01-11 10:25:02.285889 7fb2c1f0ee00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
2018-01-11 10:25:02.285892 7fb2c1f0ee00 4 rocksdb: Compile date Sep 26 2017
2018-01-11 10:25:02.285894 7fb2c1f0ee00 4 rocksdb: DB SUMMARY

2018-01-11 10:25:02.285912 7fb2c1f0ee00 4 rocksdb: CURRENT file: CURRENT

2018-01-11 10:25:02.285913 7fb2c1f0ee00 4 rocksdb: IDENTITY file: IDENTITY

2018-01-11 10:25:02.285915 7fb2c1f0ee00 4 rocksdb: MANIFEST file: MANIFEST-016219 size: 69928 Bytes

2018-01-11 10:25:02.285918 7fb2c1f0ee00 4 rocksdb: SST files in db dir, Total Num: 5, files: 015138.sst 015139.sst 015140.sst 016223.sst 016224.sst

2018-01-11 10:25:02.286104 7fb2c1f0ee00 4 rocksdb: SST files in db.slow dir, Total Num: 378, files: 007717.sst 007748.sst 008161.sst 008669.sst 008670.sst 009336.sst 009514.sst 009515.sst 009516.sst

2018-01-11 10:25:02.286153 7fb2c1f0ee00 4 rocksdb: Write Ahead Log file in db: 016220.log size: 188643 ;

2018-01-11 10:25:02.286156 7fb2c1f0ee00 4 rocksdb: Options.error_if_exists: 0
2018-01-11 10:25:02.286158 7fb2c1f0ee00 4 rocksdb: Options.create_if_missing: 0
2018-01-11 10:25:02.286159 7fb2c1f0ee00 4 rocksdb: Options.paranoid_checks: 1
2018-01-11 10:25:02.286160 7fb2c1f0ee00 4 rocksdb: Options.env: 0x556e12141e00
2018-01-11 10:25:02.286161 7fb2c1f0ee00 4 rocksdb: Options.info_log: 0x556e1235f260
2018-01-11 10:25:02.286162 7fb2c1f0ee00 4 rocksdb: Options.max_open_files: -1
2018-01-11 10:25:02.286163 7fb2c1f0ee00 4 rocksdb: Options.max_file_opening_threads: 16
2018-01-11 10:25:02.286163 7fb2c1f0ee00 4 rocksdb: Options.use_fsync: 0
2018-01-11 10:25:02.286164 7fb2c1f0ee00 4 rocksdb: Options.max_log_file_size: 0
2018-01-11 10:25:02.286165 7fb2c1f0ee00 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
2018-01-11 10:25:02.286166 7fb2c1f0ee00 4 rocksdb: Options.log_file_time_to_roll: 0
2018-01-11 10:25:02.286167 7fb2c1f0ee00 4 rocksdb: Options.keep_log_file_num: 1000
2018-01-11 10:25:02.286168 7fb2c1f0ee00 4 rocksdb: Options.recycle_log_file_num: 4
2018-01-11 10:25:02.286169 7fb2c1f0ee00 4 rocksdb: Options.allow_fallocate: 1
2018-01-11 10:25:02.286169 7fb2c1f0ee00 4 rocksdb: Options.allow_mmap_reads: 0
2018-01-11 10:25:02.286170 7fb2c1f0ee00 4 rocksdb: Options.allow_mmap_writes: 0
2018-01-11 10:25:02.286171 7fb2c1f0ee00 4 rocksdb: Options.use_direct_reads: 0
2018-01-11 10:25:02.286171 7fb2c1f0ee00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
2018-01-11 10:25:02.286172 7fb2c1f0ee00 4 rocksdb: Options.create_missing_column_families: 0
2018-01-11 10:25:02.286172 7fb2c1f0ee00 4 rocksdb: Options.db_log_dir:
2018-01-11 10:25:02.286234 7fb2c1f0ee00 4 rocksdb: Options.wal_dir: db
2018-01-11 10:25:02.286236 7fb2c1f0ee00 4 rocksdb: Options.table_cache_numshardbits: 6
2018-01-11 10:25:02.286237 7fb2c1f0ee00 4 rocksdb: Options.max_subcompactions: 1
2018-01-11 10:25:02.286238 7fb2c1f0ee00 4 rocksdb: Options.max_background_flushes: 1
2018-01-11 10:25:02.286239 7fb2c1f0ee00 4 rocksdb: Options.WAL_ttl_seconds: 0
2018-01-11 10:25:02.286240 7fb2c1f0ee00 4 rocksdb: Options.WAL_size_limit_MB: 0
2018-01-11 10:25:02.286240 7fb2c1f0ee00 4 rocksdb: Options.manifest_preallocation_size: 4194304
2018-01-11 10:25:02.286241 7fb2c1f0ee00 4 rocksdb: Options.is_fd_close_on_exec: 1
2018-01-11 10:25:02.286242 7fb2c1f0ee00 4 rocksdb: Options.advise_random_on_open: 1
2018-01-11 10:25:02.286242 7fb2c1f0ee00 4 rocksdb: Options.db_write_buffer_size: 0
2018-01-11 10:25:02.286243 7fb2c1f0ee00 4 rocksdb: Options.access_hint_on_compaction_start: 1
2018-01-11 10:25:02.286244 7fb2c1f0ee00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
2018-01-11 10:25:02.286244 7fb2c1f0ee00 4 rocksdb: Options.compaction_readahead_size: 2097152
2018-01-11 10:25:02.286245 7fb2c1f0ee00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
2018-01-11 10:25:02.286246 7fb2c1f0ee00 4 rocksdb: Options.writable_file_max_buffer_size: 0
2018-01-11 10:25:02.286246 7fb2c1f0ee00 4 rocksdb: Options.use_adaptive_mutex: 0
2018-01-11 10:25:02.286247 7fb2c1f0ee00 4 rocksdb: Options.rate_limiter: (nil)
2018-01-11 10:25:02.286255 7fb2c1f0ee00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
2018-01-11 10:25:02.286256 7fb2c1f0ee00 4 rocksdb: Options.bytes_per_sync: 0
2018-01-11 10:25:02.286257 7fb2c1f0ee00 4 rocksdb: Options.wal_bytes_per_sync: 0
2018-01-11 10:25:02.286257 7fb2c1f0ee00 4 rocksdb: Options.wal_recovery_mode: 0
2018-01-11 10:25:02.286258 7fb2c1f0ee00 4 rocksdb: Options.enable_thread_tracking: 0
2018-01-11 10:25:02.286259 7fb2c1f0ee00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
2018-01-11 10:25:02.286260 7fb2c1f0ee00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
2018-01-11 10:25:02.286260 7fb2c1f0ee00 4 rocksdb: Options.write_thread_max_yield_usec: 100
2018-01-11 10:25:02.286261 7fb2c1f0ee00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
2018-01-11 10:25:02.286262 7fb2c1f0ee00 4 rocksdb: Options.row_cache: None
2018-01-11 10:25:02.286262 7fb2c1f0ee00 4 rocksdb: Options.wal_filter: None
2018-01-11 10:25:02.286263 7fb2c1f0ee00 4 rocksdb: Options.avoid_flush_during_recovery: 0
2018-01-11 10:25:02.286264 7fb2c1f0ee00 4 rocksdb: Options.base_background_compactions: 1
2018-01-11 10:25:02.286265 7fb2c1f0ee00 4 rocksdb: Options.max_background_compactions: 1
2018-01-11 10:25:02.286266 7fb2c1f0ee00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
2018-01-11 10:25:02.286266 7fb2c1f0ee00 4 rocksdb: Options.delayed_write_rate : 16777216
2018-01-11 10:25:02.286267 7fb2c1f0ee00 4 rocksdb: Options.max_total_wal_size: 0
2018-01-11 10:25:02.286268 7fb2c1f0ee00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
2018-01-11 10:25:02.286269 7fb2c1f0ee00 4 rocksdb: Options.stats_dump_period_sec: 600
2018-01-11 10:25:02.286269 7fb2c1f0ee00 4 rocksdb: Compression algorithms supported:
2018-01-11 10:25:02.286270 7fb2c1f0ee00 4 rocksdb: Snappy supported: 0
2018-01-11 10:25:02.286271 7fb2c1f0ee00 4 rocksdb: Zlib supported: 0
2018-01-11 10:25:02.286272 7fb2c1f0ee00 4 rocksdb: Bzip supported: 0
2018-01-11 10:25:02.286272 7fb2c1f0ee00 4 rocksdb: LZ4 supported: 0
2018-01-11 10:25:02.286273 7fb2c1f0ee00 4 rocksdb: ZSTD supported: 0
2018-01-11 10:25:02.286274 7fb2c1f0ee00 4 rocksdb: Fast CRC32 supported: 0
2018-01-11 10:25:02.286505 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2609] Recovering from manifest file: MANIFEST-016219

2018-01-11 10:25:02.286583 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/column_family.cc:407] --------------- Options for column family [default]:

2018-01-11 10:25:02.286617 7fb2c1f0ee00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
2018-01-11 10:25:02.286670 7fb2c1f0ee00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
2018-01-11 10:25:02.286702 7fb2c1f0ee00 4 rocksdb: Options.compaction_filter: None
2018-01-11 10:25:02.286734 7fb2c1f0ee00 4 rocksdb: Options.compaction_filter_factory: None
2018-01-11 10:25:02.286766 7fb2c1f0ee00 4 rocksdb: Options.memtable_factory: SkipListFactory
2018-01-11 10:25:02.286797 7fb2c1f0ee00 4 rocksdb: Options.table_factory: BlockBasedTable
2018-01-11 10:25:02.286850 7fb2c1f0ee00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e11d80170)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 1
pin_l0_filter_and_index_blocks_in_cache: 1
index_type: 0
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x556e12076e70
block_cache_name: LRUCache
block_cache_options:
capacity : 536870912
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
format_version: 2

2018-01-11 10:25:02.286886 7fb2c1f0ee00 4 rocksdb: Options.write_buffer_size: 268435456
2018-01-11 10:25:02.286889 7fb2c1f0ee00 4 rocksdb: Options.max_write_buffer_number: 4
2018-01-11 10:25:02.286891 7fb2c1f0ee00 4 rocksdb: Options.compression: NoCompression
2018-01-11 10:25:02.286892 7fb2c1f0ee00 4 rocksdb: Options.bottommost_compression: Disabled
2018-01-11 10:25:02.286893 7fb2c1f0ee00 4 rocksdb: Options.prefix_extractor: nullptr
2018-01-11 10:25:02.286894 7fb2c1f0ee00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
2018-01-11 10:25:02.286895 7fb2c1f0ee00 4 rocksdb: Options.num_levels: 7
2018-01-11 10:25:02.286896 7fb2c1f0ee00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
2018-01-11 10:25:02.286912 7fb2c1f0ee00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
2018-01-11 10:25:02.286943 7fb2c1f0ee00 4 rocksdb: Options.compression_opts.window_bits: -14
2018-01-11 10:25:02.286975 7fb2c1f0ee00 4 rocksdb: Options.compression_opts.level: -1
2018-01-11 10:25:02.287006 7fb2c1f0ee00 4 rocksdb: Options.compression_opts.strategy: 0
2018-01-11 10:25:02.287038 7fb2c1f0ee00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
2018-01-11 10:25:02.287069 7fb2c1f0ee00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
2018-01-11 10:25:02.287101 7fb2c1f0ee00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
2018-01-11 10:25:02.287132 7fb2c1f0ee00 4 rocksdb: Options.level0_stop_writes_trigger: 36
2018-01-11 10:25:02.287163 7fb2c1f0ee00 4 rocksdb: Options.target_file_size_base: 67108864
2018-01-11 10:25:02.287195 7fb2c1f0ee00 4 rocksdb: Options.target_file_size_multiplier: 1
2018-01-11 10:25:02.287226 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
2018-01-11 10:25:02.287258 7fb2c1f0ee00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
2018-01-11 10:25:02.287289 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
2018-01-11 10:25:02.287323 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
2018-01-11 10:25:02.287354 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
2018-01-11 10:25:02.287385 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
2018-01-11 10:25:02.287416 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
2018-01-11 10:25:02.287448 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
2018-01-11 10:25:02.287479 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
2018-01-11 10:25:02.287510 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
2018-01-11 10:25:02.287541 7fb2c1f0ee00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
2018-01-11 10:25:02.287573 7fb2c1f0ee00 4 rocksdb: Options.max_compaction_bytes: 1677721600
2018-01-11 10:25:02.287604 7fb2c1f0ee00 4 rocksdb: Options.arena_block_size: 33554432
2018-01-11 10:25:02.287635 7fb2c1f0ee00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
2018-01-11 10:25:02.287666 7fb2c1f0ee00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
2018-01-11 10:25:02.287697 7fb2c1f0ee00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
2018-01-11 10:25:02.287729 7fb2c1f0ee00 4 rocksdb: Options.disable_auto_compactions: 0
2018-01-11 10:25:02.287760 7fb2c1f0ee00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
2018-01-11 10:25:02.287792 7fb2c1f0ee00 4 rocksdb: Options.compaction_pri: kByCompensatedSize
2018-01-11 10:25:02.287823 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
2018-01-11 10:25:02.287854 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
2018-01-11 10:25:02.287885 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
2018-01-11 10:25:02.287917 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
2018-01-11 10:25:02.287948 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
2018-01-11 10:25:02.287979 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
2018-01-11 10:25:02.288010 7fb2c1f0ee00 4 rocksdb: Options.table_properties_collectors:
2018-01-11 10:25:02.288041 7fb2c1f0ee00 4 rocksdb: Options.inplace_update_support: 0
2018-01-11 10:25:02.288072 7fb2c1f0ee00 4 rocksdb: Options.inplace_update_num_locks: 10000
2018-01-11 10:25:02.288104 7fb2c1f0ee00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
2018-01-11 10:25:02.288136 7fb2c1f0ee00 4 rocksdb: Options.memtable_huge_page_size: 0
2018-01-11 10:25:02.288167 7fb2c1f0ee00 4 rocksdb: Options.bloom_locality: 0
2018-01-11 10:25:02.288199 7fb2c1f0ee00 4 rocksdb: Options.max_successive_merges: 0
2018-01-11 10:25:02.288230 7fb2c1f0ee00 4 rocksdb: Options.optimize_filters_for_hits: 0
2018-01-11 10:25:02.288261 7fb2c1f0ee00 4 rocksdb: Options.paranoid_file_checks: 0
2018-01-11 10:25:02.288292 7fb2c1f0ee00 4 rocksdb: Options.force_consistency_checks: 0
2018-01-11 10:25:02.288323 7fb2c1f0ee00 4 rocksdb: Options.report_bg_io_stats: 0
2018-01-11 10:25:06.698124 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2859] Recovered from manifest file:db/MANIFEST-016219 succeeded,manifest_file_number is 16219, next_file_number is 16226, last_sequence is 341937396, log_number is 0,prev_log_number is 0,max_column_family is 0

2018-01-11 10:25:06.698171 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2867] Column family [default] (ID 0), log number is 16218

2018-01-11 10:25:06.698877 7fb2c1f0ee00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662706698868, "job": 1, "event": "recovery_started", "log_files": [16220]}
2018-01-11 10:25:06.698898 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:482] Recovering log #16220 mode 0
2018-01-11 10:25:06.713651 7fb2c1f0ee00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662706713639, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 16226, "file_size": 115027, "table_properties": {"data_size": 108793, "index_size": 865, "filter_size": 4393, "raw_key_size": 71288, "raw_average_key_size": 42, "raw_value_size": 49383, "raw_average_value_size": 29, "num_data_blocks": 27, "num_entries": 1659, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-01-11 10:25:06.714015 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2395] Creating manifest 16227

2018-01-11 10:25:06.716388 7fb2c1f0ee00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662706716385, "job": 1, "event": "recovery_finished"}
2018-01-11 10:25:06.718191 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:1063] DB pointer 0x556e12085a00
2018-01-11 10:25:06.718285 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
2018-01-11 10:25:06.776871 7fb2c1f0ee00 1 freelist init
2018-01-11 10:25:06.841621 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc opening allocation metadata
2018-01-11 10:25:16.883919 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc loaded 318 G in 658679 extents
2018-01-11 10:25:17.010960 7fb2c1f0ee00 0 <cls> /build/ceph-12.2.1/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
2018-01-11 10:25:17.015818 7fb2c1f0ee00 0 <cls> /build/ceph-12.2.1/src/cls/hello/cls_hello.cc:296: loading cls_hello
2018-01-11 10:25:17.015887 7fb2c1f0ee00 0 _get_class not permitted to load kvs
2018-01-11 10:25:17.016319 7fb2c1f0ee00 0 _get_class not permitted to load lua
2018-01-11 10:25:17.018489 7fb2c1f0ee00 0 _get_class not permitted to load sdk
2018-01-11 10:25:17.053649 7fb2c1f0ee00 0 osd.2 15344 crush map has features 2200130813952, adjusting msgr requires for clients
2018-01-11 10:25:17.054038 7fb2c1f0ee00 0 osd.2 15344 crush map has features 2200130813952 was 8705, adjusting msgr requires for mons
2018-01-11 10:25:17.054337 7fb2c1f0ee00 0 osd.2 15344 crush map has features 720578140510109696, adjusting msgr requires for osds
2018-01-11 10:25:34.303410 7fb2c1f0ee00 0 osd.2 15344 load_pgs
2018-01-11 10:27:20.420199 7fb2c1f0ee00 0 osd.2 15344 load_pgs opened 5429 pgs
2018-01-11 10:27:26.024880 7fb2c1f0ee00 -1 /build/ceph-12.2.1/src/osd/osd_types.cc: In function 'virtual void pi_compact_rep::add_interval(bool, const PastIntervals::pg_interval_t&)' thread 7fb2c1f0ee00 time 2018-01-11 10:27:26.021986
/build/ceph-12.2.1/src/osd/osd_types.cc: 3205: FAILED assert(interval.last > last)

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x556e06bfb3f2]
2: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x556e06884b75]
3: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x556e06862376]
4: (OSD::build_past_intervals_parallel()+0xe71) [0x556e06617f01]
5: (OSD::load_pgs()+0x14fb) [0x556e0661a92b]
6: (OSD::init()+0x2217) [0x556e06634b37]
7: (main()+0x2eb8) [0x556e065451f8]
8: (__libc_start_main()+0xf0) [0x7fb2bf37f830]
9: (_start()+0x29) [0x556e065d0a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-249> 2018-01-11 10:25:01.922446 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command perfcounters_dump hook 0x556e11d841b0
-248> 2018-01-11 10:25:01.922496 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command 1 hook 0x556e11d841b0
-247> 2018-01-11 10:25:01.922549 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command perf dump hook 0x556e11d841b0
-246> 2018-01-11 10:25:01.922554 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command perfcounters_schema hook 0x556e11d841b0
-245> 2018-01-11 10:25:01.922558 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command perf histogram dump hook 0x556e11d841b0
-244> 2018-01-11 10:25:01.922561 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command 2 hook 0x556e11d841b0
-243> 2018-01-11 10:25:01.922612 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command perf schema hook 0x556e11d841b0
-242> 2018-01-11 10:25:01.922661 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command perf histogram schema hook 0x556e11d841b0
-241> 2018-01-11 10:25:01.922673 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command perf reset hook 0x556e11d841b0
-240> 2018-01-11 10:25:01.922677 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command config show hook 0x556e11d841b0
-239> 2018-01-11 10:25:01.922681 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command config help hook 0x556e11d841b0
-238> 2018-01-11 10:25:01.922685 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command config set hook 0x556e11d841b0
-237> 2018-01-11 10:25:01.922689 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command config get hook 0x556e11d841b0
-236> 2018-01-11 10:25:01.922734 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command config diff hook 0x556e11d841b0
-235> 2018-01-11 10:25:01.922789 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command config diff get hook 0x556e11d841b0
-234> 2018-01-11 10:25:01.922842 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command log flush hook 0x556e11d841b0
-233> 2018-01-11 10:25:01.922905 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command log dump hook 0x556e11d841b0
-232> 2018-01-11 10:25:01.922955 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command log reopen hook 0x556e11d841b0
-231> 2018-01-11 10:25:01.923032 7fb2c1f0ee00 5 asok(0x556e11dd52c0) register_command dump_mempools hook 0x556e12070b88
-230> 2018-01-11 10:25:01.929742 7fb2c1f0ee00 0 ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable), process (unknown), pid 1944339
-229> 2018-01-11 10:25:01.938339 7fb2c1f0ee00 0 pidfile_write: ignore empty --pid-file
-228> 2018-01-11 10:25:01.954505 7fb2c1f0ee00 0 load: jerasure load: lrc load: isa
-227> 2018-01-11 10:25:01.954795 7fb2c1f0ee00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-226> 2018-01-11 10:25:01.954828 7fb2c1f0ee00 1 bdev(0x556e11df4b40 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-225> 2018-01-11 10:25:01.955845 7fb2c1f0ee00 1 bdev(0x556e11df4b40 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
-224> 2018-01-11 10:25:01.956376 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
-223> 2018-01-11 10:25:01.956418 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
-222> 2018-01-11 10:25:01.956426 7fb2c1f0ee00 1 bdev(0x556e11df4b40 /var/lib/ceph/osd/ceph-2/block) close
-221> 2018-01-11 10:25:02.248000 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _mount path /var/lib/ceph/osd/ceph-2
-220> 2018-01-11 10:25:02.248255 7fb2c1f0ee00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-219> 2018-01-11 10:25:02.248281 7fb2c1f0ee00 1 bdev(0x556e11df4d80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-218> 2018-01-11 10:25:02.248947 7fb2c1f0ee00 1 bdev(0x556e11df4d80 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
-217> 2018-01-11 10:25:02.249758 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes max 0.5 < ratio 0.99
-216> 2018-01-11 10:25:02.249801 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
-215> 2018-01-11 10:25:02.250034 7fb2c1f0ee00 1 bdev create path /var/lib/ceph/osd/ceph-2/block.db type kernel
-214> 2018-01-11 10:25:02.250068 7fb2c1f0ee00 1 bdev(0x556e11df4fc0 /var/lib/ceph/osd/ceph-2/block.db) open path /var/lib/ceph/osd/ceph-2/block.db
-213> 2018-01-11 10:25:02.250741 7fb2c1f0ee00 1 bdev(0x556e11df4fc0 /var/lib/ceph/osd/ceph-2/block.db) open size 29999759360 (0x6fc200000, 28610 MB) block_size 4096 (4096 B) non-rotational
-212> 2018-01-11 10:25:02.250775 7fb2c1f0ee00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block.db size 28610 MB
-211> 2018-01-11 10:25:02.252438 7fb2c1f0ee00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-210> 2018-01-11 10:25:02.252471 7fb2c1f0ee00 1 bdev(0x556e11df5200 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-209> 2018-01-11 10:25:02.253123 7fb2c1f0ee00 1 bdev(0x556e11df5200 /var/lib/ceph/osd/ceph-2/block) open size 1000098959360 (0xe8da8b1000, 931 GB) block_size 4096 (4096 B) rotational
-208> 2018-01-11 10:25:02.253156 7fb2c1f0ee00 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-2/block size 931 GB
-207> 2018-01-11 10:25:02.253231 7fb2c1f0ee00 1 bluefs mount
-206> 2018-01-11 10:25:02.285589 7fb2c1f0ee00 0 set rocksdb option compaction_readahead_size = 2097152
-205> 2018-01-11 10:25:02.285660 7fb2c1f0ee00 0 set rocksdb option compression = kNoCompression
-204> 2018-01-11 10:25:02.285666 7fb2c1f0ee00 0 set rocksdb option max_write_buffer_number = 4
-203> 2018-01-11 10:25:02.285670 7fb2c1f0ee00 0 set rocksdb option min_write_buffer_number_to_merge = 1
-202> 2018-01-11 10:25:02.285673 7fb2c1f0ee00 0 set rocksdb option recycle_log_file_num = 4
-201> 2018-01-11 10:25:02.285675 7fb2c1f0ee00 0 set rocksdb option writable_file_max_buffer_size = 0
-200> 2018-01-11 10:25:02.285679 7fb2c1f0ee00 0 set rocksdb option write_buffer_size = 268435456
-199> 2018-01-11 10:25:02.285699 7fb2c1f0ee00 0 set rocksdb option compaction_readahead_size = 2097152
-198> 2018-01-11 10:25:02.285702 7fb2c1f0ee00 0 set rocksdb option compression = kNoCompression
-197> 2018-01-11 10:25:02.285706 7fb2c1f0ee00 0 set rocksdb option max_write_buffer_number = 4
-196> 2018-01-11 10:25:02.285708 7fb2c1f0ee00 0 set rocksdb option min_write_buffer_number_to_merge = 1
-195> 2018-01-11 10:25:02.285711 7fb2c1f0ee00 0 set rocksdb option recycle_log_file_num = 4
-194> 2018-01-11 10:25:02.285714 7fb2c1f0ee00 0 set rocksdb option writable_file_max_buffer_size = 0
-193> 2018-01-11 10:25:02.285717 7fb2c1f0ee00 0 set rocksdb option write_buffer_size = 268435456
-192> 2018-01-11 10:25:02.285827 7fb2c1f0ee00 4 rocksdb: RocksDB version: 5.4.0

-191> 2018-01-11 10:25:02.285889 7fb2c1f0ee00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
-190> 2018-01-11 10:25:02.285892 7fb2c1f0ee00 4 rocksdb: Compile date Sep 26 2017
-189> 2018-01-11 10:25:02.285894 7fb2c1f0ee00 4 rocksdb: DB SUMMARY

-188> 2018-01-11 10:25:02.285912 7fb2c1f0ee00 4 rocksdb: CURRENT file: CURRENT

-187> 2018-01-11 10:25:02.285913 7fb2c1f0ee00 4 rocksdb: IDENTITY file: IDENTITY

-186> 2018-01-11 10:25:02.285915 7fb2c1f0ee00 4 rocksdb: MANIFEST file: MANIFEST-016219 size: 69928 Bytes

-185> 2018-01-11 10:25:02.285918 7fb2c1f0ee00 4 rocksdb: SST files in db dir, Total Num: 5, files: 015138.sst 015139.sst 015140.sst 016223.sst 016224.sst

-184> 2018-01-11 10:25:02.286104 7fb2c1f0ee00 4 rocksdb: SST files in db.slow dir, Total Num: 378, files: 007717.sst 007748.sst 008161.sst 008669.sst 008670.sst 009336.sst 009514.sst 009515.sst 009516.sst

-183> 2018-01-11 10:25:02.286153 7fb2c1f0ee00 4 rocksdb: Write Ahead Log file in db: 016220.log size: 188643 ;

-182> 2018-01-11 10:25:02.286156 7fb2c1f0ee00 4 rocksdb: Options.error_if_exists: 0
-181> 2018-01-11 10:25:02.286158 7fb2c1f0ee00 4 rocksdb: Options.create_if_missing: 0
-180> 2018-01-11 10:25:02.286159 7fb2c1f0ee00 4 rocksdb: Options.paranoid_checks: 1
-179> 2018-01-11 10:25:02.286160 7fb2c1f0ee00 4 rocksdb: Options.env: 0x556e12141e00
-178> 2018-01-11 10:25:02.286161 7fb2c1f0ee00 4 rocksdb: Options.info_log: 0x556e1235f260
-177> 2018-01-11 10:25:02.286162 7fb2c1f0ee00 4 rocksdb: Options.max_open_files: -1
-176> 2018-01-11 10:25:02.286163 7fb2c1f0ee00 4 rocksdb: Options.max_file_opening_threads: 16
-175> 2018-01-11 10:25:02.286163 7fb2c1f0ee00 4 rocksdb: Options.use_fsync: 0
-174> 2018-01-11 10:25:02.286164 7fb2c1f0ee00 4 rocksdb: Options.max_log_file_size: 0
-173> 2018-01-11 10:25:02.286165 7fb2c1f0ee00 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
-172> 2018-01-11 10:25:02.286166 7fb2c1f0ee00 4 rocksdb: Options.log_file_time_to_roll: 0
-171> 2018-01-11 10:25:02.286167 7fb2c1f0ee00 4 rocksdb: Options.keep_log_file_num: 1000
-170> 2018-01-11 10:25:02.286168 7fb2c1f0ee00 4 rocksdb: Options.recycle_log_file_num: 4
-169> 2018-01-11 10:25:02.286169 7fb2c1f0ee00 4 rocksdb: Options.allow_fallocate: 1
-168> 2018-01-11 10:25:02.286169 7fb2c1f0ee00 4 rocksdb: Options.allow_mmap_reads: 0
-167> 2018-01-11 10:25:02.286170 7fb2c1f0ee00 4 rocksdb: Options.allow_mmap_writes: 0
-166> 2018-01-11 10:25:02.286171 7fb2c1f0ee00 4 rocksdb: Options.use_direct_reads: 0
-165> 2018-01-11 10:25:02.286171 7fb2c1f0ee00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
-164> 2018-01-11 10:25:02.286172 7fb2c1f0ee00 4 rocksdb: Options.create_missing_column_families: 0
-163> 2018-01-11 10:25:02.286172 7fb2c1f0ee00 4 rocksdb: Options.db_log_dir:
-162> 2018-01-11 10:25:02.286234 7fb2c1f0ee00 4 rocksdb: Options.wal_dir: db
-161> 2018-01-11 10:25:02.286236 7fb2c1f0ee00 4 rocksdb: Options.table_cache_numshardbits: 6
-160> 2018-01-11 10:25:02.286237 7fb2c1f0ee00 4 rocksdb: Options.max_subcompactions: 1
-159> 2018-01-11 10:25:02.286238 7fb2c1f0ee00 4 rocksdb: Options.max_background_flushes: 1
-158> 2018-01-11 10:25:02.286239 7fb2c1f0ee00 4 rocksdb: Options.WAL_ttl_seconds: 0
-157> 2018-01-11 10:25:02.286240 7fb2c1f0ee00 4 rocksdb: Options.WAL_size_limit_MB: 0
-156> 2018-01-11 10:25:02.286240 7fb2c1f0ee00 4 rocksdb: Options.manifest_preallocation_size: 4194304
-155> 2018-01-11 10:25:02.286241 7fb2c1f0ee00 4 rocksdb: Options.is_fd_close_on_exec: 1
-154> 2018-01-11 10:25:02.286242 7fb2c1f0ee00 4 rocksdb: Options.advise_random_on_open: 1
-153> 2018-01-11 10:25:02.286242 7fb2c1f0ee00 4 rocksdb: Options.db_write_buffer_size: 0
-152> 2018-01-11 10:25:02.286243 7fb2c1f0ee00 4 rocksdb: Options.access_hint_on_compaction_start: 1
-151> 2018-01-11 10:25:02.286244 7fb2c1f0ee00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
-150> 2018-01-11 10:25:02.286244 7fb2c1f0ee00 4 rocksdb: Options.compaction_readahead_size: 2097152
-149> 2018-01-11 10:25:02.286245 7fb2c1f0ee00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
-148> 2018-01-11 10:25:02.286246 7fb2c1f0ee00 4 rocksdb: Options.writable_file_max_buffer_size: 0
-147> 2018-01-11 10:25:02.286246 7fb2c1f0ee00 4 rocksdb: Options.use_adaptive_mutex: 0
-146> 2018-01-11 10:25:02.286247 7fb2c1f0ee00 4 rocksdb: Options.rate_limiter: (nil)
-145> 2018-01-11 10:25:02.286255 7fb2c1f0ee00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
-144> 2018-01-11 10:25:02.286256 7fb2c1f0ee00 4 rocksdb: Options.bytes_per_sync: 0
-143> 2018-01-11 10:25:02.286257 7fb2c1f0ee00 4 rocksdb: Options.wal_bytes_per_sync: 0
-142> 2018-01-11 10:25:02.286257 7fb2c1f0ee00 4 rocksdb: Options.wal_recovery_mode: 0
-141> 2018-01-11 10:25:02.286258 7fb2c1f0ee00 4 rocksdb: Options.enable_thread_tracking: 0
-140> 2018-01-11 10:25:02.286259 7fb2c1f0ee00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
-139> 2018-01-11 10:25:02.286260 7fb2c1f0ee00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
-138> 2018-01-11 10:25:02.286260 7fb2c1f0ee00 4 rocksdb: Options.write_thread_max_yield_usec: 100
-137> 2018-01-11 10:25:02.286261 7fb2c1f0ee00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
-136> 2018-01-11 10:25:02.286262 7fb2c1f0ee00 4 rocksdb: Options.row_cache: None
-135> 2018-01-11 10:25:02.286262 7fb2c1f0ee00 4 rocksdb: Options.wal_filter: None
-134> 2018-01-11 10:25:02.286263 7fb2c1f0ee00 4 rocksdb: Options.avoid_flush_during_recovery: 0
-133> 2018-01-11 10:25:02.286264 7fb2c1f0ee00 4 rocksdb: Options.base_background_compactions: 1
-132> 2018-01-11 10:25:02.286265 7fb2c1f0ee00 4 rocksdb: Options.max_background_compactions: 1
-131> 2018-01-11 10:25:02.286266 7fb2c1f0ee00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
-130> 2018-01-11 10:25:02.286266 7fb2c1f0ee00 4 rocksdb: Options.delayed_write_rate : 16777216
-129> 2018-01-11 10:25:02.286267 7fb2c1f0ee00 4 rocksdb: Options.max_total_wal_size: 0
-128> 2018-01-11 10:25:02.286268 7fb2c1f0ee00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
-127> 2018-01-11 10:25:02.286269 7fb2c1f0ee00 4 rocksdb: Options.stats_dump_period_sec: 600
-126> 2018-01-11 10:25:02.286269 7fb2c1f0ee00 4 rocksdb: Compression algorithms supported:
-125> 2018-01-11 10:25:02.286270 7fb2c1f0ee00 4 rocksdb: Snappy supported: 0
-124> 2018-01-11 10:25:02.286271 7fb2c1f0ee00 4 rocksdb: Zlib supported: 0
-123> 2018-01-11 10:25:02.286272 7fb2c1f0ee00 4 rocksdb: Bzip supported: 0
-122> 2018-01-11 10:25:02.286272 7fb2c1f0ee00 4 rocksdb: LZ4 supported: 0
-121> 2018-01-11 10:25:02.286273 7fb2c1f0ee00 4 rocksdb: ZSTD supported: 0
-120> 2018-01-11 10:25:02.286274 7fb2c1f0ee00 4 rocksdb: Fast CRC32 supported: 0
-119> 2018-01-11 10:25:02.286505 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2609] Recovering from manifest file: MANIFEST-016219

-118> 2018-01-11 10:25:02.286583 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/column_family.cc:407] --------------- Options for column family [default]:

-117> 2018-01-11 10:25:02.286617 7fb2c1f0ee00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
-116> 2018-01-11 10:25:02.286670 7fb2c1f0ee00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
-115> 2018-01-11 10:25:02.286702 7fb2c1f0ee00 4 rocksdb: Options.compaction_filter: None
-114> 2018-01-11 10:25:02.286734 7fb2c1f0ee00 4 rocksdb: Options.compaction_filter_factory: None
-113> 2018-01-11 10:25:02.286766 7fb2c1f0ee00 4 rocksdb: Options.memtable_factory: SkipListFactory
-112> 2018-01-11 10:25:02.286797 7fb2c1f0ee00 4 rocksdb: Options.table_factory: BlockBasedTable
-111> 2018-01-11 10:25:02.286850 7fb2c1f0ee00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e11d80170)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 1
pin_l0_filter_and_index_blocks_in_cache: 1
index_type: 0
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x556e12076e70
block_cache_name: LRUCache
block_cache_options:
capacity : 536870912
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
format_version: 2

-110> 2018-01-11 10:25:02.286886 7fb2c1f0ee00 4 rocksdb: Options.write_buffer_size: 268435456
-109> 2018-01-11 10:25:02.286889 7fb2c1f0ee00 4 rocksdb: Options.max_write_buffer_number: 4
-108> 2018-01-11 10:25:02.286891 7fb2c1f0ee00 4 rocksdb: Options.compression: NoCompression
-107> 2018-01-11 10:25:02.286892 7fb2c1f0ee00 4 rocksdb: Options.bottommost_compression: Disabled
-106> 2018-01-11 10:25:02.286893 7fb2c1f0ee00 4 rocksdb: Options.prefix_extractor: nullptr
-105> 2018-01-11 10:25:02.286894 7fb2c1f0ee00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
-104> 2018-01-11 10:25:02.286895 7fb2c1f0ee00 4 rocksdb: Options.num_levels: 7
-103> 2018-01-11 10:25:02.286896 7fb2c1f0ee00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
-102> 2018-01-11 10:25:02.286912 7fb2c1f0ee00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
-101> 2018-01-11 10:25:02.286943 7fb2c1f0ee00 4 rocksdb: Options.compression_opts.window_bits: -14
-100> 2018-01-11 10:25:02.286975 7fb2c1f0ee00 4 rocksdb: Options.compression_opts.level: -1
-99> 2018-01-11 10:25:02.287006 7fb2c1f0ee00 4 rocksdb: Options.compression_opts.strategy: 0
-98> 2018-01-11 10:25:02.287038 7fb2c1f0ee00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
-97> 2018-01-11 10:25:02.287069 7fb2c1f0ee00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
-96> 2018-01-11 10:25:02.287101 7fb2c1f0ee00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
-95> 2018-01-11 10:25:02.287132 7fb2c1f0ee00 4 rocksdb: Options.level0_stop_writes_trigger: 36
-94> 2018-01-11 10:25:02.287163 7fb2c1f0ee00 4 rocksdb: Options.target_file_size_base: 67108864
-93> 2018-01-11 10:25:02.287195 7fb2c1f0ee00 4 rocksdb: Options.target_file_size_multiplier: 1
-92> 2018-01-11 10:25:02.287226 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
-91> 2018-01-11 10:25:02.287258 7fb2c1f0ee00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
-90> 2018-01-11 10:25:02.287289 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
-89> 2018-01-11 10:25:02.287323 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
-88> 2018-01-11 10:25:02.287354 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
-87> 2018-01-11 10:25:02.287385 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
-86> 2018-01-11 10:25:02.287416 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
-85> 2018-01-11 10:25:02.287448 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
-84> 2018-01-11 10:25:02.287479 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
-83> 2018-01-11 10:25:02.287510 7fb2c1f0ee00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
-82> 2018-01-11 10:25:02.287541 7fb2c1f0ee00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
-81> 2018-01-11 10:25:02.287573 7fb2c1f0ee00 4 rocksdb: Options.max_compaction_bytes: 1677721600
-80> 2018-01-11 10:25:02.287604 7fb2c1f0ee00 4 rocksdb: Options.arena_block_size: 33554432
-79> 2018-01-11 10:25:02.287635 7fb2c1f0ee00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
-78> 2018-01-11 10:25:02.287666 7fb2c1f0ee00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
-77> 2018-01-11 10:25:02.287697 7fb2c1f0ee00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
-76> 2018-01-11 10:25:02.287729 7fb2c1f0ee00 4 rocksdb: Options.disable_auto_compactions: 0
-75> 2018-01-11 10:25:02.287760 7fb2c1f0ee00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
-74> 2018-01-11 10:25:02.287792 7fb2c1f0ee00 4 rocksdb: Options.compaction_pri: kByCompensatedSize
-73> 2018-01-11 10:25:02.287823 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
-72> 2018-01-11 10:25:02.287854 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
-71> 2018-01-11 10:25:02.287885 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
-70> 2018-01-11 10:25:02.287917 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
-69> 2018-01-11 10:25:02.287948 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
-68> 2018-01-11 10:25:02.287979 7fb2c1f0ee00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
-67> 2018-01-11 10:25:02.288010 7fb2c1f0ee00 4 rocksdb: Options.table_properties_collectors:
-66> 2018-01-11 10:25:02.288041 7fb2c1f0ee00 4 rocksdb: Options.inplace_update_support: 0
-65> 2018-01-11 10:25:02.288072 7fb2c1f0ee00 4 rocksdb: Options.inplace_update_num_locks: 10000
-64> 2018-01-11 10:25:02.288104 7fb2c1f0ee00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
-63> 2018-01-11 10:25:02.288136 7fb2c1f0ee00 4 rocksdb: Options.memtable_huge_page_size: 0
-62> 2018-01-11 10:25:02.288167 7fb2c1f0ee00 4 rocksdb: Options.bloom_locality: 0
-61> 2018-01-11 10:25:02.288199 7fb2c1f0ee00 4 rocksdb: Options.max_successive_merges: 0
-60> 2018-01-11 10:25:02.288230 7fb2c1f0ee00 4 rocksdb: Options.optimize_filters_for_hits: 0
-59> 2018-01-11 10:25:02.288261 7fb2c1f0ee00 4 rocksdb: Options.paranoid_file_checks: 0
-58> 2018-01-11 10:25:02.288292 7fb2c1f0ee00 4 rocksdb: Options.force_consistency_checks: 0
-57> 2018-01-11 10:25:02.288323 7fb2c1f0ee00 4 rocksdb: Options.report_bg_io_stats: 0
-56> 2018-01-11 10:25:05.651896 7fb2bb118700 1 do_command 'perf dump' '
-55> 2018-01-11 10:25:05.652448 7fb2bb118700 1 do_command 'perf dump' 'result is 16218 bytes
-54> 2018-01-11 10:25:06.698124 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2859] Recovered from manifest file:db/MANIFEST-016219 succeeded,manifest_file_number is 16219, next_file_number is 16226, last_sequence is 341937396, log_number is 0,prev_log_number is 0,max_column_family is 0

-53> 2018-01-11 10:25:06.698171 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2867] Column family [default] (ID 0), log number is 16218

-52> 2018-01-11 10:25:06.698877 7fb2c1f0ee00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662706698868, "job": 1, "event": "recovery_started", "log_files": [16220]}
-51> 2018-01-11 10:25:06.698898 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:482] Recovering log #16220 mode 0
-50> 2018-01-11 10:25:06.707792 7fb2c1f0ee00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:815] [default] [WriteLevel0TableForRecovery] Level-0 table #16226: started
-49> 2018-01-11 10:25:06.713651 7fb2c1f0ee00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662706713639, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 16226, "file_size": 115027, "table_properties": {"data_size": 108793, "index_size": 865, "filter_size": 4393, "raw_key_size": 71288, "raw_average_key_size": 42, "raw_value_size": 49383, "raw_average_value_size": 29, "num_data_blocks": 27, "num_entries": 1659, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
-48> 2018-01-11 10:25:06.713676 7fb2c1f0ee00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:847] [default] [WriteLevel0TableForRecovery] Level-0 table #16226: 115027 bytes OK
-47> 2018-01-11 10:25:06.714015 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/version_set.cc:2395] Creating manifest 16227

-46> 2018-01-11 10:25:06.716388 7fb2c1f0ee00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1515662706716385, "job": 1, "event": "recovery_finished"}
-45> 2018-01-11 10:25:06.716689 7fb2c1f0ee00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_files.cc:307] [JOB 2] Delete db//MANIFEST-016219 type=3 #16219 -- OK

-44> 2018-01-11 10:25:06.716763 7fb2c1f0ee00 5 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_files.cc:307] [JOB 2] Delete db//016220.log type=0 #16220 -- OK

-43> 2018-01-11 10:25:06.718191 7fb2c1f0ee00 4 rocksdb: [/build/ceph-12.2.1/src/rocksdb/db/db_impl_open.cc:1063] DB pointer 0x556e12085a00
-42> 2018-01-11 10:25:06.718285 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
-41> 2018-01-11 10:25:06.776871 7fb2c1f0ee00 1 freelist init
-40> 2018-01-11 10:25:06.841621 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc opening allocation metadata
-39> 2018-01-11 10:25:15.588192 7fb2bb118700 1 do_command 'perf dump' '
-38> 2018-01-11 10:25:15.588539 7fb2bb118700 1 do_command 'perf dump' 'result is 17482 bytes
-37> 2018-01-11 10:25:16.883919 7fb2c1f0ee00 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_alloc loaded 318 G in 658679 extents
-36> 2018-01-11 10:25:17.010960 7fb2c1f0ee00 0 <cls> /build/ceph-12.2.1/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
-35> 2018-01-11 10:25:17.015818 7fb2c1f0ee00 0 <cls> /build/ceph-12.2.1/src/cls/hello/cls_hello.cc:296: loading cls_hello
-34> 2018-01-11 10:25:17.015887 7fb2c1f0ee00 0 _get_class not permitted to load kvs
-33> 2018-01-11 10:25:17.016319 7fb2c1f0ee00 0 _get_class not permitted to load lua
-32> 2018-01-11 10:25:17.018489 7fb2c1f0ee00 0 _get_class not permitted to load sdk
-31> 2018-01-11 10:25:17.053649 7fb2c1f0ee00 0 osd.2 15344 crush map has features 2200130813952, adjusting msgr requires for clients
-30> 2018-01-11 10:25:17.054038 7fb2c1f0ee00 0 osd.2 15344 crush map has features 2200130813952 was 8705, adjusting msgr requires for mons
-29> 2018-01-11 10:25:17.054337 7fb2c1f0ee00 0 osd.2 15344 crush map has features 720578140510109696, adjusting msgr requires for osds
-28> 2018-01-11 10:25:25.574245 7fb2bb118700 1 do_command 'perf dump' '
-27> 2018-01-11 10:25:25.574620 7fb2bb118700 1 do_command 'perf dump' 'result is 22053 bytes
-26> 2018-01-11 10:25:34.303410 7fb2c1f0ee00 0 osd.2 15344 load_pgs
-25> 2018-01-11 10:25:35.609851 7fb2bb118700 1 do_command 'perf dump' '
-24> 2018-01-11 10:25:35.610543 7fb2bb118700 1 do_command 'perf dump' 'result is 22095 bytes
-23> 2018-01-11 10:25:45.596080 7fb2bb118700 1 do_command 'perf dump' '
-22> 2018-01-11 10:25:45.596741 7fb2bb118700 1 do_command 'perf dump' 'result is 22124 bytes
-21> 2018-01-11 10:25:55.553710 7fb2bb118700 1 do_command 'perf dump' '
-20> 2018-01-11 10:25:55.554084 7fb2bb118700 1 do_command 'perf dump' 'result is 22147 bytes
-19> 2018-01-11 10:26:05.577249 7fb2bb118700 1 do_command 'perf dump' '
-18> 2018-01-11 10:26:05.577839 7fb2bb118700 1 do_command 'perf dump' 'result is 22150 bytes
-17> 2018-01-11 10:26:15.555816 7fb2bb118700 1 do_command 'perf dump' '
-16> 2018-01-11 10:26:15.556548 7fb2bb118700 1 do_command 'perf dump' 'result is 22150 bytes
-15> 2018-01-11 10:26:25.553626 7fb2bb118700 1 do_command 'perf dump' '
-14> 2018-01-11 10:26:25.553989 7fb2bb118700 1 do_command 'perf dump' 'result is 22151 bytes
-13> 2018-01-11 10:26:35.577307 7fb2bb118700 1 do_command 'perf dump' '
-12> 2018-01-11 10:26:35.577837 7fb2bb118700 1 do_command 'perf dump' 'result is 22151 bytes
-11> 2018-01-11 10:26:45.564277 7fb2bb118700 1 do_command 'perf dump' '
-10> 2018-01-11 10:26:45.564639 7fb2bb118700 1 do_command 'perf dump' 'result is 22152 bytes
-9> 2018-01-11 10:26:55.556321 7fb2bb118700 1 do_command 'perf dump' '
-8> 2018-01-11 10:26:55.556694 7fb2bb118700 1 do_command 'perf dump' 'result is 22152 bytes
-7> 2018-01-11 10:27:05.616404 7fb2bb118700 1 do_command 'perf dump' '
-6> 2018-01-11 10:27:05.617145 7fb2bb118700 1 do_command 'perf dump' 'result is 22152 bytes
-5> 2018-01-11 10:27:15.569310 7fb2bb118700 1 do_command 'perf dump' '
-4> 2018-01-11 10:27:15.569954 7fb2bb118700 1 do_command 'perf dump' 'result is 22153 bytes
-3> 2018-01-11 10:27:20.420199 7fb2c1f0ee00 0 osd.2 15344 load_pgs opened 5429 pgs
-2> 2018-01-11 10:27:25.697710 7fb2bb118700 1 do_command 'perf dump' '
-1> 2018-01-11 10:27:25.698214 7fb2bb118700 1 do_command 'perf dump' 'result is 22163 bytes
0> 2018-01-11 10:27:26.024880 7fb2c1f0ee00 -1 /build/ceph-12.2.1/src/osd/osd_types.cc: In function 'virtual void pi_compact_rep::add_interval(bool, const PastIntervals::pg_interval_t&)' thread 7fb2c1f0ee00 time 2018-01-11 10:27:26.021986
/build/ceph-12.2.1/src/osd/osd_types.cc: 3205: FAILED assert(interval.last > last)

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x556e06bfb3f2]
2: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x556e06884b75]
3: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x556e06862376]
4: (OSD::build_past_intervals_parallel()+0xe71) [0x556e06617f01]
5: (OSD::load_pgs()+0x14fb) [0x556e0661a92b]
6: (OSD::init()+0x2217) [0x556e06634b37]
7: (main()+0x2eb8) [0x556e065451f8]
8: (__libc_start_main()+0xf0) [0x7fb2bf37f830]
9: (_start()+0x29) [0x556e065d0a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 0 lockdep
0/ 0 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 0 buffer
0/ 0 timer
0/ 0 filer
0/ 1 striper
0/ 0 objecter
0/ 0 rados
0/ 0 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 0 journaler
0/ 0 objectcacher
0/ 0 client
0/ 0 osd
0/ 0 optracker
0/ 0 objclass
0/ 0 filestore
0/ 0 journal
0/ 0 ms
0/ 0 mon
0/ 0 monc
0/ 0 paxos
0/ 0 tp
0/ 0 auth
1/ 5 crypto
0/ 0 finisher
1/ 5 heartbeatmap
0/ 0 perfcounter
0/ 0 rgw
1/10 civetweb
1/ 5 javaclient
0/ 0 asok
0/ 0 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2018-01-11 10:27:26.033988 7fb2c1f0ee00 -1 *** Caught signal (Aborted) **
in thread 7fb2c1f0ee00 thread_name:ceph-osd

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (()+0xa5a634) [0x556e06bb8634]
2: (()+0x11390) [0x7fb2c03f9390]
3: (gsignal()+0x38) [0x7fb2bf394428]
4: (abort()+0x16a) [0x7fb2bf39602a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x556e06bfb57e]
6: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x556e06884b75]
7: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x556e06862376]
8: (OSD::build_past_intervals_parallel()+0xe71) [0x556e06617f01]
9: (OSD::load_pgs()+0x14fb) [0x556e0661a92b]
10: (OSD::init()+0x2217) [0x556e06634b37]
11: (main()+0x2eb8) [0x556e065451f8]
12: (__libc_start_main()+0xf0) [0x7fb2bf37f830]
13: (_start()+0x29) [0x556e065d0a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
0> 2018-01-11 10:27:26.033988 7fb2c1f0ee00 -1 *** Caught signal (Aborted) **
in thread 7fb2c1f0ee00 thread_name:ceph-osd

ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (()+0xa5a634) [0x556e06bb8634]
2: (()+0x11390) [0x7fb2c03f9390]
3: (gsignal()+0x38) [0x7fb2bf394428]
4: (abort()+0x16a) [0x7fb2bf39602a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x556e06bfb57e]
6: (pi_compact_rep::add_interval(bool, PastIntervals::pg_interval_t const&)+0x435) [0x556e06884b75]
7: (PastIntervals::check_new_interval(int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, int, int, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, unsigned int, unsigned int, std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, pg_t, IsPGRecoverablePredicate*, PastIntervals*, std::ostream*)+0x396) [0x556e06862376]
8: (OSD::build_past_intervals_parallel()+0xe71) [0x556e06617f01]
9: (OSD::load_pgs()+0x14fb) [0x556e0661a92b]
10: (OSD::init()+0x2217) [0x556e06634b37]
11: (main()+0x2eb8) [0x556e065451f8]
12: (__libc_start_main()+0xf0) [0x7fb2bf37f830]
13: (_start()+0x29) [0x556e065d0a59]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 0 lockdep
0/ 0 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 0 buffer
0/ 0 timer
0/ 0 filer
0/ 1 striper
0/ 0 objecter
0/ 0 rados
0/ 0 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 0 journaler
0/ 0 objectcacher
0/ 0 client
0/ 0 osd
0/ 0 optracker
0/ 0 objclass
0/ 0 filestore
0/ 0 journal
0/ 0 ms
0/ 0 mon
0/ 0 monc
0/ 0 paxos
0/ 0 tp
0/ 0 auth
1/ 5 crypto
0/ 0 finisher
1/ 5 heartbeatmap
0/ 0 perfcounter
0/ 0 rgw
1/10 civetweb
1/ 5 javaclient
0/ 0 asok
0/ 0 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
(2-2/7)