Project

General

Profile

Bug #24968 » ceph-osd.24.log.txt

2nd error osd start - Markus Stockhausen, 07/21/2018 07:29 PM

 
2018-07-21 21:20:44.512118 7f3d23173d80 0 set uid:gid to 167:167 (ceph:ceph)
2018-07-21 21:20:44.512138 7f3d23173d80 0 ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable), process ceph-osd, pid 14561
2018-07-21 21:20:44.521794 7f3d23173d80 0 pidfile_write: ignore empty --pid-file
2018-07-21 21:20:44.562844 7f3d23173d80 0 load: jerasure load: lrc load: isa
2018-07-21 21:20:44.562930 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block type kernel
2018-07-21 21:20:44.562944 7f3d23173d80 1 bdev(0x55ad4e392c00 /var/lib/ceph/osd/ceph-24/block) open path /var/lib/ceph/osd/ceph-24/block
2018-07-21 21:20:44.563244 7f3d23173d80 1 bdev(0x55ad4e392c00 /var/lib/ceph/osd/ceph-24/block) open size 1919711969280 (0x1bef7c00000, 1787 GB) block_size 4096 (4096 B) rotational
2018-07-21 21:20:44.563493 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _set_cache_sizes cache_size 209715200 meta 0.01 kv 0.99 data 0
2018-07-21 21:20:44.563520 7f3d23173d80 1 bdev(0x55ad4e392c00 /var/lib/ceph/osd/ceph-24/block) close
2018-07-21 21:20:44.827802 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _mount path /var/lib/ceph/osd/ceph-24
2018-07-21 21:20:44.828308 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block type kernel
2018-07-21 21:20:44.828316 7f3d23173d80 1 bdev(0x55ad4e2b7600 /var/lib/ceph/osd/ceph-24/block) open path /var/lib/ceph/osd/ceph-24/block
2018-07-21 21:20:44.828556 7f3d23173d80 1 bdev(0x55ad4e2b7600 /var/lib/ceph/osd/ceph-24/block) open size 1919711969280 (0x1bef7c00000, 1787 GB) block_size 4096 (4096 B) rotational
2018-07-21 21:20:44.828790 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _set_cache_sizes cache_size 209715200 meta 0.01 kv 0.99 data 0
2018-07-21 21:20:44.828870 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block.db type kernel
2018-07-21 21:20:44.828876 7f3d23173d80 1 bdev(0x55ad4e2b7a00 /var/lib/ceph/osd/ceph-24/block.db) open path /var/lib/ceph/osd/ceph-24/block.db
2018-07-21 21:20:44.829082 7f3d23173d80 1 bdev(0x55ad4e2b7a00 /var/lib/ceph/osd/ceph-24/block.db) open size 19999490048 (0x4a8100000, 19073 MB) block_size 4096 (4096 B) non-rotational
2018-07-21 21:20:44.829091 7f3d23173d80 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-24/block.db size 19073 MB
2018-07-21 21:20:44.829317 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block type kernel
2018-07-21 21:20:44.829325 7f3d23173d80 1 bdev(0x55ad4e395a00 /var/lib/ceph/osd/ceph-24/block) open path /var/lib/ceph/osd/ceph-24/block
2018-07-21 21:20:44.829500 7f3d23173d80 1 bdev(0x55ad4e395a00 /var/lib/ceph/osd/ceph-24/block) open size 1919711969280 (0x1bef7c00000, 1787 GB) block_size 4096 (4096 B) rotational
2018-07-21 21:20:44.829520 7f3d23173d80 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-24/block size 1787 GB
2018-07-21 21:20:44.829531 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block.wal type kernel
2018-07-21 21:20:44.829534 7f3d23173d80 1 bdev(0x55ad4e394e00 /var/lib/ceph/osd/ceph-24/block.wal) open path /var/lib/ceph/osd/ceph-24/block.wal
2018-07-21 21:20:44.829706 7f3d23173d80 1 bdev(0x55ad4e394e00 /var/lib/ceph/osd/ceph-24/block.wal) open size 1999634432 (0x77300000, 1907 MB) block_size 4096 (4096 B) non-rotational
2018-07-21 21:20:44.829718 7f3d23173d80 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-24/block.wal size 1907 MB
2018-07-21 21:20:44.829905 7f3d23173d80 1 bluefs mount
2018-07-21 21:20:44.838675 7f3d23173d80 0 set rocksdb option compaction_readahead_size = 2097152
2018-07-21 21:20:44.838692 7f3d23173d80 0 set rocksdb option compression = kNoCompression
2018-07-21 21:20:44.838698 7f3d23173d80 0 set rocksdb option max_write_buffer_number = 4
2018-07-21 21:20:44.838702 7f3d23173d80 0 set rocksdb option min_write_buffer_number_to_merge = 1
2018-07-21 21:20:44.838706 7f3d23173d80 0 set rocksdb option recycle_log_file_num = 4
2018-07-21 21:20:44.838709 7f3d23173d80 0 set rocksdb option writable_file_max_buffer_size = 0
2018-07-21 21:20:44.838713 7f3d23173d80 0 set rocksdb option write_buffer_size = 268435456
2018-07-21 21:20:44.838742 7f3d23173d80 0 set rocksdb option compaction_readahead_size = 2097152
2018-07-21 21:20:44.838750 7f3d23173d80 0 set rocksdb option compression = kNoCompression
2018-07-21 21:20:44.838754 7f3d23173d80 0 set rocksdb option max_write_buffer_number = 4
2018-07-21 21:20:44.838758 7f3d23173d80 0 set rocksdb option min_write_buffer_number_to_merge = 1
2018-07-21 21:20:44.838761 7f3d23173d80 0 set rocksdb option recycle_log_file_num = 4
2018-07-21 21:20:44.838764 7f3d23173d80 0 set rocksdb option writable_file_max_buffer_size = 0
2018-07-21 21:20:44.838767 7f3d23173d80 0 set rocksdb option write_buffer_size = 268435456
2018-07-21 21:20:44.838949 7f3d23173d80 4 rocksdb: RocksDB version: 5.4.0

2018-07-21 21:20:44.838966 7f3d23173d80 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
2018-07-21 21:20:44.838968 7f3d23173d80 4 rocksdb: Compile date Jul 16 2018
2018-07-21 21:20:44.838979 7f3d23173d80 4 rocksdb: DB SUMMARY

2018-07-21 21:20:44.839008 7f3d23173d80 4 rocksdb: CURRENT file: CURRENT

2018-07-21 21:20:44.839012 7f3d23173d80 4 rocksdb: IDENTITY file: IDENTITY

2018-07-21 21:20:44.839018 7f3d23173d80 4 rocksdb: MANIFEST file: MANIFEST-000157 size: 350 Bytes

2018-07-21 21:20:44.839021 7f3d23173d80 4 rocksdb: SST files in db dir, Total Num: 6, files: 000004.sst 000010.sst 000013.sst 000019.sst 000021.sst 000024.sst

2018-07-21 21:20:44.839025 7f3d23173d80 4 rocksdb: SST files in db.slow dir, Total Num: 0, files:

2018-07-21 21:20:44.839030 7f3d23173d80 4 rocksdb: Write Ahead Log file in db.wal: 000158.log size: 0 ;

2018-07-21 21:20:44.839033 7f3d23173d80 4 rocksdb: Options.error_if_exists: 0
2018-07-21 21:20:44.839041 7f3d23173d80 4 rocksdb: Options.create_if_missing: 0
2018-07-21 21:20:44.839043 7f3d23173d80 4 rocksdb: Options.paranoid_checks: 1
2018-07-21 21:20:44.839044 7f3d23173d80 4 rocksdb: Options.env: 0x55ad4e38d0a0
2018-07-21 21:20:44.839045 7f3d23173d80 4 rocksdb: Options.info_log: 0x55ad4e38d520
2018-07-21 21:20:44.839046 7f3d23173d80 4 rocksdb: Options.max_open_files: -1
2018-07-21 21:20:44.839047 7f3d23173d80 4 rocksdb: Options.max_file_opening_threads: 16
2018-07-21 21:20:44.839048 7f3d23173d80 4 rocksdb: Options.use_fsync: 0
2018-07-21 21:20:44.839049 7f3d23173d80 4 rocksdb: Options.max_log_file_size: 0
2018-07-21 21:20:44.839050 7f3d23173d80 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
2018-07-21 21:20:44.839051 7f3d23173d80 4 rocksdb: Options.log_file_time_to_roll: 0
2018-07-21 21:20:44.839052 7f3d23173d80 4 rocksdb: Options.keep_log_file_num: 1000
2018-07-21 21:20:44.839053 7f3d23173d80 4 rocksdb: Options.recycle_log_file_num: 4
2018-07-21 21:20:44.839054 7f3d23173d80 4 rocksdb: Options.allow_fallocate: 1
2018-07-21 21:20:44.839054 7f3d23173d80 4 rocksdb: Options.allow_mmap_reads: 0
2018-07-21 21:20:44.839055 7f3d23173d80 4 rocksdb: Options.allow_mmap_writes: 0
2018-07-21 21:20:44.839056 7f3d23173d80 4 rocksdb: Options.use_direct_reads: 0
2018-07-21 21:20:44.839057 7f3d23173d80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
2018-07-21 21:20:44.839058 7f3d23173d80 4 rocksdb: Options.create_missing_column_families: 0
2018-07-21 21:20:44.839058 7f3d23173d80 4 rocksdb: Options.db_log_dir:
2018-07-21 21:20:44.839059 7f3d23173d80 4 rocksdb: Options.wal_dir: db.wal
2018-07-21 21:20:44.839060 7f3d23173d80 4 rocksdb: Options.table_cache_numshardbits: 6
2018-07-21 21:20:44.839061 7f3d23173d80 4 rocksdb: Options.max_subcompactions: 1
2018-07-21 21:20:44.839062 7f3d23173d80 4 rocksdb: Options.max_background_flushes: 1
2018-07-21 21:20:44.839062 7f3d23173d80 4 rocksdb: Options.WAL_ttl_seconds: 0
2018-07-21 21:20:44.839063 7f3d23173d80 4 rocksdb: Options.WAL_size_limit_MB: 0
2018-07-21 21:20:44.839064 7f3d23173d80 4 rocksdb: Options.manifest_preallocation_size: 4194304
2018-07-21 21:20:44.839065 7f3d23173d80 4 rocksdb: Options.is_fd_close_on_exec: 1
2018-07-21 21:20:44.839066 7f3d23173d80 4 rocksdb: Options.advise_random_on_open: 1
2018-07-21 21:20:44.839066 7f3d23173d80 4 rocksdb: Options.db_write_buffer_size: 0
2018-07-21 21:20:44.839067 7f3d23173d80 4 rocksdb: Options.access_hint_on_compaction_start: 1
2018-07-21 21:20:44.839068 7f3d23173d80 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
2018-07-21 21:20:44.839069 7f3d23173d80 4 rocksdb: Options.compaction_readahead_size: 2097152
2018-07-21 21:20:44.839069 7f3d23173d80 4 rocksdb: Options.random_access_max_buffer_size: 1048576
2018-07-21 21:20:44.839070 7f3d23173d80 4 rocksdb: Options.writable_file_max_buffer_size: 0
2018-07-21 21:20:44.839071 7f3d23173d80 4 rocksdb: Options.use_adaptive_mutex: 0
2018-07-21 21:20:44.839072 7f3d23173d80 4 rocksdb: Options.rate_limiter: (nil)
2018-07-21 21:20:44.839077 7f3d23173d80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
2018-07-21 21:20:44.839079 7f3d23173d80 4 rocksdb: Options.bytes_per_sync: 0
2018-07-21 21:20:44.839079 7f3d23173d80 4 rocksdb: Options.wal_bytes_per_sync: 0
2018-07-21 21:20:44.839080 7f3d23173d80 4 rocksdb: Options.wal_recovery_mode: 0
2018-07-21 21:20:44.839082 7f3d23173d80 4 rocksdb: Options.enable_thread_tracking: 0
2018-07-21 21:20:44.839082 7f3d23173d80 4 rocksdb: Options.allow_concurrent_memtable_write: 1
2018-07-21 21:20:44.839083 7f3d23173d80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
2018-07-21 21:20:44.839084 7f3d23173d80 4 rocksdb: Options.write_thread_max_yield_usec: 100
2018-07-21 21:20:44.839085 7f3d23173d80 4 rocksdb: Options.write_thread_slow_yield_usec: 3
2018-07-21 21:20:44.839085 7f3d23173d80 4 rocksdb: Options.row_cache: None
2018-07-21 21:20:44.839086 7f3d23173d80 4 rocksdb: Options.wal_filter: None
2018-07-21 21:20:44.839087 7f3d23173d80 4 rocksdb: Options.avoid_flush_during_recovery: 0
2018-07-21 21:20:44.839088 7f3d23173d80 4 rocksdb: Options.base_background_compactions: 1
2018-07-21 21:20:44.839089 7f3d23173d80 4 rocksdb: Options.max_background_compactions: 1
2018-07-21 21:20:44.839089 7f3d23173d80 4 rocksdb: Options.avoid_flush_during_shutdown: 0
2018-07-21 21:20:44.839090 7f3d23173d80 4 rocksdb: Options.delayed_write_rate : 16777216
2018-07-21 21:20:44.839091 7f3d23173d80 4 rocksdb: Options.max_total_wal_size: 0
2018-07-21 21:20:44.839092 7f3d23173d80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
2018-07-21 21:20:44.839093 7f3d23173d80 4 rocksdb: Options.stats_dump_period_sec: 600
2018-07-21 21:20:44.839094 7f3d23173d80 4 rocksdb: Compression algorithms supported:
2018-07-21 21:20:44.839095 7f3d23173d80 4 rocksdb: Snappy supported: 1
2018-07-21 21:20:44.839096 7f3d23173d80 4 rocksdb: Zlib supported: 0
2018-07-21 21:20:44.839096 7f3d23173d80 4 rocksdb: Bzip supported: 0
2018-07-21 21:20:44.839097 7f3d23173d80 4 rocksdb: LZ4 supported: 0
2018-07-21 21:20:44.839098 7f3d23173d80 4 rocksdb: ZSTD supported: 0
2018-07-21 21:20:44.839099 7f3d23173d80 4 rocksdb: Fast CRC32 supported: 1
2018-07-21 21:20:44.839340 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/version_set.cc:2609] Recovering from manifest file: MANIFEST-000157

2018-07-21 21:20:44.839413 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/column_family.cc:407] --------------- Options for column family [default]:

2018-07-21 21:20:44.839423 7f3d23173d80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
2018-07-21 21:20:44.839431 7f3d23173d80 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
2018-07-21 21:20:44.839433 7f3d23173d80 4 rocksdb: Options.compaction_filter: None
2018-07-21 21:20:44.839435 7f3d23173d80 4 rocksdb: Options.compaction_filter_factory: None
2018-07-21 21:20:44.839436 7f3d23173d80 4 rocksdb: Options.memtable_factory: SkipListFactory
2018-07-21 21:20:44.839439 7f3d23173d80 4 rocksdb: Options.table_factory: BlockBasedTable
2018-07-21 21:20:44.839460 7f3d23173d80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ad4e01e228)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 1
pin_l0_filter_and_index_blocks_in_cache: 1
index_type: 0
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x55ad4e4900f8
block_cache_name: LRUCache
block_cache_options:
capacity : 207618048
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
format_version: 2

2018-07-21 21:20:44.839465 7f3d23173d80 4 rocksdb: Options.write_buffer_size: 268435456
2018-07-21 21:20:44.839471 7f3d23173d80 4 rocksdb: Options.max_write_buffer_number: 4
2018-07-21 21:20:44.839473 7f3d23173d80 4 rocksdb: Options.compression: NoCompression
2018-07-21 21:20:44.839476 7f3d23173d80 4 rocksdb: Options.bottommost_compression: Disabled
2018-07-21 21:20:44.839478 7f3d23173d80 4 rocksdb: Options.prefix_extractor: nullptr
2018-07-21 21:20:44.839479 7f3d23173d80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
2018-07-21 21:20:44.839481 7f3d23173d80 4 rocksdb: Options.num_levels: 7
2018-07-21 21:20:44.839482 7f3d23173d80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
2018-07-21 21:20:44.839483 7f3d23173d80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
2018-07-21 21:20:44.839485 7f3d23173d80 4 rocksdb: Options.compression_opts.window_bits: -14
2018-07-21 21:20:44.839486 7f3d23173d80 4 rocksdb: Options.compression_opts.level: -1
2018-07-21 21:20:44.839488 7f3d23173d80 4 rocksdb: Options.compression_opts.strategy: 0
2018-07-21 21:20:44.839489 7f3d23173d80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
2018-07-21 21:20:44.839491 7f3d23173d80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
2018-07-21 21:20:44.839492 7f3d23173d80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
2018-07-21 21:20:44.839493 7f3d23173d80 4 rocksdb: Options.level0_stop_writes_trigger: 36
2018-07-21 21:20:44.839494 7f3d23173d80 4 rocksdb: Options.target_file_size_base: 67108864
2018-07-21 21:20:44.839496 7f3d23173d80 4 rocksdb: Options.target_file_size_multiplier: 1
2018-07-21 21:20:44.839497 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_base: 268435456
2018-07-21 21:20:44.839498 7f3d23173d80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
2018-07-21 21:20:44.839500 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
2018-07-21 21:20:44.839503 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
2018-07-21 21:20:44.839505 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
2018-07-21 21:20:44.839507 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
2018-07-21 21:20:44.839508 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
2018-07-21 21:20:44.839509 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
2018-07-21 21:20:44.839511 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
2018-07-21 21:20:44.839512 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
2018-07-21 21:20:44.839513 7f3d23173d80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
2018-07-21 21:20:44.839515 7f3d23173d80 4 rocksdb: Options.max_compaction_bytes: 1677721600
2018-07-21 21:20:44.839517 7f3d23173d80 4 rocksdb: Options.arena_block_size: 33554432
2018-07-21 21:20:44.839518 7f3d23173d80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
2018-07-21 21:20:44.839519 7f3d23173d80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
2018-07-21 21:20:44.839521 7f3d23173d80 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
2018-07-21 21:20:44.839522 7f3d23173d80 4 rocksdb: Options.disable_auto_compactions: 0
2018-07-21 21:20:44.839524 7f3d23173d80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
2018-07-21 21:20:44.839526 7f3d23173d80 4 rocksdb: Options.compaction_pri: kByCompensatedSize
2018-07-21 21:20:44.839527 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
2018-07-21 21:20:44.839529 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
2018-07-21 21:20:44.839530 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
2018-07-21 21:20:44.839532 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
2018-07-21 21:20:44.839533 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
2018-07-21 21:20:44.839535 7f3d23173d80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
2018-07-21 21:20:44.839536 7f3d23173d80 4 rocksdb: Options.table_properties_collectors:
2018-07-21 21:20:44.839537 7f3d23173d80 4 rocksdb: Options.inplace_update_support: 0
2018-07-21 21:20:44.839539 7f3d23173d80 4 rocksdb: Options.inplace_update_num_locks: 10000
2018-07-21 21:20:44.839540 7f3d23173d80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
2018-07-21 21:20:44.839542 7f3d23173d80 4 rocksdb: Options.memtable_huge_page_size: 0
2018-07-21 21:20:44.839544 7f3d23173d80 4 rocksdb: Options.bloom_locality: 0
2018-07-21 21:20:44.839545 7f3d23173d80 4 rocksdb: Options.max_successive_merges: 0
2018-07-21 21:20:44.839546 7f3d23173d80 4 rocksdb: Options.optimize_filters_for_hits: 0
2018-07-21 21:20:44.839547 7f3d23173d80 4 rocksdb: Options.paranoid_file_checks: 0
2018-07-21 21:20:44.839548 7f3d23173d80 4 rocksdb: Options.force_consistency_checks: 0
2018-07-21 21:20:44.839549 7f3d23173d80 4 rocksdb: Options.report_bg_io_stats: 0
2018-07-21 21:20:44.841488 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/version_set.cc:2859] Recovered from manifest file:db/MANIFEST-000157 succeeded,manifest_file_number is 157, next_file_number is 159, last_sequence is 253029, log_number is 0,prev_log_number is 0,max_column_family is 0

2018-07-21 21:20:44.841501 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/version_set.cc:2867] Column family [default] (ID 0), log number is 156

2018-07-21 21:20:44.841556 7f3d23173d80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1532200844841547, "job": 1, "event": "recovery_started", "log_files": [158]}
2018-07-21 21:20:44.841563 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_open.cc:482] Recovering log #158 mode 0
2018-07-21 21:20:44.841577 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/version_set.cc:2395] Creating manifest 160

2018-07-21 21:20:44.842391 7f3d23173d80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1532200844842388, "job": 1, "event": "recovery_finished"}
2018-07-21 21:20:44.845424 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_open.cc:1063] DB pointer 0x55ad4e2c2b00
2018-07-21 21:20:44.845488 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
2018-07-21 21:20:44.845760 7f3d0e782700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl.cc:447] ------- DUMPING STATS -------
2018-07-21 21:20:44.845790 7f3d0e782700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl.cc:448]
** DB Stats **
Uptime(secs): 0.0 total, 0.0 interval
Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s
Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s
Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s
Interval stall: 00:00:0.000 H:M:S, 0.0 percent

** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
----------------------------------------------------------------------------------------------------------------------------------------------------------
L0 6/0 4.77 MB 1.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Sum 6/0 4.77 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Uptime(secs): 0.0 total, 0.0 interval
Flush(GB): cumulative 0.000, interval 0.000
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** File Read Latency Histogram By Level [default] **

** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
----------------------------------------------------------------------------------------------------------------------------------------------------------
L0 6/0 4.77 MB 1.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Sum 6/0 4.77 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Uptime(secs): 0.0 total, 0.0 interval
Flush(GB): cumulative 0.000, interval 0.000
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** File Read Latency Histogram By Level [default] **

2018-07-21 21:20:44.845926 7f3d0e782700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1403] [default] [JOB 3] Compacting 6@0 files to L1, score 1.50
2018-07-21 21:20:44.845954 7f3d0e782700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1407] [default] Compaction start summary: Base version 2 Base level 0, inputs: [24(1406B) 21(14KB) 19(4561KB) 13(308KB) 10(1391B) 4(1456B)]

2018-07-21 21:20:44.846000 7f3d0e782700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1532200844845965, "job": 3, "event": "compaction_started", "files_L0": [24, 21, 19, 13, 10, 4], "score": 1.5, "input_data_size": 5006316}
2018-07-21 21:20:44.846921 7f3d23173d80 1 freelist init
2018-07-21 21:20:44.847517 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _open_alloc opening allocation metadata
2018-07-21 21:20:44.847652 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _open_alloc loaded 1787 G in 1 extents
2018-07-21 21:20:44.854250 7f3d23173d80 0 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
2018-07-21 21:20:44.854289 7f3d23173d80 0 _get_class not permitted to load lua
2018-07-21 21:20:44.854303 7f3d23173d80 0 _get_class not permitted to load kvs
2018-07-21 21:20:44.859996 7f3d23173d80 0 _get_class not permitted to load sdk
2018-07-21 21:20:44.860460 7f3d23173d80 0 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/hello/cls_hello.cc:296: loading cls_hello
2018-07-21 21:20:44.864106 7f3d0e782700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_compaction_flush.cc:1591] Compaction error: Corruption: block checksum mismatch
2018-07-21 21:20:44.864130 7f3d0e782700 4 rocksdb: (Original Log Time 2018/07/21-21:20:44.864026) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:621] [default] compacted to: base level 1 max bytes base 268435456 files[6 0 0 0 0 0 0] max score 0.00, MB/sec: 278.4 rd, 20.3 wr, level 1, files in(6, 0) out(1) MB in(4.8, 0.0) out(0.3), read-write-amplify(1.1) write-amplify(0.1) Corruption: block checksum mismatch, records in: 4782
2018-07-21 21:20:44.864140 7f3d0e782700 4 rocksdb: (Original Log Time 2018/07/21-21:20:44.864089) EVENT_LOG_v1 {"time_micros": 1532200844864063, "job": 3, "event": "compaction_finished", "compaction_time_micros": 17983, "output_level": 1, "num_output_files": 1, "total_output_size": 365633, "num_input_records": 3539, "num_output_records": 1817, "num_subcompactions": 1, "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [6, 0, 0, 0, 0, 0, 0]}
2018-07-21 21:20:44.864146 7f3d0e782700 2 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_compaction_flush.cc:1275] Waiting after background compaction error: Corruption: block checksum mismatch, Accumulated background error counts: 1
2018-07-21 21:20:44.865208 7f3d23173d80 0 osd.24 763 crush map has features 288514051259236352, adjusting msgr requires for clients
2018-07-21 21:20:44.865220 7f3d23173d80 0 osd.24 763 crush map has features 288514051259236352 was 8705, adjusting msgr requires for mons
2018-07-21 21:20:44.865226 7f3d23173d80 0 osd.24 763 crush map has features 1009089991638532096, adjusting msgr requires for osds
2018-07-21 21:20:44.873029 7f3d23173d80 0 osd.24 763 load_pgs
2018-07-21 21:20:44.907849 7f3d0ff85700 -1 rocksdb: submit_transaction error: Corruption: block checksum mismatch code = 2 Rocksdb transaction:
Put( Prefix = O key = 0x7f80000000000000097000000021213dfffffffffffffffeffffffffffffffff'o' Value size = 29)
Put( Prefix = S key = 'nid_max' Value size = 8)
Put( Prefix = S key = 'blobid_max' Value size = 8)
2018-07-21 21:20:44.911270 7f3d0ff85700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f3d0ff85700 time 2018-07-21 21:20:44.907898
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/os/bluestore/BlueStore.cc: 8526: FAILED assert(r == 0)

ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x55ad44902850]
2: (BlueStore::_kv_sync_thread()+0x3472) [0x55ad447b1042]
3: (BlueStore::KVSyncThread::entry()+0xd) [0x55ad447f792d]
4: (()+0x7e25) [0x7f3d20845e25]
5: (clone()+0x6d) [0x7f3d1f936bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-275> 2018-07-21 21:20:44.503577 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command perfcounters_dump hook 0x55ad4e020190
-274> 2018-07-21 21:20:44.503600 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command 1 hook 0x55ad4e020190
-273> 2018-07-21 21:20:44.503609 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command perf dump hook 0x55ad4e020190
-272> 2018-07-21 21:20:44.503613 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command perfcounters_schema hook 0x55ad4e020190
-271> 2018-07-21 21:20:44.503619 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command perf histogram dump hook 0x55ad4e020190
-270> 2018-07-21 21:20:44.503624 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command 2 hook 0x55ad4e020190
-269> 2018-07-21 21:20:44.503628 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command perf schema hook 0x55ad4e020190
-268> 2018-07-21 21:20:44.503632 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command perf histogram schema hook 0x55ad4e020190
-267> 2018-07-21 21:20:44.503643 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command perf reset hook 0x55ad4e020190
-266> 2018-07-21 21:20:44.503648 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command config show hook 0x55ad4e020190
-265> 2018-07-21 21:20:44.503654 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command config help hook 0x55ad4e020190
-264> 2018-07-21 21:20:44.503658 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command config set hook 0x55ad4e020190
-263> 2018-07-21 21:20:44.503663 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command config get hook 0x55ad4e020190
-262> 2018-07-21 21:20:44.503668 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command config diff hook 0x55ad4e020190
-261> 2018-07-21 21:20:44.503672 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command config diff get hook 0x55ad4e020190
-260> 2018-07-21 21:20:44.503677 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command log flush hook 0x55ad4e020190
-259> 2018-07-21 21:20:44.503683 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command log dump hook 0x55ad4e020190
-258> 2018-07-21 21:20:44.503686 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command log reopen hook 0x55ad4e020190
-257> 2018-07-21 21:20:44.503699 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command dump_mempools hook 0x55ad4e217228
-256> 2018-07-21 21:20:44.512118 7f3d23173d80 0 set uid:gid to 167:167 (ceph:ceph)
-255> 2018-07-21 21:20:44.512138 7f3d23173d80 0 ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable), process ceph-osd, pid 14561
-254> 2018-07-21 21:20:44.512180 7f3d23173d80 5 object store type is bluestore
-253> 2018-07-21 21:20:44.517652 7f3d1df64700 2 Event(0x55ad4e2b6480 nevent=5000 time_id=1).set_owner idx=0 owner=139900472411904
-252> 2018-07-21 21:20:44.517710 7f3d1d763700 2 Event(0x55ad4e2b6880 nevent=5000 time_id=1).set_owner idx=1 owner=139900464019200
-251> 2018-07-21 21:20:44.517742 7f3d1cf62700 2 Event(0x55ad4e2b6c80 nevent=5000 time_id=1).set_owner idx=2 owner=139900455626496
-250> 2018-07-21 21:20:44.518865 7f3d23173d80 1 -- 100.64.252.8:0/0 learned_addr learned my addr 100.64.252.8:0/0
-249> 2018-07-21 21:20:44.518901 7f3d23173d80 1 -- 100.64.252.8:6815/14561 _finish_bind bind my_inst.addr is 100.64.252.8:6815/14561
-248> 2018-07-21 21:20:44.519641 7f3d23173d80 1 -- 100.64.252.8:0/0 learned_addr learned my addr 100.64.252.8:0/0
-247> 2018-07-21 21:20:44.519650 7f3d23173d80 1 -- 100.64.252.8:6831/14561 _finish_bind bind my_inst.addr is 100.64.252.8:6831/14561
-246> 2018-07-21 21:20:44.520632 7f3d23173d80 1 -- 100.64.252.8:0/0 learned_addr learned my addr 100.64.252.8:0/0
-245> 2018-07-21 21:20:44.520640 7f3d23173d80 1 -- 100.64.252.8:6843/14561 _finish_bind bind my_inst.addr is 100.64.252.8:6843/14561
-244> 2018-07-21 21:20:44.521781 7f3d23173d80 1 -- 100.64.252.8:0/0 learned_addr learned my addr 100.64.252.8:0/0
-243> 2018-07-21 21:20:44.521788 7f3d23173d80 1 -- 100.64.252.8:6851/14561 _finish_bind bind my_inst.addr is 100.64.252.8:6851/14561
-242> 2018-07-21 21:20:44.521794 7f3d23173d80 0 pidfile_write: ignore empty --pid-file
-241> 2018-07-21 21:20:44.554613 7f3d23173d80 5 asok(0x55ad4e0581c0) init /var/run/ceph/ceph-osd.24.asok
-240> 2018-07-21 21:20:44.554640 7f3d23173d80 5 asok(0x55ad4e0581c0) bind_and_listen /var/run/ceph/ceph-osd.24.asok
-239> 2018-07-21 21:20:44.554719 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command 0 hook 0x55ad4e01e1d8
-238> 2018-07-21 21:20:44.554727 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command version hook 0x55ad4e01e1d8
-237> 2018-07-21 21:20:44.554731 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command git_version hook 0x55ad4e01e1d8
-236> 2018-07-21 21:20:44.554734 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command help hook 0x55ad4e020590
-235> 2018-07-21 21:20:44.554736 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command get_command_descriptions hook 0x55ad4e0205a0
-234> 2018-07-21 21:20:44.554836 7f3d23173d80 10 monclient: build_initial_monmap
-233> 2018-07-21 21:20:44.554864 7f3d1b7e4700 5 asok(0x55ad4e0581c0) entry start
-232> 2018-07-21 21:20:44.562844 7f3d23173d80 0 load: jerasure load: lrc load: isa
-231> 2018-07-21 21:20:44.562884 7f3d23173d80 5 adding auth protocol: cephx
-230> 2018-07-21 21:20:44.562888 7f3d23173d80 5 adding auth protocol: cephx
-229> 2018-07-21 21:20:44.562930 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block type kernel
-228> 2018-07-21 21:20:44.562944 7f3d23173d80 1 bdev(0x55ad4e392c00 /var/lib/ceph/osd/ceph-24/block) open path /var/lib/ceph/osd/ceph-24/block
-227> 2018-07-21 21:20:44.563244 7f3d23173d80 1 bdev(0x55ad4e392c00 /var/lib/ceph/osd/ceph-24/block) open size 1919711969280 (0x1bef7c00000, 1787 GB) block_size 4096 (4096 B) rotational
-226> 2018-07-21 21:20:44.563493 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _set_cache_sizes cache_size 209715200 meta 0.01 kv 0.99 data 0
-225> 2018-07-21 21:20:44.563520 7f3d23173d80 1 bdev(0x55ad4e392c00 /var/lib/ceph/osd/ceph-24/block) close
-224> 2018-07-21 21:20:44.827449 7f3d23173d80 5 asok(0x55ad4e0581c0) register_command objecter_requests hook 0x55ad4e020620
-223> 2018-07-21 21:20:44.827549 7f3d23173d80 1 -- 100.64.252.8:6815/14561 start start
-222> 2018-07-21 21:20:44.827563 7f3d23173d80 1 -- - start start
-221> 2018-07-21 21:20:44.827565 7f3d23173d80 1 -- - start start
-220> 2018-07-21 21:20:44.827566 7f3d23173d80 1 -- 100.64.252.8:6851/14561 start start
-219> 2018-07-21 21:20:44.827570 7f3d23173d80 1 -- 100.64.252.8:6843/14561 start start
-218> 2018-07-21 21:20:44.827572 7f3d23173d80 1 -- 100.64.252.8:6831/14561 start start
-217> 2018-07-21 21:20:44.827575 7f3d23173d80 1 -- - start start
-216> 2018-07-21 21:20:44.827769 7f3d23173d80 2 osd.24 0 init /var/lib/ceph/osd/ceph-24 (looks like hdd)
-215> 2018-07-21 21:20:44.827779 7f3d23173d80 2 osd.24 0 journal /var/lib/ceph/osd/ceph-24/journal
-214> 2018-07-21 21:20:44.827802 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _mount path /var/lib/ceph/osd/ceph-24
-213> 2018-07-21 21:20:44.828308 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block type kernel
-212> 2018-07-21 21:20:44.828316 7f3d23173d80 1 bdev(0x55ad4e2b7600 /var/lib/ceph/osd/ceph-24/block) open path /var/lib/ceph/osd/ceph-24/block
-211> 2018-07-21 21:20:44.828556 7f3d23173d80 1 bdev(0x55ad4e2b7600 /var/lib/ceph/osd/ceph-24/block) open size 1919711969280 (0x1bef7c00000, 1787 GB) block_size 4096 (4096 B) rotational
-210> 2018-07-21 21:20:44.828790 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _set_cache_sizes cache_size 209715200 meta 0.01 kv 0.99 data 0
-209> 2018-07-21 21:20:44.828870 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block.db type kernel
-208> 2018-07-21 21:20:44.828876 7f3d23173d80 1 bdev(0x55ad4e2b7a00 /var/lib/ceph/osd/ceph-24/block.db) open path /var/lib/ceph/osd/ceph-24/block.db
-207> 2018-07-21 21:20:44.829082 7f3d23173d80 1 bdev(0x55ad4e2b7a00 /var/lib/ceph/osd/ceph-24/block.db) open size 19999490048 (0x4a8100000, 19073 MB) block_size 4096 (4096 B) non-rotational
-206> 2018-07-21 21:20:44.829091 7f3d23173d80 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-24/block.db size 19073 MB
-205> 2018-07-21 21:20:44.829317 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block type kernel
-204> 2018-07-21 21:20:44.829325 7f3d23173d80 1 bdev(0x55ad4e395a00 /var/lib/ceph/osd/ceph-24/block) open path /var/lib/ceph/osd/ceph-24/block
-203> 2018-07-21 21:20:44.829500 7f3d23173d80 1 bdev(0x55ad4e395a00 /var/lib/ceph/osd/ceph-24/block) open size 1919711969280 (0x1bef7c00000, 1787 GB) block_size 4096 (4096 B) rotational
-202> 2018-07-21 21:20:44.829520 7f3d23173d80 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-24/block size 1787 GB
-201> 2018-07-21 21:20:44.829531 7f3d23173d80 1 bdev create path /var/lib/ceph/osd/ceph-24/block.wal type kernel
-200> 2018-07-21 21:20:44.829534 7f3d23173d80 1 bdev(0x55ad4e394e00 /var/lib/ceph/osd/ceph-24/block.wal) open path /var/lib/ceph/osd/ceph-24/block.wal
-199> 2018-07-21 21:20:44.829706 7f3d23173d80 1 bdev(0x55ad4e394e00 /var/lib/ceph/osd/ceph-24/block.wal) open size 1999634432 (0x77300000, 1907 MB) block_size 4096 (4096 B) non-rotational
-198> 2018-07-21 21:20:44.829718 7f3d23173d80 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-24/block.wal size 1907 MB
-197> 2018-07-21 21:20:44.829905 7f3d23173d80 1 bluefs mount
-196> 2018-07-21 21:20:44.838675 7f3d23173d80 0 set rocksdb option compaction_readahead_size = 2097152
-195> 2018-07-21 21:20:44.838692 7f3d23173d80 0 set rocksdb option compression = kNoCompression
-194> 2018-07-21 21:20:44.838698 7f3d23173d80 0 set rocksdb option max_write_buffer_number = 4
-193> 2018-07-21 21:20:44.838702 7f3d23173d80 0 set rocksdb option min_write_buffer_number_to_merge = 1
-192> 2018-07-21 21:20:44.838706 7f3d23173d80 0 set rocksdb option recycle_log_file_num = 4
-191> 2018-07-21 21:20:44.838709 7f3d23173d80 0 set rocksdb option writable_file_max_buffer_size = 0
-190> 2018-07-21 21:20:44.838713 7f3d23173d80 0 set rocksdb option write_buffer_size = 268435456
-189> 2018-07-21 21:20:44.838742 7f3d23173d80 0 set rocksdb option compaction_readahead_size = 2097152
-188> 2018-07-21 21:20:44.838750 7f3d23173d80 0 set rocksdb option compression = kNoCompression
-187> 2018-07-21 21:20:44.838754 7f3d23173d80 0 set rocksdb option max_write_buffer_number = 4
-186> 2018-07-21 21:20:44.838758 7f3d23173d80 0 set rocksdb option min_write_buffer_number_to_merge = 1
-185> 2018-07-21 21:20:44.838761 7f3d23173d80 0 set rocksdb option recycle_log_file_num = 4
-184> 2018-07-21 21:20:44.838764 7f3d23173d80 0 set rocksdb option writable_file_max_buffer_size = 0
-183> 2018-07-21 21:20:44.838767 7f3d23173d80 0 set rocksdb option write_buffer_size = 268435456
-182> 2018-07-21 21:20:44.838949 7f3d23173d80 4 rocksdb: RocksDB version: 5.4.0

-181> 2018-07-21 21:20:44.838966 7f3d23173d80 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
-180> 2018-07-21 21:20:44.838968 7f3d23173d80 4 rocksdb: Compile date Jul 16 2018
-179> 2018-07-21 21:20:44.838979 7f3d23173d80 4 rocksdb: DB SUMMARY

-178> 2018-07-21 21:20:44.839008 7f3d23173d80 4 rocksdb: CURRENT file: CURRENT

-177> 2018-07-21 21:20:44.839012 7f3d23173d80 4 rocksdb: IDENTITY file: IDENTITY

-176> 2018-07-21 21:20:44.839018 7f3d23173d80 4 rocksdb: MANIFEST file: MANIFEST-000157 size: 350 Bytes

-175> 2018-07-21 21:20:44.839021 7f3d23173d80 4 rocksdb: SST files in db dir, Total Num: 6, files: 000004.sst 000010.sst 000013.sst 000019.sst 000021.sst 000024.sst

-174> 2018-07-21 21:20:44.839025 7f3d23173d80 4 rocksdb: SST files in db.slow dir, Total Num: 0, files:

-173> 2018-07-21 21:20:44.839030 7f3d23173d80 4 rocksdb: Write Ahead Log file in db.wal: 000158.log size: 0 ;

-172> 2018-07-21 21:20:44.839033 7f3d23173d80 4 rocksdb: Options.error_if_exists: 0
-171> 2018-07-21 21:20:44.839041 7f3d23173d80 4 rocksdb: Options.create_if_missing: 0
-170> 2018-07-21 21:20:44.839043 7f3d23173d80 4 rocksdb: Options.paranoid_checks: 1
-169> 2018-07-21 21:20:44.839044 7f3d23173d80 4 rocksdb: Options.env: 0x55ad4e38d0a0
-168> 2018-07-21 21:20:44.839045 7f3d23173d80 4 rocksdb: Options.info_log: 0x55ad4e38d520
-167> 2018-07-21 21:20:44.839046 7f3d23173d80 4 rocksdb: Options.max_open_files: -1
-166> 2018-07-21 21:20:44.839047 7f3d23173d80 4 rocksdb: Options.max_file_opening_threads: 16
-165> 2018-07-21 21:20:44.839048 7f3d23173d80 4 rocksdb: Options.use_fsync: 0
-164> 2018-07-21 21:20:44.839049 7f3d23173d80 4 rocksdb: Options.max_log_file_size: 0
-163> 2018-07-21 21:20:44.839050 7f3d23173d80 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
-162> 2018-07-21 21:20:44.839051 7f3d23173d80 4 rocksdb: Options.log_file_time_to_roll: 0
-161> 2018-07-21 21:20:44.839052 7f3d23173d80 4 rocksdb: Options.keep_log_file_num: 1000
-160> 2018-07-21 21:20:44.839053 7f3d23173d80 4 rocksdb: Options.recycle_log_file_num: 4
-159> 2018-07-21 21:20:44.839054 7f3d23173d80 4 rocksdb: Options.allow_fallocate: 1
-158> 2018-07-21 21:20:44.839054 7f3d23173d80 4 rocksdb: Options.allow_mmap_reads: 0
-157> 2018-07-21 21:20:44.839055 7f3d23173d80 4 rocksdb: Options.allow_mmap_writes: 0
-156> 2018-07-21 21:20:44.839056 7f3d23173d80 4 rocksdb: Options.use_direct_reads: 0
-155> 2018-07-21 21:20:44.839057 7f3d23173d80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
-154> 2018-07-21 21:20:44.839058 7f3d23173d80 4 rocksdb: Options.create_missing_column_families: 0
-153> 2018-07-21 21:20:44.839058 7f3d23173d80 4 rocksdb: Options.db_log_dir:
-152> 2018-07-21 21:20:44.839059 7f3d23173d80 4 rocksdb: Options.wal_dir: db.wal
-151> 2018-07-21 21:20:44.839060 7f3d23173d80 4 rocksdb: Options.table_cache_numshardbits: 6
-150> 2018-07-21 21:20:44.839061 7f3d23173d80 4 rocksdb: Options.max_subcompactions: 1
-149> 2018-07-21 21:20:44.839062 7f3d23173d80 4 rocksdb: Options.max_background_flushes: 1
-148> 2018-07-21 21:20:44.839062 7f3d23173d80 4 rocksdb: Options.WAL_ttl_seconds: 0
-147> 2018-07-21 21:20:44.839063 7f3d23173d80 4 rocksdb: Options.WAL_size_limit_MB: 0
-146> 2018-07-21 21:20:44.839064 7f3d23173d80 4 rocksdb: Options.manifest_preallocation_size: 4194304
-145> 2018-07-21 21:20:44.839065 7f3d23173d80 4 rocksdb: Options.is_fd_close_on_exec: 1
-144> 2018-07-21 21:20:44.839066 7f3d23173d80 4 rocksdb: Options.advise_random_on_open: 1
-143> 2018-07-21 21:20:44.839066 7f3d23173d80 4 rocksdb: Options.db_write_buffer_size: 0
-142> 2018-07-21 21:20:44.839067 7f3d23173d80 4 rocksdb: Options.access_hint_on_compaction_start: 1
-141> 2018-07-21 21:20:44.839068 7f3d23173d80 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
-140> 2018-07-21 21:20:44.839069 7f3d23173d80 4 rocksdb: Options.compaction_readahead_size: 2097152
-139> 2018-07-21 21:20:44.839069 7f3d23173d80 4 rocksdb: Options.random_access_max_buffer_size: 1048576
-138> 2018-07-21 21:20:44.839070 7f3d23173d80 4 rocksdb: Options.writable_file_max_buffer_size: 0
-137> 2018-07-21 21:20:44.839071 7f3d23173d80 4 rocksdb: Options.use_adaptive_mutex: 0
-136> 2018-07-21 21:20:44.839072 7f3d23173d80 4 rocksdb: Options.rate_limiter: (nil)
-135> 2018-07-21 21:20:44.839077 7f3d23173d80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
-134> 2018-07-21 21:20:44.839079 7f3d23173d80 4 rocksdb: Options.bytes_per_sync: 0
-133> 2018-07-21 21:20:44.839079 7f3d23173d80 4 rocksdb: Options.wal_bytes_per_sync: 0
-132> 2018-07-21 21:20:44.839080 7f3d23173d80 4 rocksdb: Options.wal_recovery_mode: 0
-131> 2018-07-21 21:20:44.839082 7f3d23173d80 4 rocksdb: Options.enable_thread_tracking: 0
-130> 2018-07-21 21:20:44.839082 7f3d23173d80 4 rocksdb: Options.allow_concurrent_memtable_write: 1
-129> 2018-07-21 21:20:44.839083 7f3d23173d80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
-128> 2018-07-21 21:20:44.839084 7f3d23173d80 4 rocksdb: Options.write_thread_max_yield_usec: 100
-127> 2018-07-21 21:20:44.839085 7f3d23173d80 4 rocksdb: Options.write_thread_slow_yield_usec: 3
-126> 2018-07-21 21:20:44.839085 7f3d23173d80 4 rocksdb: Options.row_cache: None
-125> 2018-07-21 21:20:44.839086 7f3d23173d80 4 rocksdb: Options.wal_filter: None
-124> 2018-07-21 21:20:44.839087 7f3d23173d80 4 rocksdb: Options.avoid_flush_during_recovery: 0
-123> 2018-07-21 21:20:44.839088 7f3d23173d80 4 rocksdb: Options.base_background_compactions: 1
-122> 2018-07-21 21:20:44.839089 7f3d23173d80 4 rocksdb: Options.max_background_compactions: 1
-121> 2018-07-21 21:20:44.839089 7f3d23173d80 4 rocksdb: Options.avoid_flush_during_shutdown: 0
-120> 2018-07-21 21:20:44.839090 7f3d23173d80 4 rocksdb: Options.delayed_write_rate : 16777216
-119> 2018-07-21 21:20:44.839091 7f3d23173d80 4 rocksdb: Options.max_total_wal_size: 0
-118> 2018-07-21 21:20:44.839092 7f3d23173d80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
-117> 2018-07-21 21:20:44.839093 7f3d23173d80 4 rocksdb: Options.stats_dump_period_sec: 600
-116> 2018-07-21 21:20:44.839094 7f3d23173d80 4 rocksdb: Compression algorithms supported:
-115> 2018-07-21 21:20:44.839095 7f3d23173d80 4 rocksdb: Snappy supported: 1
-114> 2018-07-21 21:20:44.839096 7f3d23173d80 4 rocksdb: Zlib supported: 0
-113> 2018-07-21 21:20:44.839096 7f3d23173d80 4 rocksdb: Bzip supported: 0
-112> 2018-07-21 21:20:44.839097 7f3d23173d80 4 rocksdb: LZ4 supported: 0
-111> 2018-07-21 21:20:44.839098 7f3d23173d80 4 rocksdb: ZSTD supported: 0
-110> 2018-07-21 21:20:44.839099 7f3d23173d80 4 rocksdb: Fast CRC32 supported: 1
-109> 2018-07-21 21:20:44.839340 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/version_set.cc:2609] Recovering from manifest file: MANIFEST-000157

-108> 2018-07-21 21:20:44.839413 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/column_family.cc:407] --------------- Options for column family [default]:

-107> 2018-07-21 21:20:44.839423 7f3d23173d80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
-106> 2018-07-21 21:20:44.839431 7f3d23173d80 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
-105> 2018-07-21 21:20:44.839433 7f3d23173d80 4 rocksdb: Options.compaction_filter: None
-104> 2018-07-21 21:20:44.839435 7f3d23173d80 4 rocksdb: Options.compaction_filter_factory: None
-103> 2018-07-21 21:20:44.839436 7f3d23173d80 4 rocksdb: Options.memtable_factory: SkipListFactory
-102> 2018-07-21 21:20:44.839439 7f3d23173d80 4 rocksdb: Options.table_factory: BlockBasedTable
-101> 2018-07-21 21:20:44.839460 7f3d23173d80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ad4e01e228)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 1
pin_l0_filter_and_index_blocks_in_cache: 1
index_type: 0
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x55ad4e4900f8
block_cache_name: LRUCache
block_cache_options:
capacity : 207618048
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
format_version: 2

-100> 2018-07-21 21:20:44.839465 7f3d23173d80 4 rocksdb: Options.write_buffer_size: 268435456
-99> 2018-07-21 21:20:44.839471 7f3d23173d80 4 rocksdb: Options.max_write_buffer_number: 4
-98> 2018-07-21 21:20:44.839473 7f3d23173d80 4 rocksdb: Options.compression: NoCompression
-97> 2018-07-21 21:20:44.839476 7f3d23173d80 4 rocksdb: Options.bottommost_compression: Disabled
-96> 2018-07-21 21:20:44.839478 7f3d23173d80 4 rocksdb: Options.prefix_extractor: nullptr
-95> 2018-07-21 21:20:44.839479 7f3d23173d80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
-94> 2018-07-21 21:20:44.839481 7f3d23173d80 4 rocksdb: Options.num_levels: 7
-93> 2018-07-21 21:20:44.839482 7f3d23173d80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
-92> 2018-07-21 21:20:44.839483 7f3d23173d80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
-91> 2018-07-21 21:20:44.839485 7f3d23173d80 4 rocksdb: Options.compression_opts.window_bits: -14
-90> 2018-07-21 21:20:44.839486 7f3d23173d80 4 rocksdb: Options.compression_opts.level: -1
-89> 2018-07-21 21:20:44.839488 7f3d23173d80 4 rocksdb: Options.compression_opts.strategy: 0
-88> 2018-07-21 21:20:44.839489 7f3d23173d80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
-87> 2018-07-21 21:20:44.839491 7f3d23173d80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
-86> 2018-07-21 21:20:44.839492 7f3d23173d80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
-85> 2018-07-21 21:20:44.839493 7f3d23173d80 4 rocksdb: Options.level0_stop_writes_trigger: 36
-84> 2018-07-21 21:20:44.839494 7f3d23173d80 4 rocksdb: Options.target_file_size_base: 67108864
-83> 2018-07-21 21:20:44.839496 7f3d23173d80 4 rocksdb: Options.target_file_size_multiplier: 1
-82> 2018-07-21 21:20:44.839497 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_base: 268435456
-81> 2018-07-21 21:20:44.839498 7f3d23173d80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
-80> 2018-07-21 21:20:44.839500 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
-79> 2018-07-21 21:20:44.839503 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
-78> 2018-07-21 21:20:44.839505 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
-77> 2018-07-21 21:20:44.839507 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
-76> 2018-07-21 21:20:44.839508 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
-75> 2018-07-21 21:20:44.839509 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
-74> 2018-07-21 21:20:44.839511 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
-73> 2018-07-21 21:20:44.839512 7f3d23173d80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
-72> 2018-07-21 21:20:44.839513 7f3d23173d80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
-71> 2018-07-21 21:20:44.839515 7f3d23173d80 4 rocksdb: Options.max_compaction_bytes: 1677721600
-70> 2018-07-21 21:20:44.839517 7f3d23173d80 4 rocksdb: Options.arena_block_size: 33554432
-69> 2018-07-21 21:20:44.839518 7f3d23173d80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
-68> 2018-07-21 21:20:44.839519 7f3d23173d80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
-67> 2018-07-21 21:20:44.839521 7f3d23173d80 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
-66> 2018-07-21 21:20:44.839522 7f3d23173d80 4 rocksdb: Options.disable_auto_compactions: 0
-65> 2018-07-21 21:20:44.839524 7f3d23173d80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
-64> 2018-07-21 21:20:44.839526 7f3d23173d80 4 rocksdb: Options.compaction_pri: kByCompensatedSize
-63> 2018-07-21 21:20:44.839527 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
-62> 2018-07-21 21:20:44.839529 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
-61> 2018-07-21 21:20:44.839530 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
-60> 2018-07-21 21:20:44.839532 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
-59> 2018-07-21 21:20:44.839533 7f3d23173d80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
-58> 2018-07-21 21:20:44.839535 7f3d23173d80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
-57> 2018-07-21 21:20:44.839536 7f3d23173d80 4 rocksdb: Options.table_properties_collectors:
-56> 2018-07-21 21:20:44.839537 7f3d23173d80 4 rocksdb: Options.inplace_update_support: 0
-55> 2018-07-21 21:20:44.839539 7f3d23173d80 4 rocksdb: Options.inplace_update_num_locks: 10000
-54> 2018-07-21 21:20:44.839540 7f3d23173d80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
-53> 2018-07-21 21:20:44.839542 7f3d23173d80 4 rocksdb: Options.memtable_huge_page_size: 0
-52> 2018-07-21 21:20:44.839544 7f3d23173d80 4 rocksdb: Options.bloom_locality: 0
-51> 2018-07-21 21:20:44.839545 7f3d23173d80 4 rocksdb: Options.max_successive_merges: 0
-50> 2018-07-21 21:20:44.839546 7f3d23173d80 4 rocksdb: Options.optimize_filters_for_hits: 0
-49> 2018-07-21 21:20:44.839547 7f3d23173d80 4 rocksdb: Options.paranoid_file_checks: 0
-48> 2018-07-21 21:20:44.839548 7f3d23173d80 4 rocksdb: Options.force_consistency_checks: 0
-47> 2018-07-21 21:20:44.839549 7f3d23173d80 4 rocksdb: Options.report_bg_io_stats: 0
-46> 2018-07-21 21:20:44.841488 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/version_set.cc:2859] Recovered from manifest file:db/MANIFEST-000157 succeeded,manifest_file_number is 157, next_file_number is 159, last_sequence is 253029, log_number is 0,prev_log_number is 0,max_column_family is 0

-45> 2018-07-21 21:20:44.841501 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/version_set.cc:2867] Column family [default] (ID 0), log number is 156

-44> 2018-07-21 21:20:44.841556 7f3d23173d80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1532200844841547, "job": 1, "event": "recovery_started", "log_files": [158]}
-43> 2018-07-21 21:20:44.841563 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_open.cc:482] Recovering log #158 mode 0
-42> 2018-07-21 21:20:44.841577 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/version_set.cc:2395] Creating manifest 160

-41> 2018-07-21 21:20:44.842391 7f3d23173d80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1532200844842388, "job": 1, "event": "recovery_finished"}
-40> 2018-07-21 21:20:44.842447 7f3d23173d80 5 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:307] [JOB 2] Delete db.wal/000158.log type=0 #158 -- OK

-39> 2018-07-21 21:20:44.842459 7f3d23173d80 5 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:307] [JOB 2] Delete db//MANIFEST-000157 type=3 #157 -- OK

-38> 2018-07-21 21:20:44.845424 7f3d23173d80 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_open.cc:1063] DB pointer 0x55ad4e2c2b00
-37> 2018-07-21 21:20:44.845488 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
-36> 2018-07-21 21:20:44.845760 7f3d0e782700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl.cc:447] ------- DUMPING STATS -------
-35> 2018-07-21 21:20:44.845790 7f3d0e782700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl.cc:448]
** DB Stats **
Uptime(secs): 0.0 total, 0.0 interval
Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s
Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s
Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s
Interval stall: 00:00:0.000 H:M:S, 0.0 percent

** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
----------------------------------------------------------------------------------------------------------------------------------------------------------
L0 6/0 4.77 MB 1.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Sum 6/0 4.77 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Uptime(secs): 0.0 total, 0.0 interval
Flush(GB): cumulative 0.000, interval 0.000
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** File Read Latency Histogram By Level [default] **

** Compaction Stats [default] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
----------------------------------------------------------------------------------------------------------------------------------------------------------
L0 6/0 4.77 MB 1.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Sum 6/0 4.77 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0
Uptime(secs): 0.0 total, 0.0 interval
Flush(GB): cumulative 0.000, interval 0.000
AddFile(GB): cumulative 0.000, interval 0.000
AddFile(Total Files): cumulative 0, interval 0
AddFile(L0 Files): cumulative 0, interval 0
AddFile(Keys): cumulative 0, interval 0
Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** File Read Latency Histogram By Level [default] **

-34> 2018-07-21 21:20:44.845926 7f3d0e782700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1403] [default] [JOB 3] Compacting 6@0 files to L1, score 1.50
-33> 2018-07-21 21:20:44.845954 7f3d0e782700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1407] [default] Compaction start summary: Base version 2 Base level 0, inputs: [24(1406B) 21(14KB) 19(4561KB) 13(308KB) 10(1391B) 4(1456B)]

-32> 2018-07-21 21:20:44.846000 7f3d0e782700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1532200844845965, "job": 3, "event": "compaction_started", "files_L0": [24, 21, 19, 13, 10, 4], "score": 1.5, "input_data_size": 5006316}
-31> 2018-07-21 21:20:44.846921 7f3d23173d80 1 freelist init
-30> 2018-07-21 21:20:44.847517 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _open_alloc opening allocation metadata
-29> 2018-07-21 21:20:44.847652 7f3d23173d80 1 bluestore(/var/lib/ceph/osd/ceph-24) _open_alloc loaded 1787 G in 1 extents
-28> 2018-07-21 21:20:44.853191 7f3d23173d80 2 osd.24 0 journal looks like ssd
-27> 2018-07-21 21:20:44.853206 7f3d23173d80 2 osd.24 0 boot
-26> 2018-07-21 21:20:44.854250 7f3d23173d80 0 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
-25> 2018-07-21 21:20:44.854289 7f3d23173d80 0 _get_class not permitted to load lua
-24> 2018-07-21 21:20:44.854303 7f3d23173d80 0 _get_class not permitted to load kvs
-23> 2018-07-21 21:20:44.854780 7f3d23173d80 1 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/timeindex/cls_timeindex.cc:242: Loaded timeindex class!
-22> 2018-07-21 21:20:44.855020 7f3d23173d80 1 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/statelog/cls_statelog.cc:279: Loaded log class!
-21> 2018-07-21 21:20:44.855429 7f3d23173d80 1 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/user/cls_user.cc:410: Loaded user class!
-20> 2018-07-21 21:20:44.859325 7f3d23173d80 1 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/rgw/cls_rgw.cc:3831: Loaded rgw class!
-19> 2018-07-21 21:20:44.859963 7f3d23173d80 1 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/log/cls_log.cc:299: Loaded log class!
-18> 2018-07-21 21:20:44.859996 7f3d23173d80 0 _get_class not permitted to load sdk
-17> 2018-07-21 21:20:44.860221 7f3d23173d80 1 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/replica_log/cls_replica_log.cc:135: Loaded replica log class!
-16> 2018-07-21 21:20:44.860460 7f3d23173d80 0 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/hello/cls_hello.cc:296: loading cls_hello
-15> 2018-07-21 21:20:44.864106 7f3d0e782700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_compaction_flush.cc:1591] Compaction error: Corruption: block checksum mismatch
-14> 2018-07-21 21:20:44.864130 7f3d0e782700 4 rocksdb: (Original Log Time 2018/07/21-21:20:44.864026) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:621] [default] compacted to: base level 1 max bytes base 268435456 files[6 0 0 0 0 0 0] max score 0.00, MB/sec: 278.4 rd, 20.3 wr, level 1, files in(6, 0) out(1) MB in(4.8, 0.0) out(0.3), read-write-amplify(1.1) write-amplify(0.1) Corruption: block checksum mismatch, records in: 4782
-13> 2018-07-21 21:20:44.864140 7f3d0e782700 4 rocksdb: (Original Log Time 2018/07/21-21:20:44.864089) EVENT_LOG_v1 {"time_micros": 1532200844864063, "job": 3, "event": "compaction_finished", "compaction_time_micros": 17983, "output_level": 1, "num_output_files": 1, "total_output_size": 365633, "num_input_records": 3539, "num_output_records": 1817, "num_subcompactions": 1, "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [6, 0, 0, 0, 0, 0, 0]}
-12> 2018-07-21 21:20:44.864146 7f3d0e782700 2 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_compaction_flush.cc:1275] Waiting after background compaction error: Corruption: block checksum mismatch, Accumulated background error counts: 1
-11> 2018-07-21 21:20:44.864464 7f3d23173d80 1 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/refcount/cls_refcount.cc:222: Loaded refcount class!
-10> 2018-07-21 21:20:44.864682 7f3d23173d80 1 <cls> /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/cls/version/cls_version.cc:214: Loaded version class!
-9> 2018-07-21 21:20:44.865208 7f3d23173d80 0 osd.24 763 crush map has features 288514051259236352, adjusting msgr requires for clients
-8> 2018-07-21 21:20:44.865220 7f3d23173d80 0 osd.24 763 crush map has features 288514051259236352 was 8705, adjusting msgr requires for mons
-7> 2018-07-21 21:20:44.865226 7f3d23173d80 0 osd.24 763 crush map has features 1009089991638532096, adjusting msgr requires for osds
-6> 2018-07-21 21:20:44.873029 7f3d23173d80 0 osd.24 763 load_pgs
-5> 2018-07-21 21:20:44.873582 7f3d23173d80 5 osd.24 pg_epoch: 763 pg[9.e(unlocked)] enter Initial
-4> 2018-07-21 21:20:44.907499 7f3d23173d80 5 osd.24 pg_epoch: 763 pg[9.e( v 763'1570 (0'0,763'1570] local-lis/les=758/759 n=2 ec=729/729 lis/c 758/758 les/c/f 759/759/0 758/758/755) [11,25,24] r=2 lpr=0 crt=763'1570 lcod 0'0 unknown NOTIFY] exit Initial 0.033931 0 0.000000
-3> 2018-07-21 21:20:44.907562 7f3d23173d80 5 osd.24 pg_epoch: 763 pg[9.e( v 763'1570 (0'0,763'1570] local-lis/les=758/759 n=2 ec=729/729 lis/c 758/758 les/c/f 759/759/0 758/758/755) [11,25,24] r=2 lpr=0 crt=763'1570 lcod 0'0 unknown NOTIFY] enter Reset
-2> 2018-07-21 21:20:44.907595 7f3d23173d80 5 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0
-1> 2018-07-21 21:20:44.907849 7f3d0ff85700 -1 rocksdb: submit_transaction error: Corruption: block checksum mismatch code = 2 Rocksdb transaction:
Put( Prefix = O key = 0x7f80000000000000097000000021213dfffffffffffffffeffffffffffffffff'o' Value size = 29)
Put( Prefix = S key = 'nid_max' Value size = 8)
Put( Prefix = S key = 'blobid_max' Value size = 8)
0> 2018-07-21 21:20:44.911270 7f3d0ff85700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f3d0ff85700 time 2018-07-21 21:20:44.907898
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/os/bluestore/BlueStore.cc: 8526: FAILED assert(r == 0)

ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x55ad44902850]
2: (BlueStore::_kv_sync_thread()+0x3472) [0x55ad447b1042]
3: (BlueStore::KVSyncThread::entry()+0xd) [0x55ad447f792d]
4: (()+0x7e25) [0x7f3d20845e25]
5: (clone()+0x6d) [0x7f3d1f936bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.24.log
--- end dump of recent events ---
2018-07-21 21:20:44.917216 7f3d0ff85700 -1 *** Caught signal (Aborted) **
in thread 7f3d0ff85700 thread_name:bstore_kv_sync

ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable)
1: (()+0xa48ec1) [0x55ad448c3ec1]
2: (()+0xf6d0) [0x7f3d2084d6d0]
3: (gsignal()+0x37) [0x7f3d1f86e277]
4: (abort()+0x148) [0x7f3d1f86f968]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x284) [0x55ad449029c4]
6: (BlueStore::_kv_sync_thread()+0x3472) [0x55ad447b1042]
7: (BlueStore::KVSyncThread::entry()+0xd) [0x55ad447f792d]
8: (()+0x7e25) [0x7f3d20845e25]
9: (clone()+0x6d) [0x7f3d1f936bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
0> 2018-07-21 21:20:44.917216 7f3d0ff85700 -1 *** Caught signal (Aborted) **
in thread 7f3d0ff85700 thread_name:bstore_kv_sync

ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable)
1: (()+0xa48ec1) [0x55ad448c3ec1]
2: (()+0xf6d0) [0x7f3d2084d6d0]
3: (gsignal()+0x37) [0x7f3d1f86e277]
4: (abort()+0x148) [0x7f3d1f86f968]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x284) [0x55ad449029c4]
6: (BlueStore::_kv_sync_thread()+0x3472) [0x55ad447b1042]
7: (BlueStore::KVSyncThread::entry()+0xd) [0x55ad447f792d]
8: (()+0x7e25) [0x7f3d20845e25]
9: (clone()+0x6d) [0x7f3d1f936bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.24.log
--- end dump of recent events ---
(5-5/5)