Project

General

Profile

Bug #53466 ยป osd-2.log

Jan Tilsner, 12/02/2021 03:10 PM

 
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 set uid:gid to 167:167 (ceph:ceph)
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable), process ceph-osd, pid 1
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 pidfile_write: ignore empty --pid-file
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:06.608+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:06.840+0000 7f86cebf1080 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
debug 2021-11-15T12:49:06.840+0000 7f86cebf1080 -1 Falling back to public interface
debug 2021-11-15T12:49:06.852+0000 7f86cebf1080 0 load: jerasure load: lrc load: isa
debug 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:07.452+0000 7f86cebf1080 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:07.752+0000 7f86cebf1080 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:08.048+0000 7f86cebf1080 0 osd.2:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:08.344+0000 7f86cebf1080 0 osd.2:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:08.628+0000 7f86cebf1080 0 osd.2:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:08.920+0000 7f86cebf1080 0 osd.2:5.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:09.216+0000 7f86cebf1080 0 osd.2:6.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:09.512+0000 7f86cebf1080 0 osd.2:7.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _mount path /var/lib/ceph/osd/ceph-2
debug 2021-11-15T12:49:09.512+0000 7f86cebf1080 0 bluestore(/var/lib/ceph/osd/ceph-2) _open_db_and_around read-only:0 repair:0
debug 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs mount
debug 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs _init_alloc shared, id 1, capacity 0x31ffc00000, block size 0x10000
debug 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 bluefs mount shared_bdev_used = 0
debug 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _prepare_db_environment set db_paths to db,204006961971 db.slow,204006961971
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: RocksDB version: 6.8.1

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Compile date Sep 16 2021
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: DB SUMMARY

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: CURRENT file: CURRENT

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: IDENTITY file: IDENTITY

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: MANIFEST file: MANIFEST-031387 size: 238322 Bytes

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: SST files in db dir, Total Num: 11, files: 033348.sst 033428.sst 033429.sst 033430.sst 033431.sst 033432.sst 033433.sst 033434.sst 033443.sst

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: SST files in db.slow dir, Total Num: 0, files:

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Write Ahead Log file in db.wal: 033446.log size: 221571739 ;

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.error_if_exists: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.create_if_missing: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.paranoid_checks: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.env: 0x561fe20cb080
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.fs: Legacy File System
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.info_log: 0x561fe20cbb20
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_file_opening_threads: 16
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.statistics: (nil)
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_fsync: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_log_file_size: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_manifest_file_size: 1073741824
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.log_file_time_to_roll: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.keep_log_file_num: 1000
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.recycle_log_file_num: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_fallocate: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_reads: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_writes: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_direct_reads: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.create_missing_column_families: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.db_log_dir:
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_dir: db.wal
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_cache_numshardbits: 6
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_subcompactions: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_flushes: -1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.WAL_ttl_seconds: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.WAL_size_limit_MB: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.manifest_preallocation_size: 4194304
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.is_fd_close_on_exec: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.advise_random_on_open: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.db_write_buffer_size: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_manager: 0x561fe2ea6ab0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.access_hint_on_compaction_start: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.random_access_max_buffer_size: 1048576
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_adaptive_mutex: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.rate_limiter: (nil)
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_recovery_mode: 2
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_thread_tracking: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_pipelined_write: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.unordered_write: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_concurrent_memtable_write: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_thread_max_yield_usec: 100
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_thread_slow_yield_usec: 3
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.row_cache: None
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_filter: None
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_recovery: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_ingest_behind: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.preserve_deletes: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.two_write_queues: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.manual_wal_flush: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.atomic_flush: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.persist_stats_to_disk: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_dbid_to_manifest: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.log_readahead_size: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.sst_file_checksum_func: Unknown
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_jobs: 2
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_compactions: 2
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_shutdown: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.writable_file_max_buffer_size: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.delayed_write_rate : 16777216
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_total_wal_size: 1073741824
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_dump_period_sec: 600
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_persist_period_sec: 600
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_history_buffer_size: 1048576
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_open_files: -1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bytes_per_sync: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_bytes_per_sync: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.strict_bytes_per_sync: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_readahead_size: 2097152
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Compression algorithms supported:
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZSTDNotFinalCompression supported: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZSTD supported: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kXpressCompression supported: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kLZ4HCCompression supported: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kLZ4Compression supported: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kBZip2Compression supported: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZlibCompression supported: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kSnappyCompression supported: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Fast CRC32 supported: Supported on x86
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_readonly.cc:23] Opening the db in read only mode
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: db/MANIFEST-031387

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]:

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter: None
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter_factory: None
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_factory: SkipListFactory
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_factory: BlockBasedTable
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561fe205e7b0)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 0
pin_l0_filter_and_index_blocks_in_cache: 0
pin_top_level_index_and_filter: 1
index_type: 0
data_block_index_type: 0
index_shortening: 1
data_block_hash_table_util_ratio: 0.750000
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x561fe20a7210
block_cache_name: BinnedLRUCache
block_cache_options:
capacity : 1449551462
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
metadata_block_size: 4096
partition_filters: 0
use_delta_encoding: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
verify_compression: 0
read_amp_bytes_per_bit: 0
format_version: 2
enable_index_compression: 1
block_align: 0

debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_size: 268435456
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number: 4
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression: NoCompression
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression: Disabled
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.prefix_extractor: nullptr
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.num_levels: 7
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.level: 32767
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.enabled: false
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.window_bits: -14
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.level: 32767
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.strategy: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.enabled: false
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_stop_writes_trigger: 36
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_base: 67108864
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_multiplier: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_base: 268435456
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_compaction_bytes: 1677721600
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.arena_block_size: 33554432
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.disable_auto_compactions: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_properties_collectors:
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_support: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_num_locks: 10000
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_whole_key_filtering: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_huge_page_size: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bloom_locality: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_successive_merges: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.optimize_filters_for_hits: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.paranoid_file_checks: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.force_consistency_checks: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.report_bg_io_stats: 0
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.ttl: 2592000
debug 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.periodic_compaction_seconds: 0
debug 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:db/MANIFEST-031387 succeeded,manifest_file_number is 31387, next_file_number is 33449, last_sequence is 9496239791, log_number is 33446,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0

debug 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 33446

debug 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980549639149, "job": 1, "event": "recovery_started", "log_files": [33446]}
debug 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #33446 mode 2
debug 2021-11-15T12:49:12.072+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980552076908, "job": 1, "event": "recovery_finished"}
debug 2021-11-15T12:49:12.072+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824
debug 2021-11-15T12:49:12.072+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta old nid_max 5338942
debug 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta old blobid_max 296960
debug 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta freelist_type bitmap
debug 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta ondisk_format 4 compat_ondisk_format 3
debug 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta min_alloc_size 0x1000
debug 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 freelist init
debug 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 freelist _read_cfg
debug 2021-11-15T12:49:12.120+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _init_alloc opening allocation metadata
debug 2021-11-15T12:49:12.952+0000 7f86cebf1080 1 HybridAllocator _spillover_range constructing fallback allocator
debug 2021-11-15T12:49:13.480+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _init_alloc loaded 24 GiB in 1287936 extents, allocator type hybrid, capacity 0x31ffc00000, block size 0x1000, free 0x5fd19c000, fragmentation 0.205099
debug 2021-11-15T12:49:13.480+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work
debug 2021-11-15T12:49:13.480+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete
debug 2021-11-15T12:49:13.484+0000 7f86cebf1080 1 bluefs umount
debug 2021-11-15T12:49:13.484+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) close
debug 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs mount
debug 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs _init_alloc shared, id 1, capacity 0x31ffc00000, block size 0x10000
debug 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 bluefs mount shared_bdev_used = 756088832
debug 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _prepare_db_environment set db_paths to db,204006961971 db.slow,204006961971
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: RocksDB version: 6.8.1

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Compile date Sep 16 2021
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: DB SUMMARY

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: CURRENT file: CURRENT

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: IDENTITY file: IDENTITY

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: MANIFEST file: MANIFEST-031387 size: 238322 Bytes

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: SST files in db dir, Total Num: 11, files: 033348.sst 033428.sst 033429.sst 033430.sst 033431.sst 033432.sst 033433.sst 033434.sst 033443.sst

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: SST files in db.slow dir, Total Num: 0, files:

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Write Ahead Log file in db.wal: 033446.log size: 221571739 ;

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.error_if_exists: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.create_if_missing: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.paranoid_checks: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.env: 0x561ff4e9cae0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.fs: Legacy File System
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.info_log: 0x561ff4945ba0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_file_opening_threads: 16
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.statistics: (nil)
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_fsync: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_log_file_size: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_manifest_file_size: 1073741824
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.log_file_time_to_roll: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.keep_log_file_num: 1000
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.recycle_log_file_num: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_fallocate: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_reads: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_writes: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_direct_reads: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.create_missing_column_families: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.db_log_dir:
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_dir: db.wal
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_cache_numshardbits: 6
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_subcompactions: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_flushes: -1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.WAL_ttl_seconds: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.WAL_size_limit_MB: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.manifest_preallocation_size: 4194304
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.is_fd_close_on_exec: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.advise_random_on_open: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.db_write_buffer_size: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_manager: 0x561fe2ea73e0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.access_hint_on_compaction_start: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.random_access_max_buffer_size: 1048576
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_adaptive_mutex: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.rate_limiter: (nil)
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_recovery_mode: 2
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_thread_tracking: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_pipelined_write: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.unordered_write: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_concurrent_memtable_write: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_thread_max_yield_usec: 100
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_thread_slow_yield_usec: 3
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.row_cache: None
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_filter: None
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_recovery: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_ingest_behind: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.preserve_deletes: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.two_write_queues: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.manual_wal_flush: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.atomic_flush: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.persist_stats_to_disk: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_dbid_to_manifest: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.log_readahead_size: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.sst_file_checksum_func: Unknown
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_jobs: 2
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_compactions: 2
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_shutdown: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.writable_file_max_buffer_size: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.delayed_write_rate : 16777216
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_total_wal_size: 1073741824
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_dump_period_sec: 600
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_persist_period_sec: 600
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_history_buffer_size: 1048576
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_open_files: -1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bytes_per_sync: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_bytes_per_sync: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.strict_bytes_per_sync: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_readahead_size: 2097152
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Compression algorithms supported:
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZSTDNotFinalCompression supported: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZSTD supported: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kXpressCompression supported: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kLZ4HCCompression supported: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kLZ4Compression supported: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kBZip2Compression supported: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZlibCompression supported: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kSnappyCompression supported: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Fast CRC32 supported: Supported on x86
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: db/MANIFEST-031387

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]:

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter: None
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter_factory: None
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_factory: SkipListFactory
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_factory: BlockBasedTable
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561fe214e608)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 0
pin_l0_filter_and_index_blocks_in_cache: 0
pin_top_level_index_and_filter: 1
index_type: 0
data_block_index_type: 0
index_shortening: 1
data_block_hash_table_util_ratio: 0.750000
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x561fe20a7210
block_cache_name: BinnedLRUCache
block_cache_options:
capacity : 1449551462
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
metadata_block_size: 4096
partition_filters: 0
use_delta_encoding: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
verify_compression: 0
read_amp_bytes_per_bit: 0
format_version: 2
enable_index_compression: 1
block_align: 0

debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_size: 268435456
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number: 4
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression: NoCompression
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression: Disabled
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.prefix_extractor: nullptr
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.num_levels: 7
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.level: 32767
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.enabled: false
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.window_bits: -14
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.level: 32767
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.strategy: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.enabled: false
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_stop_writes_trigger: 36
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_base: 67108864
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_multiplier: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_base: 268435456
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_compaction_bytes: 1677721600
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.arena_block_size: 33554432
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.disable_auto_compactions: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_properties_collectors:
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_support: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_num_locks: 10000
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_whole_key_filtering: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_huge_page_size: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bloom_locality: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_successive_merges: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.optimize_filters_for_hits: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.paranoid_file_checks: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.force_consistency_checks: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.report_bg_io_stats: 0
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.ttl: 2592000
debug 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.periodic_compaction_seconds: 0
debug 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:db/MANIFEST-031387 succeeded,manifest_file_number is 31387, next_file_number is 33449, last_sequence is 9496239791, log_number is 33446,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0

debug 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 33446

debug 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980553626461, "job": 1, "event": "recovery_started", "log_files": [33446]}
debug 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #33446 mode 2
debug 2021-11-15T12:49:14.524+0000 7f86cebf1080 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5.
debug 2021-11-15T12:49:14.556+0000 7f86cebf1080 1 bluefs _allocate unable to allocate 0x90000 on bdev 1, allocator name block, allocator type hybrid, capacity 0x31ffc00000, block size 0x1000, free 0x5d008c000, fragmentation 0.211058, allocated 0x0
debug 2021-11-15T12:49:14.556+0000 7f86cebf1080 -1 bluefs _allocate allocation failed, needed 0x80c40
debug 2021-11-15T12:49:14.556+0000 7f86cebf1080 -1 bluefs _flush_range allocated: 0x0 offset: 0x0 length: 0x80c40
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.2.6/rpm/el8/BUILD/ceph-16.2.6/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f86cebf1080 time 2021-11-15T12:49:14.560240+0000
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.2.6/rpm/el8/BUILD/ceph-16.2.6/src/os/bluestore/BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")
ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable)
1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x561fd7f10c8c]
2: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1131) [0x561fd8607901]
3: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x90) [0x561fd8607be0]
4: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x561fd8618cf2]
5: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x11b) [0x561fd863132b]
6: (rocksdb::LegacyWritableFileWrapper::Append(rocksdb::Slice const&, rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x561fd8ac3acf]
7: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x58a) [0x561fd8bd581a]
8: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2d0) [0x561fd8bd6c70]
9: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x561fd8cf2416]
10: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x561fd8cf2d5c]
11: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x561fd8cf347c]
12: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x561fd8cf350d]
13: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2b8) [0x561fd8cf6978]
14: (rocksdb::BuildTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::Env*, rocksdb::FileSystem*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase<rocksdb::Slice>*, std::vector<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> >, std::allocator<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> > > >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::SnapshotChecker*, rocksdb::CompressionType, unsigned long, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int, unsigned long, unsigned long, rocksdb::Env::WriteLifeTimeHint, unsigned long)+0xa45) [0x561fd8ca13d5]
15: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0xcf5) [0x561fd8b06415]
16: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool, bool*)+0x1c2e) [0x561fd8b08b4e]
17: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0xae8) [0x561fd8b09ea8]
18: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x59d) [0x561fd8b03bcd]
19: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x561fd8b04f65]
20: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10c1) [0x561fd8a7cec1]
21: (BlueStore::_open_db(bool, bool, bool)+0x948) [0x561fd85024d8]
22: (BlueStore::_open_db_and_around(bool, bool)+0x2f7) [0x561fd856c657]
23: (BlueStore::_mount()+0x204) [0x561fd856f514]
24: (OSD::init()+0x380) [0x561fd8044a10]
25: main()
26: __libc_start_main()
27: _start()
*** Caught signal (Aborted) **
in thread 7f86cebf1080 thread_name:ceph-osd
debug 2021-11-15T12:49:14.568+0000 7f86cebf1080 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.2.6/rpm/el8/BUILD/ceph-16.2.6/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f86cebf1080 time 2021-11-15T12:49:14.560240+0000
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.2.6/rpm/el8/BUILD/ceph-16.2.6/src/os/bluestore/BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")

ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable)
1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x561fd7f10c8c]
2: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1131) [0x561fd8607901]
3: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x90) [0x561fd8607be0]
4: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x561fd8618cf2]
5: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x11b) [0x561fd863132b]
6: (rocksdb::LegacyWritableFileWrapper::Append(rocksdb::Slice const&, rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x561fd8ac3acf]
7: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x58a) [0x561fd8bd581a]
8: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2d0) [0x561fd8bd6c70]
9: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x561fd8cf2416]
10: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x561fd8cf2d5c]
11: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x561fd8cf347c]
12: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x561fd8cf350d]
13: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2b8) [0x561fd8cf6978]
14: (rocksdb::BuildTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::Env*, rocksdb::FileSystem*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase<rocksdb::Slice>*, std::vector<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> >, std::allocator<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> > > >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::SnapshotChecker*, rocksdb::CompressionType, unsigned long, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int, unsigned long, unsigned long, rocksdb::Env::WriteLifeTimeHint, unsigned long)+0xa45) [0x561fd8ca13d5]
15: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0xcf5) [0x561fd8b06415]
16: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool, bool*)+0x1c2e) [0x561fd8b08b4e]
17: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0xae8) [0x561fd8b09ea8]
18: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x59d) [0x561fd8b03bcd]
19: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x561fd8b04f65]
20: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10c1) [0x561fd8a7cec1]
21: (BlueStore::_open_db(bool, bool, bool)+0x948) [0x561fd85024d8]
22: (BlueStore::_open_db_and_around(bool, bool)+0x2f7) [0x561fd856c657]
23: (BlueStore::_mount()+0x204) [0x561fd856f514]
24: (OSD::init()+0x380) [0x561fd8044a10]
25: main()
26: __libc_start_main()
27: _start()

ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable)
1: /lib64/libpthread.so.0(+0x12b20) [0x7f86ccb97b20]
2: gsignal()
3: abort()
4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b6) [0x561fd7f10d5d]
5: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1131) [0x561fd8607901]
6: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x90) [0x561fd8607be0]
7: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x561fd8618cf2]
8: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x11b) [0x561fd863132b]
9: (rocksdb::LegacyWritableFileWrapper::Append(rocksdb::Slice const&, rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x561fd8ac3acf]
10: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x58a) [0x561fd8bd581a]
11: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2d0) [0x561fd8bd6c70]
12: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x561fd8cf2416]
13: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x561fd8cf2d5c]
14: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x561fd8cf347c]
15: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x561fd8cf350d]
16: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2b8) [0x561fd8cf6978]
17: (rocksdb::BuildTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::Env*, rocksdb::FileSystem*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase<rocksdb::Slice>*, std::vector<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> >, std::allocator<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> > > >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::SnapshotChecker*, rocksdb::CompressionType, unsigned long, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int, unsigned long, unsigned long, rocksdb::Env::WriteLifeTimeHint, unsigned long)+0xa45) [0x561fd8ca13d5]
18: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0xcf5) [0x561fd8b06415]
19: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool, bool*)+0x1c2e) [0x561fd8b08b4e]
20: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0xae8) [0x561fd8b09ea8]
21: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x59d) [0x561fd8b03bcd]
22: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x561fd8b04f65]
23: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10c1) [0x561fd8a7cec1]
24: (BlueStore::_open_db(bool, bool, bool)+0x948) [0x561fd85024d8]
25: (BlueStore::_open_db_and_around(bool, bool)+0x2f7) [0x561fd856c657]
26: (BlueStore::_mount()+0x204) [0x561fd856f514]
27: (OSD::init()+0x380) [0x561fd8044a10]
28: main()
29: __libc_start_main()
30: _start()
debug 2021-11-15T12:49:14.576+0000 7f86cebf1080 -1 *** Caught signal (Aborted) **
in thread 7f86cebf1080 thread_name:ceph-osd

ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable)
1: /lib64/libpthread.so.0(+0x12b20) [0x7f86ccb97b20]
2: gsignal()
3: abort()
4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b6) [0x561fd7f10d5d]
5: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1131) [0x561fd8607901]
6: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x90) [0x561fd8607be0]
7: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x561fd8618cf2]
8: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x11b) [0x561fd863132b]
9: (rocksdb::LegacyWritableFileWrapper::Append(rocksdb::Slice const&, rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x561fd8ac3acf]
10: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x58a) [0x561fd8bd581a]
11: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2d0) [0x561fd8bd6c70]
12: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x561fd8cf2416]
13: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x561fd8cf2d5c]
14: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x561fd8cf347c]
15: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x561fd8cf350d]
16: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2b8) [0x561fd8cf6978]
17: (rocksdb::BuildTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::Env*, rocksdb::FileSystem*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase<rocksdb::Slice>*, std::vector<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> >, std::allocator<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> > > >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::SnapshotChecker*, rocksdb::CompressionType, unsigned long, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int, unsigned long, unsigned long, rocksdb::Env::WriteLifeTimeHint, unsigned long)+0xa45) [0x561fd8ca13d5]
18: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0xcf5) [0x561fd8b06415]
19: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool, bool*)+0x1c2e) [0x561fd8b08b4e]
20: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0xae8) [0x561fd8b09ea8]
21: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x59d) [0x561fd8b03bcd]
22: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x561fd8b04f65]
23: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10c1) [0x561fd8a7cec1]
24: (BlueStore::_open_db(bool, bool, bool)+0x948) [0x561fd85024d8]
25: (BlueStore::_open_db_and_around(bool, bool)+0x2f7) [0x561fd856c657]
26: (BlueStore::_mount()+0x204) [0x561fd856f514]
27: (OSD::init()+0x380) [0x561fd8044a10]
28: main()
29: __libc_start_main()
30: _start()
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
debug -767> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command assert hook 0x561fe205a540
debug -766> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command abort hook 0x561fe205a540
debug -765> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command leak_some_memory hook 0x561fe205a540
debug -764> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perfcounters_dump hook 0x561fe205a540
debug -763> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command 1 hook 0x561fe205a540
debug -762> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf dump hook 0x561fe205a540
debug -761> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perfcounters_schema hook 0x561fe205a540
debug -760> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf histogram dump hook 0x561fe205a540
debug -759> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command 2 hook 0x561fe205a540
debug -758> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf schema hook 0x561fe205a540
debug -757> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf histogram schema hook 0x561fe205a540
debug -756> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf reset hook 0x561fe205a540
debug -755> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config show hook 0x561fe205a540
debug -754> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config help hook 0x561fe205a540
debug -753> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config set hook 0x561fe205a540
debug -752> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config unset hook 0x561fe205a540
debug -751> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config get hook 0x561fe205a540
debug -750> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config diff hook 0x561fe205a540
debug -749> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config diff get hook 0x561fe205a540
debug -748> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command injectargs hook 0x561fe205a540
debug -747> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command log flush hook 0x561fe205a540
debug -746> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command log dump hook 0x561fe205a540
debug -745> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command log reopen hook 0x561fe205a540
debug -744> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command dump_mempools hook 0x561fe205c5e8
debug -743> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: get_monmap_and_config
debug -742> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: build_initial_monmap
debug -741> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: monmap:
epoch 0
fsid addad34a-62d2-446a-8553-55461cacb31a
last_changed 2021-11-15T12:49:06.294343+0000
created 2021-11-15T12:49:06.294343+0000
min_mon_release 0 (unknown)
election_strategy: 1
0: [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0] mon.noname-c
1: [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0] mon.noname-b
2: [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0] mon.noname-a

debug -740> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -739> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -738> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -737> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: none
debug -736> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -735> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -734> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -733> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -732> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -731> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -730> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -729> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -728> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -727> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -726> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -725> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -724> 2021-11-15T12:49:06.292+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -723> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: init
debug -722> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding auth protocol: cephx
debug -721> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding auth protocol: cephx
debug -720> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding auth protocol: cephx
debug -719> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding auth protocol: none
debug -718> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -717> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -716> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -715> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -714> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -713> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -712> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -711> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -710> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -709> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -708> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -707> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -706> 2021-11-15T12:49:06.292+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -705> 2021-11-15T12:49:06.292+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -704> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: _reopen_session rank -1
debug -703> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: _add_conns ranks=[1,0,2]
debug -702> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): picked mon.noname-b con 0x561fe2108c00 addr [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0]
debug -701> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): picked mon.noname-c con 0x561fe2109000 addr [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0]
debug -700> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): picked mon.noname-a con 0x561fe2109400 addr [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0]
debug -699> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): start opening mon connection
debug -698> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): start opening mon connection
debug -697> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): start opening mon connection
debug -696> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): _renew_subs
debug -695> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): authenticate will time out at 2021-11-15T12:54:06.295395+0000
debug -694> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): get_auth_request con 0x561fe2108c00 auth_method 0
debug -693> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
debug -692> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): _init_auth method 2
debug -691> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): _init_auth creating new auth
debug -690> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): handle_auth_reply_more payload 9
debug -689> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): handle_auth_reply_more payload_len 9
debug -688> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
debug -687> 2021-11-15T12:49:06.292+0000 7f86c8ba0700 10 monclient(hunting): get_auth_request con 0x561fe2109400 auth_method 0
debug -686> 2021-11-15T12:49:06.292+0000 7f86c8ba0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
debug -685> 2021-11-15T12:49:06.292+0000 7f86c8ba0700 10 monclient(hunting): _init_auth method 2
debug -684> 2021-11-15T12:49:06.292+0000 7f86c8ba0700 10 monclient(hunting): _init_auth creating new auth
debug -683> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): handle_auth_done global_id 177814234 payload 274
debug -682> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient: _finish_hunting 0
debug -681> 2021-11-15T12:49:06.292+0000 7f86c839f700 1 monclient: found mon.noname-b
debug -680> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient: _send_mon_message to mon.noname-b at v2:10.83.19.24:3300/0
debug -679> 2021-11-15T12:49:06.292+0000 7f86c7b9e700 10 monclient: get_auth_request con 0x561fe2109000 auth_method 0
debug -678> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: handle_monmap mon_map magic: 0 v1
debug -677> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: got monmap 6 from mon.noname-b (according to old e6)
debug -676> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: dump:
epoch 6
fsid addad34a-62d2-446a-8553-55461cacb31a
last_changed 2021-10-19T09:21:56.109623+0000
created 2020-09-21T15:53:16.921162+0000
min_mon_release 16 (pacific)
election_strategy: 1
0: [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0] mon.a
1: [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0] mon.d
2: [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0] mon.e

debug -675> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: _finish_auth 0
debug -674> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-11-15T12:48:36.297485+0000)
debug -673> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: _send_mon_message to mon.d at v2:10.83.19.24:3300/0
debug -672> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 monclient: authenticate success, global_id 177814234
debug -671> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: handle_config config(8 keys) v1
debug -670> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: get_monmap_and_config success
debug -669> 2021-11-15T12:49:06.292+0000 7f86cebf1080 4 set_mon_vals no callback set
debug -668> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: handle_monmap mon_map magic: 0 v1
debug -667> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 set_mon_vals log_to_file = false
debug -666> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 set_mon_vals mon_allow_pool_delete = true
debug -665> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 set_mon_vals mon_allow_pool_size_one = true
debug -664> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 set_mon_vals mon_cluster_log_file =
debug -663> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: got monmap 6 from mon.d (according to old e6)
debug -662> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 set_mon_vals mon_pg_warn_min_per_osd = 0
debug -661> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 set_mon_vals osd_pool_default_pg_autoscale_mode = on
debug -660> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 set_mon_vals osd_scrub_auto_repair = true
debug -659> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 set_mon_vals rbd_default_features = 3
debug -658> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: dump:
epoch 6
fsid addad34a-62d2-446a-8553-55461cacb31a
last_changed 2021-10-19T09:21:56.109623+0000
created 2020-09-21T15:53:16.921162+0000
min_mon_release 16 (pacific)
election_strategy: 1
0: [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0] mon.a
1: [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0] mon.d
2: [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0] mon.e

debug -657> 2021-11-15T12:49:06.296+0000 7f86c739d700 10 monclient: _finish_auth 0
debug -656> 2021-11-15T12:49:06.296+0000 7f86c739d700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2021-11-15T12:48:36.298298+0000)
debug -655> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 monclient: shutdown
debug -654> 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 set uid:gid to 167:167 (ceph:ceph)
debug -653> 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable), process ceph-osd, pid 1
debug -652> 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 pidfile_write: ignore empty --pid-file
debug -651> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) init /var/run/ceph/ceph-osd.2.asok
debug -650> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
debug -649> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command 0 hook 0x561fe205e1e8
debug -648> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command version hook 0x561fe205e1e8
debug -647> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command git_version hook 0x561fe205e1e8
debug -646> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command help hook 0x561fe205a390
debug -645> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command get_command_descriptions hook 0x561fe205a330
debug -644> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 object store type is bluestore
debug -643> 2021-11-15T12:49:06.296+0000 7f86c739d700 5 asok(0x561fe210c000) entry start
debug -642> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -641> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -640> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -639> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore bluefs device info hook 0x561fe205a4e0
debug -638> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs stats hook 0x561fe205a4e0
debug -637> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs files list hook 0x561fe205a4e0
debug -636> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs debug_inject_read_zeros hook 0x561fe205a4e0
debug -635> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -634> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -633> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug -632> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs debug_inject_read_zeros
debug -631> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs files list
debug -630> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs stats
debug -629> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluestore bluefs device info
debug -628> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) close
debug -627> 2021-11-15T12:49:06.608+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) close
debug -626> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -625> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -624> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -623> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: none
debug -622> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -621> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -620> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -619> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -618> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -617> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -616> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -615> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -614> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -613> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -612> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -611> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -610> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -609> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding auth protocol: cephx
debug -608> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding auth protocol: cephx
debug -607> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding auth protocol: cephx
debug -606> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding auth protocol: none
debug -605> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -604> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -603> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -602> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -601> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -600> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -599> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -598> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -597> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -596> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -595> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -594> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -593> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -592> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding auth protocol: cephx
debug -591> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding auth protocol: cephx
debug -590> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding auth protocol: cephx
debug -589> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding auth protocol: none
debug -588> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -587> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -586> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -585> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -584> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -583> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -582> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -581> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -580> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -579> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -578> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -577> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -576> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -575> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding auth protocol: cephx
debug -574> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding auth protocol: cephx
debug -573> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding auth protocol: cephx
debug -572> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding auth protocol: none
debug -571> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -570> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -569> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -568> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -567> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -566> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -565> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -564> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -563> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -562> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -561> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -560> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -559> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -558> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding auth protocol: cephx
debug -557> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding auth protocol: cephx
debug -556> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding auth protocol: cephx
debug -555> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding auth protocol: none
debug -554> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -553> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -552> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -551> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -550> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -549> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -548> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -547> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -546> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -545> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -544> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -543> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -542> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -541> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding auth protocol: cephx
debug -540> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding auth protocol: cephx
debug -539> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding auth protocol: cephx
debug -538> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding auth protocol: none
debug -537> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -536> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -535> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -534> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -533> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -532> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -531> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -530> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -529> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -528> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -527> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -526> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -525> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -524> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding auth protocol: cephx
debug -523> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding auth protocol: cephx
debug -522> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding auth protocol: cephx
debug -521> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding auth protocol: none
debug -520> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -519> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -518> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -517> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -516> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -515> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -514> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -513> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -512> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -511> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -510> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -509> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -508> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -507> 2021-11-15T12:49:06.840+0000 7f86cebf1080 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
debug -506> 2021-11-15T12:49:06.840+0000 7f86cebf1080 -1 Falling back to public interface
debug -505> 2021-11-15T12:49:06.844+0000 7f86cebf1080 10 monclient: build_initial_monmap
debug -504> 2021-11-15T12:49:06.844+0000 7f86cebf1080 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
election_strategy: 1
0: [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0] mon.noname-a
1: [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0] mon.noname-b
2: [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0] mon.noname-c

debug -503> 2021-11-15T12:49:06.852+0000 7f86cebf1080 0 load: jerasure load: lrc load: isa
debug -502> 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -501> 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -500> 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -499> 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -498> 2021-11-15T12:49:07.160+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command objecter_requests hook 0x561fe205a990
debug -497> 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -496> 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -495> 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -494> 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -493> 2021-11-15T12:49:07.452+0000 7f86cebf1080 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -492> 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -491> 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -490> 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -489> 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -488> 2021-11-15T12:49:07.752+0000 7f86cebf1080 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -487> 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -486> 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -485> 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -484> 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -483> 2021-11-15T12:49:08.048+0000 7f86cebf1080 0 osd.2:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -482> 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -481> 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -480> 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -479> 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -478> 2021-11-15T12:49:08.344+0000 7f86cebf1080 0 osd.2:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -477> 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -476> 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -475> 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -474> 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -473> 2021-11-15T12:49:08.628+0000 7f86cebf1080 0 osd.2:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -472> 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -471> 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -470> 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -469> 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -468> 2021-11-15T12:49:08.920+0000 7f86cebf1080 0 osd.2:5.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -467> 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -466> 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -465> 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -464> 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -463> 2021-11-15T12:49:09.216+0000 7f86cebf1080 0 osd.2:6.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -462> 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -461> 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -460> 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -459> 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -458> 2021-11-15T12:49:09.512+0000 7f86cebf1080 0 osd.2:7.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -457> 2021-11-15T12:49:09.512+0000 7f86cebf1080 2 osd.2 0 init /var/lib/ceph/osd/ceph-2 (looks like ssd)
debug -456> 2021-11-15T12:49:09.512+0000 7f86cebf1080 2 osd.2 0 journal /var/lib/ceph/osd/ceph-2/journal
debug -455> 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _mount path /var/lib/ceph/osd/ceph-2
debug -454> 2021-11-15T12:49:09.512+0000 7f86cebf1080 0 bluestore(/var/lib/ceph/osd/ceph-2) _open_db_and_around read-only:0 repair:0
debug -453> 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -452> 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -451> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -450> 2021-11-15T12:49:09.516+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore bluefs device info hook 0x561fe205a9a0
debug -449> 2021-11-15T12:49:09.516+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs stats hook 0x561fe205a9a0
debug -448> 2021-11-15T12:49:09.516+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs files list hook 0x561fe205a9a0
debug -447> 2021-11-15T12:49:09.516+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs debug_inject_read_zeros hook 0x561fe205a9a0
debug -446> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -445> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -444> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug -443> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs mount
debug -442> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs _init_alloc shared, id 1, capacity 0x31ffc00000, block size 0x10000
debug -441> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 bluefs mount shared_bdev_used = 0
debug -440> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _prepare_db_environment set db_paths to db,204006961971 db.slow,204006961971
debug -439> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_total_wal_size = 1073741824
debug -438> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option compaction_readahead_size = 2097152
debug -437> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_write_buffer_number = 4
debug -436> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_background_compactions = 2
debug -435> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option compression = kNoCompression
debug -434> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option writable_file_max_buffer_size = 0
debug -433> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option min_write_buffer_number_to_merge = 1
debug -432> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option recycle_log_file_num = 4
debug -431> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option write_buffer_size = 268435456
debug -430> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_total_wal_size = 1073741824
debug -429> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option compaction_readahead_size = 2097152
debug -428> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_write_buffer_number = 4
debug -427> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_background_compactions = 2
debug -426> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option compression = kNoCompression
debug -425> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option writable_file_max_buffer_size = 0
debug -424> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option min_write_buffer_number_to_merge = 1
debug -423> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option recycle_log_file_num = 4
debug -422> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option write_buffer_size = 268435456
debug -421> 2021-11-15T12:49:09.604+0000 7f86cebf1080 5 rocksdb: verify_sharding column families from rocksdb: [default]
debug -420> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: RocksDB version: 6.8.1

debug -419> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
debug -418> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Compile date Sep 16 2021
debug -417> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: DB SUMMARY

debug -416> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: CURRENT file: CURRENT

debug -415> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: IDENTITY file: IDENTITY

debug -414> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: MANIFEST file: MANIFEST-031387 size: 238322 Bytes

debug -413> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: SST files in db dir, Total Num: 11, files: 033348.sst 033428.sst 033429.sst 033430.sst 033431.sst 033432.sst 033433.sst 033434.sst 033443.sst

debug -412> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: SST files in db.slow dir, Total Num: 0, files:

debug -411> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Write Ahead Log file in db.wal: 033446.log size: 221571739 ;

debug -410> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.error_if_exists: 0
debug -409> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.create_if_missing: 0
debug -408> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.paranoid_checks: 1
debug -407> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.env: 0x561fe20cb080
debug -406> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.fs: Legacy File System
debug -405> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.info_log: 0x561fe20cbb20
debug -404> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_file_opening_threads: 16
debug -403> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.statistics: (nil)
debug -402> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_fsync: 0
debug -401> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_log_file_size: 0
debug -400> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_manifest_file_size: 1073741824
debug -399> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.log_file_time_to_roll: 0
debug -398> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.keep_log_file_num: 1000
debug -397> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.recycle_log_file_num: 0
debug -396> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_fallocate: 1
debug -395> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_reads: 0
debug -394> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_writes: 0
debug -393> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_direct_reads: 0
debug -392> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
debug -391> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.create_missing_column_families: 0
debug -390> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.db_log_dir:
debug -389> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_dir: db.wal
debug -388> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_cache_numshardbits: 6
debug -387> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_subcompactions: 1
debug -386> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_flushes: -1
debug -385> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.WAL_ttl_seconds: 0
debug -384> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.WAL_size_limit_MB: 0
debug -383> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576
debug -382> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.manifest_preallocation_size: 4194304
debug -381> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.is_fd_close_on_exec: 1
debug -380> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.advise_random_on_open: 1
debug -379> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.db_write_buffer_size: 0
debug -378> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_manager: 0x561fe2ea6ab0
debug -377> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.access_hint_on_compaction_start: 1
debug -376> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
debug -375> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.random_access_max_buffer_size: 1048576
debug -374> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_adaptive_mutex: 0
debug -373> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.rate_limiter: (nil)
debug -372> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
debug -371> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_recovery_mode: 2
debug -370> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_thread_tracking: 0
debug -369> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_pipelined_write: 0
debug -368> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.unordered_write: 0
debug -367> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_concurrent_memtable_write: 1
debug -366> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
debug -365> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_thread_max_yield_usec: 100
debug -364> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_thread_slow_yield_usec: 3
debug -363> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.row_cache: None
debug -362> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_filter: None
debug -361> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_recovery: 0
debug -360> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_ingest_behind: 0
debug -359> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.preserve_deletes: 0
debug -358> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.two_write_queues: 0
debug -357> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.manual_wal_flush: 0
debug -356> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.atomic_flush: 0
debug -355> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
debug -354> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.persist_stats_to_disk: 0
debug -353> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_dbid_to_manifest: 0
debug -352> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.log_readahead_size: 0
debug -351> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.sst_file_checksum_func: Unknown
debug -350> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_jobs: 2
debug -349> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_compactions: 2
debug -348> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_shutdown: 0
debug -347> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.writable_file_max_buffer_size: 0
debug -346> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.delayed_write_rate : 16777216
debug -345> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_total_wal_size: 1073741824
debug -344> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
debug -343> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_dump_period_sec: 600
debug -342> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_persist_period_sec: 600
debug -341> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_history_buffer_size: 1048576
debug -340> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_open_files: -1
debug -339> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bytes_per_sync: 0
debug -338> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_bytes_per_sync: 0
debug -337> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.strict_bytes_per_sync: 0
debug -336> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_readahead_size: 2097152
debug -335> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Compression algorithms supported:
debug -334> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZSTDNotFinalCompression supported: 0
debug -333> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZSTD supported: 0
debug -332> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kXpressCompression supported: 0
debug -331> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kLZ4HCCompression supported: 1
debug -330> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kLZ4Compression supported: 1
debug -329> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kBZip2Compression supported: 0
debug -328> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZlibCompression supported: 1
debug -327> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kSnappyCompression supported: 1
debug -326> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Fast CRC32 supported: Supported on x86
debug -325> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_readonly.cc:23] Opening the db in read only mode
debug -324> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: db/MANIFEST-031387

debug -323> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]:

debug -322> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
debug -321> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
debug -320> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter: None
debug -319> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter_factory: None
debug -318> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_factory: SkipListFactory
debug -317> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_factory: BlockBasedTable
debug -316> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561fe205e7b0)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 0
pin_l0_filter_and_index_blocks_in_cache: 0
pin_top_level_index_and_filter: 1
index_type: 0
data_block_index_type: 0
index_shortening: 1
data_block_hash_table_util_ratio: 0.750000
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x561fe20a7210
block_cache_name: BinnedLRUCache
block_cache_options:
capacity : 1449551462
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
metadata_block_size: 4096
partition_filters: 0
use_delta_encoding: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
verify_compression: 0
read_amp_bytes_per_bit: 0
format_version: 2
enable_index_compression: 1
block_align: 0

debug -315> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_size: 268435456
debug -314> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number: 4
debug -313> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression: NoCompression
debug -312> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression: Disabled
debug -311> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.prefix_extractor: nullptr
debug -310> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
debug -309> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.num_levels: 7
debug -308> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
debug -307> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
debug -306> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0
debug -305> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
debug -304> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.level: 32767
debug -303> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
debug -302> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
debug -301> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
debug -300> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.enabled: false
debug -299> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.window_bits: -14
debug -298> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.level: 32767
debug -297> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.strategy: 0
debug -296> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
debug -295> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
debug -294> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.enabled: false
debug -293> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
debug -292> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
debug -291> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_stop_writes_trigger: 36
debug -290> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_base: 67108864
debug -289> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_multiplier: 1
debug -288> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_base: 268435456
debug -287> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
debug -286> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
debug -285> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
debug -284> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
debug -283> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
debug -282> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
debug -281> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
debug -280> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
debug -279> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
debug -278> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
debug -277> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_compaction_bytes: 1677721600
debug -276> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.arena_block_size: 33554432
debug -275> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
debug -274> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
debug -273> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
debug -272> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.disable_auto_compactions: 0
debug -271> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
debug -270> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
debug -269> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
debug -268> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
debug -267> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
debug -266> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
debug -265> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
debug -264> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
debug -263> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
debug -262> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
debug -261> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_properties_collectors:
debug -260> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_support: 0
debug -259> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_num_locks: 10000
debug -258> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
debug -257> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_whole_key_filtering: 0
debug -256> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_huge_page_size: 0
debug -255> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bloom_locality: 0
debug -254> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_successive_merges: 0
debug -253> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.optimize_filters_for_hits: 0
debug -252> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.paranoid_file_checks: 0
debug -251> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.force_consistency_checks: 0
debug -250> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.report_bg_io_stats: 0
debug -249> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.ttl: 2592000
debug -248> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.periodic_compaction_seconds: 0
debug -247> 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:db/MANIFEST-031387 succeeded,manifest_file_number is 31387, next_file_number is 33449, last_sequence is 9496239791, log_number is 33446,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0

debug -246> 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 33446

debug -245> 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980549639149, "job": 1, "event": "recovery_started", "log_files": [33446]}
debug -244> 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #33446 mode 2
debug -243> 2021-11-15T12:49:12.072+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980552076908, "job": 1, "event": "recovery_finished"}
debug -242> 2021-11-15T12:49:12.072+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824
debug -241> 2021-11-15T12:49:12.072+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta old nid_max 5338942
debug -240> 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta old blobid_max 296960
debug -239> 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta freelist_type bitmap
debug -238> 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta ondisk_format 4 compat_ondisk_format 3
debug -237> 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta min_alloc_size 0x1000
debug -236> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 get compressor snappy = 0
debug -235> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 load compressor snappy
debug -234> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 add compressor snappy 0x561fe2ea7e90
debug -233> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 get compressor snappy = 0x561fe2ea7e90
debug -232> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 load: compressor snappy loaded and registered
debug -231> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 get compressor snappy = 0x561fe2ea7e90
debug -230> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 freelist init
debug -229> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 freelist _read_cfg
debug -228> 2021-11-15T12:49:12.120+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator dump block hook 0x561fe2ea7da0
debug -227> 2021-11-15T12:49:12.120+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator score block hook 0x561fe2ea7da0
debug -226> 2021-11-15T12:49:12.120+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator fragmentation block hook 0x561fe2ea7da0
debug -225> 2021-11-15T12:49:12.120+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _init_alloc opening allocation metadata
debug -224> 2021-11-15T12:49:12.952+0000 7f86cebf1080 1 HybridAllocator _spillover_range constructing fallback allocator
debug -223> 2021-11-15T12:49:12.952+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator dump block.fallback hook 0x561fe2ea7d10
debug -222> 2021-11-15T12:49:12.952+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator score block.fallback hook 0x561fe2ea7d10
debug -221> 2021-11-15T12:49:12.952+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator fragmentation block.fallback hook 0x561fe2ea7d10
debug -220> 2021-11-15T12:49:13.480+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _init_alloc loaded 24 GiB in 1287936 extents, allocator type hybrid, capacity 0x31ffc00000, block size 0x1000, free 0x5fd19c000, fragmentation 0.205099
debug -219> 2021-11-15T12:49:13.480+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work
debug -218> 2021-11-15T12:49:13.480+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete
debug -217> 2021-11-15T12:49:13.484+0000 7f86cebf1080 1 bluefs umount
debug -216> 2021-11-15T12:49:13.484+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs debug_inject_read_zeros
debug -215> 2021-11-15T12:49:13.484+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs files list
debug -214> 2021-11-15T12:49:13.484+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs stats
debug -213> 2021-11-15T12:49:13.484+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluestore bluefs device info
debug -212> 2021-11-15T12:49:13.484+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) close
debug -211> 2021-11-15T12:49:13.560+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore bluefs device info hook 0x561fe205a9a0
debug -210> 2021-11-15T12:49:13.560+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs stats hook 0x561fe205a9a0
debug -209> 2021-11-15T12:49:13.560+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs files list hook 0x561fe205a9a0
debug -208> 2021-11-15T12:49:13.560+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs debug_inject_read_zeros hook 0x561fe205a9a0
debug -207> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -206> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -205> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug -204> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs mount
debug -203> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs _init_alloc shared, id 1, capacity 0x31ffc00000, block size 0x10000
debug -202> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 bluefs mount shared_bdev_used = 756088832
debug -201> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _prepare_db_environment set db_paths to db,204006961971 db.slow,204006961971
debug -200> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_total_wal_size = 1073741824
debug -199> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option compaction_readahead_size = 2097152
debug -198> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_write_buffer_number = 4
debug -197> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_background_compactions = 2
debug -196> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option compression = kNoCompression
debug -195> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option writable_file_max_buffer_size = 0
debug -194> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option min_write_buffer_number_to_merge = 1
debug -193> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option recycle_log_file_num = 4
debug -192> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option write_buffer_size = 268435456
debug -191> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_total_wal_size = 1073741824
debug -190> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option compaction_readahead_size = 2097152
debug -189> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_write_buffer_number = 4
debug -188> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_background_compactions = 2
debug -187> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option compression = kNoCompression
debug -186> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option writable_file_max_buffer_size = 0
debug -185> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option min_write_buffer_number_to_merge = 1
debug -184> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option recycle_log_file_num = 4
debug -183> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option write_buffer_size = 268435456
debug -182> 2021-11-15T12:49:13.620+0000 7f86cebf1080 5 rocksdb: verify_sharding column families from rocksdb: [default]
debug -181> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: RocksDB version: 6.8.1

debug -180> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
debug -179> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Compile date Sep 16 2021
debug -178> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: DB SUMMARY

debug -177> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: CURRENT file: CURRENT

debug -176> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: IDENTITY file: IDENTITY

debug -175> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: MANIFEST file: MANIFEST-031387 size: 238322 Bytes

debug -174> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: SST files in db dir, Total Num: 11, files: 033348.sst 033428.sst 033429.sst 033430.sst 033431.sst 033432.sst 033433.sst 033434.sst 033443.sst

debug -173> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: SST files in db.slow dir, Total Num: 0, files:

debug -172> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Write Ahead Log file in db.wal: 033446.log size: 221571739 ;

debug -171> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.error_if_exists: 0
debug -170> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.create_if_missing: 0
debug -169> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.paranoid_checks: 1
debug -168> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.env: 0x561ff4e9cae0
debug -167> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.fs: Legacy File System
debug -166> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.info_log: 0x561ff4945ba0
debug -165> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_file_opening_threads: 16
debug -164> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.statistics: (nil)
debug -163> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_fsync: 0
debug -162> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_log_file_size: 0
debug -161> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_manifest_file_size: 1073741824
debug -160> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.log_file_time_to_roll: 0
debug -159> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.keep_log_file_num: 1000
debug -158> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.recycle_log_file_num: 0
debug -157> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_fallocate: 1
debug -156> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_reads: 0
debug -155> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_writes: 0
debug -154> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_direct_reads: 0
debug -153> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
debug -152> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.create_missing_column_families: 0
debug -151> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.db_log_dir:
debug -150> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_dir: db.wal
debug -149> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_cache_numshardbits: 6
debug -148> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_subcompactions: 1
debug -147> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_flushes: -1
debug -146> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.WAL_ttl_seconds: 0
debug -145> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.WAL_size_limit_MB: 0
debug -144> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576
debug -143> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.manifest_preallocation_size: 4194304
debug -142> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.is_fd_close_on_exec: 1
debug -141> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.advise_random_on_open: 1
debug -140> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.db_write_buffer_size: 0
debug -139> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_manager: 0x561fe2ea73e0
debug -138> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.access_hint_on_compaction_start: 1
debug -137> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
debug -136> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.random_access_max_buffer_size: 1048576
debug -135> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_adaptive_mutex: 0
debug -134> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.rate_limiter: (nil)
debug -133> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
debug -132> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_recovery_mode: 2
debug -131> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_thread_tracking: 0
debug -130> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_pipelined_write: 0
debug -129> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.unordered_write: 0
debug -128> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_concurrent_memtable_write: 1
debug -127> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
debug -126> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_thread_max_yield_usec: 100
debug -125> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_thread_slow_yield_usec: 3
debug -124> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.row_cache: None
debug -123> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_filter: None
debug -122> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_recovery: 0
debug -121> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_ingest_behind: 0
debug -120> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.preserve_deletes: 0
debug -119> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.two_write_queues: 0
debug -118> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.manual_wal_flush: 0
debug -117> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.atomic_flush: 0
debug -116> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
debug -115> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.persist_stats_to_disk: 0
debug -114> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_dbid_to_manifest: 0
debug -113> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.log_readahead_size: 0
debug -112> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.sst_file_checksum_func: Unknown
debug -111> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_jobs: 2
debug -110> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_compactions: 2
debug -109> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_shutdown: 0
debug -108> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.writable_file_max_buffer_size: 0
debug -107> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.delayed_write_rate : 16777216
debug -106> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_total_wal_size: 1073741824
debug -105> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
debug -104> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_dump_period_sec: 600
debug -103> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_persist_period_sec: 600
debug -102> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_history_buffer_size: 1048576
debug -101> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_open_files: -1
debug -100> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bytes_per_sync: 0
debug -99> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_bytes_per_sync: 0
debug -98> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.strict_bytes_per_sync: 0
debug -97> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_readahead_size: 2097152
debug -96> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Compression algorithms supported:
debug -95> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZSTDNotFinalCompression supported: 0
debug -94> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZSTD supported: 0
debug -93> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kXpressCompression supported: 0
debug -92> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kLZ4HCCompression supported: 1
debug -91> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kLZ4Compression supported: 1
debug -90> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kBZip2Compression supported: 0
debug -89> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZlibCompression supported: 1
debug -88> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kSnappyCompression supported: 1
debug -87> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Fast CRC32 supported: Supported on x86
debug -86> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: db/MANIFEST-031387

debug -85> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]:

debug -84> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
debug -83> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
debug -82> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter: None
debug -81> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter_factory: None
debug -80> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_factory: SkipListFactory
debug -79> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_factory: BlockBasedTable
debug -78> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561fe214e608)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 0
pin_l0_filter_and_index_blocks_in_cache: 0
pin_top_level_index_and_filter: 1
index_type: 0
data_block_index_type: 0
index_shortening: 1
data_block_hash_table_util_ratio: 0.750000
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x561fe20a7210
block_cache_name: BinnedLRUCache
block_cache_options:
capacity : 1449551462
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
metadata_block_size: 4096
partition_filters: 0
use_delta_encoding: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
verify_compression: 0
read_amp_bytes_per_bit: 0
format_version: 2
enable_index_compression: 1
block_align: 0

debug -77> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_size: 268435456
debug -76> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number: 4
debug -75> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression: NoCompression
debug -74> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression: Disabled
debug -73> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.prefix_extractor: nullptr
debug -72> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
debug -71> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.num_levels: 7
debug -70> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
debug -69> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
debug -68> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0
debug -67> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
debug -66> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.level: 32767
debug -65> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
debug -64> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
debug -63> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
debug -62> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.enabled: false
debug -61> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.window_bits: -14
debug -60> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.level: 32767
debug -59> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.strategy: 0
debug -58> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
debug -57> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
debug -56> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.enabled: false
debug -55> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
debug -54> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
debug -53> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_stop_writes_trigger: 36
debug -52> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_base: 67108864
debug -51> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_multiplier: 1
debug -50> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_base: 268435456
debug -49> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
debug -48> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
debug -47> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
debug -46> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
debug -45> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
debug -44> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
debug -43> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
debug -42> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
debug -41> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
debug -40> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
debug -39> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_compaction_bytes: 1677721600
debug -38> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.arena_block_size: 33554432
debug -37> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
debug -36> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
debug -35> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
debug -34> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.disable_auto_compactions: 0
debug -33> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
debug -32> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
debug -31> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
debug -30> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
debug -29> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
debug -28> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
debug -27> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
debug -26> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
debug -25> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
debug -24> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
debug -23> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_properties_collectors:
debug -22> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_support: 0
debug -21> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_num_locks: 10000
debug -20> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
debug -19> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_whole_key_filtering: 0
debug -18> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_huge_page_size: 0
debug -17> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bloom_locality: 0
debug -16> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_successive_merges: 0
debug -15> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.optimize_filters_for_hits: 0
debug -14> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.paranoid_file_checks: 0
debug -13> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.force_consistency_checks: 0
debug -12> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.report_bg_io_stats: 0
debug -11> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.ttl: 2592000
debug -10> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.periodic_compaction_seconds: 0
debug -9> 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:db/MANIFEST-031387 succeeded,manifest_file_number is 31387, next_file_number is 33449, last_sequence is 9496239791, log_number is 33446,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0

debug -8> 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 33446

debug -7> 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980553626461, "job": 1, "event": "recovery_started", "log_files": [33446]}
debug -6> 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #33446 mode 2
debug -5> 2021-11-15T12:49:14.524+0000 7f86cebf1080 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5.
debug -4> 2021-11-15T12:49:14.556+0000 7f86cebf1080 1 bluefs _allocate unable to allocate 0x90000 on bdev 1, allocator name block, allocator type hybrid, capacity 0x31ffc00000, block size 0x1000, free 0x5d008c000, fragmentation 0.211058, allocated 0x0
debug -3> 2021-11-15T12:49:14.556+0000 7f86cebf1080 -1 bluefs _allocate allocation failed, needed 0x80c40
debug -2> 2021-11-15T12:49:14.556+0000 7f86cebf1080 -1 bluefs _flush_range allocated: 0x0 offset: 0x0 length: 0x80c40
debug -1> 2021-11-15T12:49:14.568+0000 7f86cebf1080 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.2.6/rpm/el8/BUILD/ceph-16.2.6/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f86cebf1080 time 2021-11-15T12:49:14.560240+0000
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.2.6/rpm/el8/BUILD/ceph-16.2.6/src/os/bluestore/BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")

ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable)
1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x561fd7f10c8c]
2: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1131) [0x561fd8607901]
3: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x90) [0x561fd8607be0]
4: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x561fd8618cf2]
5: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x11b) [0x561fd863132b]
6: (rocksdb::LegacyWritableFileWrapper::Append(rocksdb::Slice const&, rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x561fd8ac3acf]
7: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x58a) [0x561fd8bd581a]
8: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2d0) [0x561fd8bd6c70]
9: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x561fd8cf2416]
10: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x561fd8cf2d5c]
11: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x561fd8cf347c]
12: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x561fd8cf350d]
13: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2b8) [0x561fd8cf6978]
14: (rocksdb::BuildTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::Env*, rocksdb::FileSystem*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase<rocksdb::Slice>*, std::vector<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> >, std::allocator<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> > > >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::SnapshotChecker*, rocksdb::CompressionType, unsigned long, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int, unsigned long, unsigned long, rocksdb::Env::WriteLifeTimeHint, unsigned long)+0xa45) [0x561fd8ca13d5]
15: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0xcf5) [0x561fd8b06415]
16: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool, bool*)+0x1c2e) [0x561fd8b08b4e]
17: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0xae8) [0x561fd8b09ea8]
18: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x59d) [0x561fd8b03bcd]
19: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x561fd8b04f65]
20: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10c1) [0x561fd8a7cec1]
21: (BlueStore::_open_db(bool, bool, bool)+0x948) [0x561fd85024d8]
22: (BlueStore::_open_db_and_around(bool, bool)+0x2f7) [0x561fd856c657]
23: (BlueStore::_mount()+0x204) [0x561fd856f514]
24: (OSD::init()+0x380) [0x561fd8044a10]
25: main()
26: __libc_start_main()
27: _start()

debug 0> 2021-11-15T12:49:14.576+0000 7f86cebf1080 -1 *** Caught signal (Aborted) **
in thread 7f86cebf1080 thread_name:ceph-osd

ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable)
1: /lib64/libpthread.so.0(+0x12b20) [0x7f86ccb97b20]
2: gsignal()
3: abort()
4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b6) [0x561fd7f10d5d]
5: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1131) [0x561fd8607901]
6: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x90) [0x561fd8607be0]
7: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x561fd8618cf2]
8: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x11b) [0x561fd863132b]
9: (rocksdb::LegacyWritableFileWrapper::Append(rocksdb::Slice const&, rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x561fd8ac3acf]
10: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x58a) [0x561fd8bd581a]
11: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2d0) [0x561fd8bd6c70]
12: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x561fd8cf2416]
13: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x561fd8cf2d5c]
14: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x561fd8cf347c]
15: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x561fd8cf350d]
16: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2b8) [0x561fd8cf6978]
17: (rocksdb::BuildTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::Env*, rocksdb::FileSystem*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase<rocksdb::Slice>*, std::vector<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> >, std::allocator<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> > > >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::SnapshotChecker*, rocksdb::CompressionType, unsigned long, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int, unsigned long, unsigned long, rocksdb::Env::WriteLifeTimeHint, unsigned long)+0xa45) [0x561fd8ca13d5]
18: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0xcf5) [0x561fd8b06415]
19: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool, bool*)+0x1c2e) [0x561fd8b08b4e]
20: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0xae8) [0x561fd8b09ea8]
21: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x59d) [0x561fd8b03bcd]
22: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x561fd8b04f65]
23: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10c1) [0x561fd8a7cec1]
24: (BlueStore::_open_db(bool, bool, bool)+0x948) [0x561fd85024d8]
25: (BlueStore::_open_db_and_around(bool, bool)+0x2f7) [0x561fd856c657]
26: (BlueStore::_mount()+0x204) [0x561fd856f514]
27: (OSD::init()+0x380) [0x561fd8044a10]
28: main()
29: __libc_start_main()
30: _start()
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_pwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
2/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
0/ 5 cephfs_mirror
0/ 5 cephsqlite
-2/-2 (syslog threshold)
99/99 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
140216844801792 / admin_socket
140216853194496 / msgr-worker-2
140216861587200 / msgr-worker-1
140216869979904 / msgr-worker-0
140216970973312 / ceph-osd
max_recent 10000
max_new 10000
log_file
--- end dump of recent events ---
--- begin dump of recent events ---
debug -767> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command assert hook 0x561fe205a540
debug -766> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command abort hook 0x561fe205a540
debug -765> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command leak_some_memory hook 0x561fe205a540
debug -764> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perfcounters_dump hook 0x561fe205a540
debug -763> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command 1 hook 0x561fe205a540
debug -762> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf dump hook 0x561fe205a540
debug -761> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perfcounters_schema hook 0x561fe205a540
debug -760> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf histogram dump hook 0x561fe205a540
debug -759> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command 2 hook 0x561fe205a540
debug -758> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf schema hook 0x561fe205a540
debug -757> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf histogram schema hook 0x561fe205a540
debug -756> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command perf reset hook 0x561fe205a540
debug -755> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config show hook 0x561fe205a540
debug -754> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config help hook 0x561fe205a540
debug -753> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config set hook 0x561fe205a540
debug -752> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config unset hook 0x561fe205a540
debug -751> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config get hook 0x561fe205a540
debug -750> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config diff hook 0x561fe205a540
debug -749> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command config diff get hook 0x561fe205a540
debug -748> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command injectargs hook 0x561fe205a540
debug -747> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command log flush hook 0x561fe205a540
debug -746> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command log dump hook 0x561fe205a540
debug -745> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command log reopen hook 0x561fe205a540
debug -744> 2021-11-15T12:49:06.280+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command dump_mempools hook 0x561fe205c5e8
debug -743> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: get_monmap_and_config
debug -742> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: build_initial_monmap
debug -741> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: monmap:
epoch 0
fsid addad34a-62d2-446a-8553-55461cacb31a
last_changed 2021-11-15T12:49:06.294343+0000
created 2021-11-15T12:49:06.294343+0000
min_mon_release 0 (unknown)
election_strategy: 1
0: [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0] mon.noname-c
1: [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0] mon.noname-b
2: [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0] mon.noname-a

debug -740> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -739> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -738> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -737> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: none
debug -736> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -735> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -734> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -733> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -732> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -731> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -730> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -729> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -728> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -727> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -726> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -725> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -724> 2021-11-15T12:49:06.292+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -723> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: init
debug -722> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding auth protocol: cephx
debug -721> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding auth protocol: cephx
debug -720> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding auth protocol: cephx
debug -719> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding auth protocol: none
debug -718> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -717> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -716> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -715> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -714> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -713> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -712> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -711> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -710> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -709> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -708> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: crc
debug -707> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 AuthRegistry(0x7fff8190f260) adding con mode: secure
debug -706> 2021-11-15T12:49:06.292+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -705> 2021-11-15T12:49:06.292+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -704> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: _reopen_session rank -1
debug -703> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: _add_conns ranks=[1,0,2]
debug -702> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): picked mon.noname-b con 0x561fe2108c00 addr [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0]
debug -701> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): picked mon.noname-c con 0x561fe2109000 addr [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0]
debug -700> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): picked mon.noname-a con 0x561fe2109400 addr [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0]
debug -699> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): start opening mon connection
debug -698> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): start opening mon connection
debug -697> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): start opening mon connection
debug -696> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): _renew_subs
debug -695> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient(hunting): authenticate will time out at 2021-11-15T12:54:06.295395+0000
debug -694> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): get_auth_request con 0x561fe2108c00 auth_method 0
debug -693> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
debug -692> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): _init_auth method 2
debug -691> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): _init_auth creating new auth
debug -690> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): handle_auth_reply_more payload 9
debug -689> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): handle_auth_reply_more payload_len 9
debug -688> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
debug -687> 2021-11-15T12:49:06.292+0000 7f86c8ba0700 10 monclient(hunting): get_auth_request con 0x561fe2109400 auth_method 0
debug -686> 2021-11-15T12:49:06.292+0000 7f86c8ba0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
debug -685> 2021-11-15T12:49:06.292+0000 7f86c8ba0700 10 monclient(hunting): _init_auth method 2
debug -684> 2021-11-15T12:49:06.292+0000 7f86c8ba0700 10 monclient(hunting): _init_auth creating new auth
debug -683> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient(hunting): handle_auth_done global_id 177814234 payload 274
debug -682> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient: _finish_hunting 0
debug -681> 2021-11-15T12:49:06.292+0000 7f86c839f700 1 monclient: found mon.noname-b
debug -680> 2021-11-15T12:49:06.292+0000 7f86c839f700 10 monclient: _send_mon_message to mon.noname-b at v2:10.83.19.24:3300/0
debug -679> 2021-11-15T12:49:06.292+0000 7f86c7b9e700 10 monclient: get_auth_request con 0x561fe2109000 auth_method 0
debug -678> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: handle_monmap mon_map magic: 0 v1
debug -677> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: got monmap 6 from mon.noname-b (according to old e6)
debug -676> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: dump:
epoch 6
fsid addad34a-62d2-446a-8553-55461cacb31a
last_changed 2021-10-19T09:21:56.109623+0000
created 2020-09-21T15:53:16.921162+0000
min_mon_release 16 (pacific)
election_strategy: 1
0: [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0] mon.a
1: [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0] mon.d
2: [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0] mon.e

debug -675> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: _finish_auth 0
debug -674> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-11-15T12:48:36.297485+0000)
debug -673> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: _send_mon_message to mon.d at v2:10.83.19.24:3300/0
debug -672> 2021-11-15T12:49:06.292+0000 7f86cebf1080 5 monclient: authenticate success, global_id 177814234
debug -671> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: handle_config config(8 keys) v1
debug -670> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 monclient: get_monmap_and_config success
debug -669> 2021-11-15T12:49:06.292+0000 7f86cebf1080 4 set_mon_vals no callback set
debug -668> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: handle_monmap mon_map magic: 0 v1
debug -667> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 set_mon_vals log_to_file = false
debug -666> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 set_mon_vals mon_allow_pool_delete = true
debug -665> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 set_mon_vals mon_allow_pool_size_one = true
debug -664> 2021-11-15T12:49:06.292+0000 7f86cebf1080 10 set_mon_vals mon_cluster_log_file =
debug -663> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: got monmap 6 from mon.d (according to old e6)
debug -662> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 set_mon_vals mon_pg_warn_min_per_osd = 0
debug -661> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 set_mon_vals osd_pool_default_pg_autoscale_mode = on
debug -660> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 set_mon_vals osd_scrub_auto_repair = true
debug -659> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 set_mon_vals rbd_default_features = 3
debug -658> 2021-11-15T12:49:06.292+0000 7f86c739d700 10 monclient: dump:
epoch 6
fsid addad34a-62d2-446a-8553-55461cacb31a
last_changed 2021-10-19T09:21:56.109623+0000
created 2020-09-21T15:53:16.921162+0000
min_mon_release 16 (pacific)
election_strategy: 1
0: [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0] mon.a
1: [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0] mon.d
2: [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0] mon.e

debug -657> 2021-11-15T12:49:06.296+0000 7f86c739d700 10 monclient: _finish_auth 0
debug -656> 2021-11-15T12:49:06.296+0000 7f86c739d700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2021-11-15T12:48:36.298298+0000)
debug -655> 2021-11-15T12:49:06.296+0000 7f86cebf1080 10 monclient: shutdown
debug -654> 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 set uid:gid to 167:167 (ceph:ceph)
debug -653> 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable), process ceph-osd, pid 1
debug -652> 2021-11-15T12:49:06.296+0000 7f86cebf1080 0 pidfile_write: ignore empty --pid-file
debug -651> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) init /var/run/ceph/ceph-osd.2.asok
debug -650> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
debug -649> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command 0 hook 0x561fe205e1e8
debug -648> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command version hook 0x561fe205e1e8
debug -647> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command git_version hook 0x561fe205e1e8
debug -646> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command help hook 0x561fe205a390
debug -645> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command get_command_descriptions hook 0x561fe205a330
debug -644> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 object store type is bluestore
debug -643> 2021-11-15T12:49:06.296+0000 7f86c739d700 5 asok(0x561fe210c000) entry start
debug -642> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -641> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -640> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -639> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore bluefs device info hook 0x561fe205a4e0
debug -638> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs stats hook 0x561fe205a4e0
debug -637> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs files list hook 0x561fe205a4e0
debug -636> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs debug_inject_read_zeros hook 0x561fe205a4e0
debug -635> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -634> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -633> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug -632> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs debug_inject_read_zeros
debug -631> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs files list
debug -630> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs stats
debug -629> 2021-11-15T12:49:06.296+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluestore bluefs device info
debug -628> 2021-11-15T12:49:06.296+0000 7f86cebf1080 1 bdev(0x561fe2108c00 /var/lib/ceph/osd/ceph-2/block) close
debug -627> 2021-11-15T12:49:06.608+0000 7f86cebf1080 1 bdev(0x561fe2108800 /var/lib/ceph/osd/ceph-2/block) close
debug -626> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -625> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -624> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: cephx
debug -623> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding auth protocol: none
debug -622> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -621> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -620> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -619> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -618> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -617> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -616> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -615> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -614> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -613> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -612> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: crc
debug -611> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0a40) adding con mode: secure
debug -610> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -609> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding auth protocol: cephx
debug -608> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding auth protocol: cephx
debug -607> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding auth protocol: cephx
debug -606> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding auth protocol: none
debug -605> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -604> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -603> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -602> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -601> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -600> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -599> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -598> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -597> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -596> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -595> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: crc
debug -594> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce0140) adding con mode: secure
debug -593> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -592> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding auth protocol: cephx
debug -591> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding auth protocol: cephx
debug -590> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding auth protocol: cephx
debug -589> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding auth protocol: none
debug -588> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -587> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -586> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -585> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -584> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -583> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -582> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -581> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -580> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -579> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -578> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: crc
debug -577> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1340) adding con mode: secure
debug -576> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -575> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding auth protocol: cephx
debug -574> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding auth protocol: cephx
debug -573> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding auth protocol: cephx
debug -572> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding auth protocol: none
debug -571> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -570> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -569> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -568> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -567> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -566> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -565> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -564> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -563> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -562> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -561> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: crc
debug -560> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce1c40) adding con mode: secure
debug -559> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -558> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding auth protocol: cephx
debug -557> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding auth protocol: cephx
debug -556> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding auth protocol: cephx
debug -555> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding auth protocol: none
debug -554> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -553> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -552> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -551> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -550> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -549> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -548> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -547> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -546> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -545> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -544> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: crc
debug -543> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2540) adding con mode: secure
debug -542> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -541> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding auth protocol: cephx
debug -540> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding auth protocol: cephx
debug -539> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding auth protocol: cephx
debug -538> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding auth protocol: none
debug -537> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -536> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -535> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -534> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -533> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -532> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -531> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -530> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -529> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -528> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -527> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: crc
debug -526> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce2e40) adding con mode: secure
debug -525> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -524> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding auth protocol: cephx
debug -523> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding auth protocol: cephx
debug -522> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding auth protocol: cephx
debug -521> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding auth protocol: none
debug -520> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -519> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -518> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -517> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -516> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -515> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -514> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -513> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -512> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -511> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -510> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: crc
debug -509> 2021-11-15T12:49:06.840+0000 7f86cebf1080 5 AuthRegistry(0x561fe2ce3740) adding con mode: secure
debug -508> 2021-11-15T12:49:06.840+0000 7f86cebf1080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
debug -507> 2021-11-15T12:49:06.840+0000 7f86cebf1080 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
debug -506> 2021-11-15T12:49:06.840+0000 7f86cebf1080 -1 Falling back to public interface
debug -505> 2021-11-15T12:49:06.844+0000 7f86cebf1080 10 monclient: build_initial_monmap
debug -504> 2021-11-15T12:49:06.844+0000 7f86cebf1080 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
election_strategy: 1
0: [v2:10.83.19.23:3300/0,v1:10.83.19.23:6789/0] mon.noname-a
1: [v2:10.83.19.24:3300/0,v1:10.83.19.24:6789/0] mon.noname-b
2: [v2:10.83.19.25:3300/0,v1:10.83.19.25:6789/0] mon.noname-c

debug -503> 2021-11-15T12:49:06.852+0000 7f86cebf1080 0 load: jerasure load: lrc load: isa
debug -502> 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -501> 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -500> 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -499> 2021-11-15T12:49:06.852+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -498> 2021-11-15T12:49:07.160+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command objecter_requests hook 0x561fe205a990
debug -497> 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -496> 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -495> 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -494> 2021-11-15T12:49:07.160+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -493> 2021-11-15T12:49:07.452+0000 7f86cebf1080 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -492> 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -491> 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -490> 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -489> 2021-11-15T12:49:07.452+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -488> 2021-11-15T12:49:07.752+0000 7f86cebf1080 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -487> 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -486> 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -485> 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -484> 2021-11-15T12:49:07.752+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -483> 2021-11-15T12:49:08.048+0000 7f86cebf1080 0 osd.2:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -482> 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -481> 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -480> 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -479> 2021-11-15T12:49:08.048+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -478> 2021-11-15T12:49:08.344+0000 7f86cebf1080 0 osd.2:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -477> 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -476> 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -475> 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -474> 2021-11-15T12:49:08.344+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -473> 2021-11-15T12:49:08.628+0000 7f86cebf1080 0 osd.2:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -472> 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -471> 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -470> 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -469> 2021-11-15T12:49:08.628+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -468> 2021-11-15T12:49:08.920+0000 7f86cebf1080 0 osd.2:5.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -467> 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -466> 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -465> 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -464> 2021-11-15T12:49:08.920+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -463> 2021-11-15T12:49:09.216+0000 7f86cebf1080 0 osd.2:6.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -462> 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -461> 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -460> 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -459> 2021-11-15T12:49:09.216+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) close
debug -458> 2021-11-15T12:49:09.512+0000 7f86cebf1080 0 osd.2:7.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
debug -457> 2021-11-15T12:49:09.512+0000 7f86cebf1080 2 osd.2 0 init /var/lib/ceph/osd/ceph-2 (looks like ssd)
debug -456> 2021-11-15T12:49:09.512+0000 7f86cebf1080 2 osd.2 0 journal /var/lib/ceph/osd/ceph-2/journal
debug -455> 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _mount path /var/lib/ceph/osd/ceph-2
debug -454> 2021-11-15T12:49:09.512+0000 7f86cebf1080 0 bluestore(/var/lib/ceph/osd/ceph-2) _open_db_and_around read-only:0 repair:0
debug -453> 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -452> 2021-11-15T12:49:09.512+0000 7f86cebf1080 1 bdev(0x561fe2dde400 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -451> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06
debug -450> 2021-11-15T12:49:09.516+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore bluefs device info hook 0x561fe205a9a0
debug -449> 2021-11-15T12:49:09.516+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs stats hook 0x561fe205a9a0
debug -448> 2021-11-15T12:49:09.516+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs files list hook 0x561fe205a9a0
debug -447> 2021-11-15T12:49:09.516+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs debug_inject_read_zeros hook 0x561fe205a9a0
debug -446> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -445> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -444> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug -443> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs mount
debug -442> 2021-11-15T12:49:09.516+0000 7f86cebf1080 1 bluefs _init_alloc shared, id 1, capacity 0x31ffc00000, block size 0x10000
debug -441> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 bluefs mount shared_bdev_used = 0
debug -440> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _prepare_db_environment set db_paths to db,204006961971 db.slow,204006961971
debug -439> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_total_wal_size = 1073741824
debug -438> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option compaction_readahead_size = 2097152
debug -437> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_write_buffer_number = 4
debug -436> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_background_compactions = 2
debug -435> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option compression = kNoCompression
debug -434> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option writable_file_max_buffer_size = 0
debug -433> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option min_write_buffer_number_to_merge = 1
debug -432> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option recycle_log_file_num = 4
debug -431> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option write_buffer_size = 268435456
debug -430> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_total_wal_size = 1073741824
debug -429> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option compaction_readahead_size = 2097152
debug -428> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_write_buffer_number = 4
debug -427> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option max_background_compactions = 2
debug -426> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option compression = kNoCompression
debug -425> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option writable_file_max_buffer_size = 0
debug -424> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option min_write_buffer_number_to_merge = 1
debug -423> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option recycle_log_file_num = 4
debug -422> 2021-11-15T12:49:09.600+0000 7f86cebf1080 1 set rocksdb option write_buffer_size = 268435456
debug -421> 2021-11-15T12:49:09.604+0000 7f86cebf1080 5 rocksdb: verify_sharding column families from rocksdb: [default]
debug -420> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: RocksDB version: 6.8.1

debug -419> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
debug -418> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Compile date Sep 16 2021
debug -417> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: DB SUMMARY

debug -416> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: CURRENT file: CURRENT

debug -415> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: IDENTITY file: IDENTITY

debug -414> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: MANIFEST file: MANIFEST-031387 size: 238322 Bytes

debug -413> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: SST files in db dir, Total Num: 11, files: 033348.sst 033428.sst 033429.sst 033430.sst 033431.sst 033432.sst 033433.sst 033434.sst 033443.sst

debug -412> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: SST files in db.slow dir, Total Num: 0, files:

debug -411> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Write Ahead Log file in db.wal: 033446.log size: 221571739 ;

debug -410> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.error_if_exists: 0
debug -409> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.create_if_missing: 0
debug -408> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.paranoid_checks: 1
debug -407> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.env: 0x561fe20cb080
debug -406> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.fs: Legacy File System
debug -405> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.info_log: 0x561fe20cbb20
debug -404> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_file_opening_threads: 16
debug -403> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.statistics: (nil)
debug -402> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_fsync: 0
debug -401> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_log_file_size: 0
debug -400> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_manifest_file_size: 1073741824
debug -399> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.log_file_time_to_roll: 0
debug -398> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.keep_log_file_num: 1000
debug -397> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.recycle_log_file_num: 0
debug -396> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_fallocate: 1
debug -395> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_reads: 0
debug -394> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_writes: 0
debug -393> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_direct_reads: 0
debug -392> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
debug -391> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.create_missing_column_families: 0
debug -390> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.db_log_dir:
debug -389> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_dir: db.wal
debug -388> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_cache_numshardbits: 6
debug -387> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_subcompactions: 1
debug -386> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_flushes: -1
debug -385> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.WAL_ttl_seconds: 0
debug -384> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.WAL_size_limit_MB: 0
debug -383> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576
debug -382> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.manifest_preallocation_size: 4194304
debug -381> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.is_fd_close_on_exec: 1
debug -380> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.advise_random_on_open: 1
debug -379> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.db_write_buffer_size: 0
debug -378> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_manager: 0x561fe2ea6ab0
debug -377> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.access_hint_on_compaction_start: 1
debug -376> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
debug -375> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.random_access_max_buffer_size: 1048576
debug -374> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.use_adaptive_mutex: 0
debug -373> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.rate_limiter: (nil)
debug -372> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
debug -371> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_recovery_mode: 2
debug -370> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_thread_tracking: 0
debug -369> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_pipelined_write: 0
debug -368> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.unordered_write: 0
debug -367> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_concurrent_memtable_write: 1
debug -366> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
debug -365> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_thread_max_yield_usec: 100
debug -364> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_thread_slow_yield_usec: 3
debug -363> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.row_cache: None
debug -362> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_filter: None
debug -361> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_recovery: 0
debug -360> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.allow_ingest_behind: 0
debug -359> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.preserve_deletes: 0
debug -358> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.two_write_queues: 0
debug -357> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.manual_wal_flush: 0
debug -356> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.atomic_flush: 0
debug -355> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
debug -354> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.persist_stats_to_disk: 0
debug -353> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_dbid_to_manifest: 0
debug -352> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.log_readahead_size: 0
debug -351> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.sst_file_checksum_func: Unknown
debug -350> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_jobs: 2
debug -349> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_background_compactions: 2
debug -348> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_shutdown: 0
debug -347> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.writable_file_max_buffer_size: 0
debug -346> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.delayed_write_rate : 16777216
debug -345> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_total_wal_size: 1073741824
debug -344> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
debug -343> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_dump_period_sec: 600
debug -342> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_persist_period_sec: 600
debug -341> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.stats_history_buffer_size: 1048576
debug -340> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_open_files: -1
debug -339> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bytes_per_sync: 0
debug -338> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.wal_bytes_per_sync: 0
debug -337> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.strict_bytes_per_sync: 0
debug -336> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_readahead_size: 2097152
debug -335> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Compression algorithms supported:
debug -334> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZSTDNotFinalCompression supported: 0
debug -333> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZSTD supported: 0
debug -332> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kXpressCompression supported: 0
debug -331> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kLZ4HCCompression supported: 1
debug -330> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kLZ4Compression supported: 1
debug -329> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kBZip2Compression supported: 0
debug -328> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kZlibCompression supported: 1
debug -327> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: kSnappyCompression supported: 1
debug -326> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Fast CRC32 supported: Supported on x86
debug -325> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_readonly.cc:23] Opening the db in read only mode
debug -324> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: db/MANIFEST-031387

debug -323> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]:

debug -322> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
debug -321> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
debug -320> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter: None
debug -319> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter_factory: None
debug -318> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_factory: SkipListFactory
debug -317> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_factory: BlockBasedTable
debug -316> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561fe205e7b0)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 0
pin_l0_filter_and_index_blocks_in_cache: 0
pin_top_level_index_and_filter: 1
index_type: 0
data_block_index_type: 0
index_shortening: 1
data_block_hash_table_util_ratio: 0.750000
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x561fe20a7210
block_cache_name: BinnedLRUCache
block_cache_options:
capacity : 1449551462
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
metadata_block_size: 4096
partition_filters: 0
use_delta_encoding: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
verify_compression: 0
read_amp_bytes_per_bit: 0
format_version: 2
enable_index_compression: 1
block_align: 0

debug -315> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_size: 268435456
debug -314> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number: 4
debug -313> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression: NoCompression
debug -312> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression: Disabled
debug -311> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.prefix_extractor: nullptr
debug -310> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
debug -309> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.num_levels: 7
debug -308> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
debug -307> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
debug -306> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0
debug -305> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
debug -304> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.level: 32767
debug -303> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
debug -302> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
debug -301> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
debug -300> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.enabled: false
debug -299> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.window_bits: -14
debug -298> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.level: 32767
debug -297> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.strategy: 0
debug -296> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
debug -295> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
debug -294> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.enabled: false
debug -293> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
debug -292> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
debug -291> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level0_stop_writes_trigger: 36
debug -290> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_base: 67108864
debug -289> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_multiplier: 1
debug -288> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_base: 268435456
debug -287> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
debug -286> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
debug -285> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
debug -284> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
debug -283> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
debug -282> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
debug -281> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
debug -280> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
debug -279> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
debug -278> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
debug -277> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_compaction_bytes: 1677721600
debug -276> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.arena_block_size: 33554432
debug -275> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
debug -274> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
debug -273> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
debug -272> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.disable_auto_compactions: 0
debug -271> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
debug -270> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
debug -269> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
debug -268> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
debug -267> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
debug -266> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
debug -265> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
debug -264> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
debug -263> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
debug -262> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
debug -261> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.table_properties_collectors:
debug -260> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_support: 0
debug -259> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_num_locks: 10000
debug -258> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
debug -257> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_whole_key_filtering: 0
debug -256> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.memtable_huge_page_size: 0
debug -255> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.bloom_locality: 0
debug -254> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.max_successive_merges: 0
debug -253> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.optimize_filters_for_hits: 0
debug -252> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.paranoid_file_checks: 0
debug -251> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.force_consistency_checks: 0
debug -250> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.report_bg_io_stats: 0
debug -249> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.ttl: 2592000
debug -248> 2021-11-15T12:49:09.604+0000 7f86cebf1080 4 rocksdb: Options.periodic_compaction_seconds: 0
debug -247> 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:db/MANIFEST-031387 succeeded,manifest_file_number is 31387, next_file_number is 33449, last_sequence is 9496239791, log_number is 33446,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0

debug -246> 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 33446

debug -245> 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980549639149, "job": 1, "event": "recovery_started", "log_files": [33446]}
debug -244> 2021-11-15T12:49:09.636+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #33446 mode 2
debug -243> 2021-11-15T12:49:12.072+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980552076908, "job": 1, "event": "recovery_finished"}
debug -242> 2021-11-15T12:49:12.072+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824
debug -241> 2021-11-15T12:49:12.072+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta old nid_max 5338942
debug -240> 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta old blobid_max 296960
debug -239> 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta freelist_type bitmap
debug -238> 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta ondisk_format 4 compat_ondisk_format 3
debug -237> 2021-11-15T12:49:12.096+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _open_super_meta min_alloc_size 0x1000
debug -236> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 get compressor snappy = 0
debug -235> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 load compressor snappy
debug -234> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 add compressor snappy 0x561fe2ea7e90
debug -233> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 get compressor snappy = 0x561fe2ea7e90
debug -232> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 load: compressor snappy loaded and registered
debug -231> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 get compressor snappy = 0x561fe2ea7e90
debug -230> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 freelist init
debug -229> 2021-11-15T12:49:12.112+0000 7f86cebf1080 1 freelist _read_cfg
debug -228> 2021-11-15T12:49:12.120+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator dump block hook 0x561fe2ea7da0
debug -227> 2021-11-15T12:49:12.120+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator score block hook 0x561fe2ea7da0
debug -226> 2021-11-15T12:49:12.120+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator fragmentation block hook 0x561fe2ea7da0
debug -225> 2021-11-15T12:49:12.120+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _init_alloc opening allocation metadata
debug -224> 2021-11-15T12:49:12.952+0000 7f86cebf1080 1 HybridAllocator _spillover_range constructing fallback allocator
debug -223> 2021-11-15T12:49:12.952+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator dump block.fallback hook 0x561fe2ea7d10
debug -222> 2021-11-15T12:49:12.952+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator score block.fallback hook 0x561fe2ea7d10
debug -221> 2021-11-15T12:49:12.952+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore allocator fragmentation block.fallback hook 0x561fe2ea7d10
debug -220> 2021-11-15T12:49:13.480+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _init_alloc loaded 24 GiB in 1287936 extents, allocator type hybrid, capacity 0x31ffc00000, block size 0x1000, free 0x5fd19c000, fragmentation 0.205099
debug -219> 2021-11-15T12:49:13.480+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work
debug -218> 2021-11-15T12:49:13.480+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete
debug -217> 2021-11-15T12:49:13.484+0000 7f86cebf1080 1 bluefs umount
debug -216> 2021-11-15T12:49:13.484+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs debug_inject_read_zeros
debug -215> 2021-11-15T12:49:13.484+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs files list
debug -214> 2021-11-15T12:49:13.484+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluefs stats
debug -213> 2021-11-15T12:49:13.484+0000 7f86cebf1080 5 asok(0x561fe210c000) unregister_commands bluestore bluefs device info
debug -212> 2021-11-15T12:49:13.484+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) close
debug -211> 2021-11-15T12:49:13.560+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluestore bluefs device info hook 0x561fe205a9a0
debug -210> 2021-11-15T12:49:13.560+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs stats hook 0x561fe205a9a0
debug -209> 2021-11-15T12:49:13.560+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs files list hook 0x561fe205a9a0
debug -208> 2021-11-15T12:49:13.560+0000 7f86cebf1080 5 asok(0x561fe210c000) register_command bluefs debug_inject_read_zeros hook 0x561fe205a9a0
debug -207> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
debug -206> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bdev(0x561fe2dde800 /var/lib/ceph/osd/ceph-2/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size 4096 (4 KiB) non-rotational discard supported
debug -205> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-2/block size 200 GiB
debug -204> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs mount
debug -203> 2021-11-15T12:49:13.560+0000 7f86cebf1080 1 bluefs _init_alloc shared, id 1, capacity 0x31ffc00000, block size 0x10000
debug -202> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 bluefs mount shared_bdev_used = 756088832
debug -201> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 bluestore(/var/lib/ceph/osd/ceph-2) _prepare_db_environment set db_paths to db,204006961971 db.slow,204006961971
debug -200> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_total_wal_size = 1073741824
debug -199> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option compaction_readahead_size = 2097152
debug -198> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_write_buffer_number = 4
debug -197> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_background_compactions = 2
debug -196> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option compression = kNoCompression
debug -195> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option writable_file_max_buffer_size = 0
debug -194> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option min_write_buffer_number_to_merge = 1
debug -193> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option recycle_log_file_num = 4
debug -192> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option write_buffer_size = 268435456
debug -191> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_total_wal_size = 1073741824
debug -190> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option compaction_readahead_size = 2097152
debug -189> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_write_buffer_number = 4
debug -188> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option max_background_compactions = 2
debug -187> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option compression = kNoCompression
debug -186> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option writable_file_max_buffer_size = 0
debug -185> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option min_write_buffer_number_to_merge = 1
debug -184> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option recycle_log_file_num = 4
debug -183> 2021-11-15T12:49:13.616+0000 7f86cebf1080 1 set rocksdb option write_buffer_size = 268435456
debug -182> 2021-11-15T12:49:13.620+0000 7f86cebf1080 5 rocksdb: verify_sharding column families from rocksdb: [default]
debug -181> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: RocksDB version: 6.8.1

debug -180> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
debug -179> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Compile date Sep 16 2021
debug -178> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: DB SUMMARY

debug -177> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: CURRENT file: CURRENT

debug -176> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: IDENTITY file: IDENTITY

debug -175> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: MANIFEST file: MANIFEST-031387 size: 238322 Bytes

debug -174> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: SST files in db dir, Total Num: 11, files: 033348.sst 033428.sst 033429.sst 033430.sst 033431.sst 033432.sst 033433.sst 033434.sst 033443.sst

debug -173> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: SST files in db.slow dir, Total Num: 0, files:

debug -172> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Write Ahead Log file in db.wal: 033446.log size: 221571739 ;

debug -171> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.error_if_exists: 0
debug -170> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.create_if_missing: 0
debug -169> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.paranoid_checks: 1
debug -168> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.env: 0x561ff4e9cae0
debug -167> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.fs: Legacy File System
debug -166> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.info_log: 0x561ff4945ba0
debug -165> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_file_opening_threads: 16
debug -164> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.statistics: (nil)
debug -163> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_fsync: 0
debug -162> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_log_file_size: 0
debug -161> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_manifest_file_size: 1073741824
debug -160> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.log_file_time_to_roll: 0
debug -159> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.keep_log_file_num: 1000
debug -158> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.recycle_log_file_num: 0
debug -157> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_fallocate: 1
debug -156> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_reads: 0
debug -155> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_mmap_writes: 0
debug -154> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_direct_reads: 0
debug -153> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
debug -152> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.create_missing_column_families: 0
debug -151> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.db_log_dir:
debug -150> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_dir: db.wal
debug -149> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_cache_numshardbits: 6
debug -148> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_subcompactions: 1
debug -147> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_flushes: -1
debug -146> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.WAL_ttl_seconds: 0
debug -145> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.WAL_size_limit_MB: 0
debug -144> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576
debug -143> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.manifest_preallocation_size: 4194304
debug -142> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.is_fd_close_on_exec: 1
debug -141> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.advise_random_on_open: 1
debug -140> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.db_write_buffer_size: 0
debug -139> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_manager: 0x561fe2ea73e0
debug -138> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.access_hint_on_compaction_start: 1
debug -137> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
debug -136> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.random_access_max_buffer_size: 1048576
debug -135> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.use_adaptive_mutex: 0
debug -134> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.rate_limiter: (nil)
debug -133> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
debug -132> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_recovery_mode: 2
debug -131> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_thread_tracking: 0
debug -130> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_pipelined_write: 0
debug -129> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.unordered_write: 0
debug -128> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_concurrent_memtable_write: 1
debug -127> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
debug -126> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_thread_max_yield_usec: 100
debug -125> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_thread_slow_yield_usec: 3
debug -124> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.row_cache: None
debug -123> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_filter: None
debug -122> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_recovery: 0
debug -121> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.allow_ingest_behind: 0
debug -120> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.preserve_deletes: 0
debug -119> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.two_write_queues: 0
debug -118> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.manual_wal_flush: 0
debug -117> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.atomic_flush: 0
debug -116> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
debug -115> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.persist_stats_to_disk: 0
debug -114> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_dbid_to_manifest: 0
debug -113> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.log_readahead_size: 0
debug -112> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.sst_file_checksum_func: Unknown
debug -111> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_jobs: 2
debug -110> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_background_compactions: 2
debug -109> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.avoid_flush_during_shutdown: 0
debug -108> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.writable_file_max_buffer_size: 0
debug -107> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.delayed_write_rate : 16777216
debug -106> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_total_wal_size: 1073741824
debug -105> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
debug -104> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_dump_period_sec: 600
debug -103> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_persist_period_sec: 600
debug -102> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.stats_history_buffer_size: 1048576
debug -101> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_open_files: -1
debug -100> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bytes_per_sync: 0
debug -99> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.wal_bytes_per_sync: 0
debug -98> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.strict_bytes_per_sync: 0
debug -97> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_readahead_size: 2097152
debug -96> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Compression algorithms supported:
debug -95> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZSTDNotFinalCompression supported: 0
debug -94> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZSTD supported: 0
debug -93> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kXpressCompression supported: 0
debug -92> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kLZ4HCCompression supported: 1
debug -91> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kLZ4Compression supported: 1
debug -90> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kBZip2Compression supported: 0
debug -89> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kZlibCompression supported: 1
debug -88> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: kSnappyCompression supported: 1
debug -87> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Fast CRC32 supported: Supported on x86
debug -86> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: db/MANIFEST-031387

debug -85> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]:

debug -84> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
debug -83> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
debug -82> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter: None
debug -81> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_filter_factory: None
debug -80> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_factory: SkipListFactory
debug -79> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_factory: BlockBasedTable
debug -78> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561fe214e608)
cache_index_and_filter_blocks: 1
cache_index_and_filter_blocks_with_high_priority: 0
pin_l0_filter_and_index_blocks_in_cache: 0
pin_top_level_index_and_filter: 1
index_type: 0
data_block_index_type: 0
index_shortening: 1
data_block_hash_table_util_ratio: 0.750000
hash_index_allow_collision: 1
checksum: 1
no_block_cache: 0
block_cache: 0x561fe20a7210
block_cache_name: BinnedLRUCache
block_cache_options:
capacity : 1449551462
num_shard_bits : 4
strict_capacity_limit : 0
high_pri_pool_ratio: 0.000
block_cache_compressed: (nil)
persistent_cache: (nil)
block_size: 4096
block_size_deviation: 10
block_restart_interval: 16
index_block_restart_interval: 1
metadata_block_size: 4096
partition_filters: 0
use_delta_encoding: 1
filter_policy: rocksdb.BuiltinBloomFilter
whole_key_filtering: 1
verify_compression: 0
read_amp_bytes_per_bit: 0
format_version: 2
enable_index_compression: 1
block_align: 0

debug -77> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.write_buffer_size: 268435456
debug -76> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number: 4
debug -75> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression: NoCompression
debug -74> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression: Disabled
debug -73> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.prefix_extractor: nullptr
debug -72> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
debug -71> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.num_levels: 7
debug -70> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
debug -69> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
debug -68> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0
debug -67> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
debug -66> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.level: 32767
debug -65> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
debug -64> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
debug -63> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
debug -62> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bottommost_compression_opts.enabled: false
debug -61> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.window_bits: -14
debug -60> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.level: 32767
debug -59> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.strategy: 0
debug -58> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
debug -57> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
debug -56> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compression_opts.enabled: false
debug -55> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
debug -54> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
debug -53> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level0_stop_writes_trigger: 36
debug -52> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_base: 67108864
debug -51> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.target_file_size_multiplier: 1
debug -50> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_base: 268435456
debug -49> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
debug -48> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
debug -47> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
debug -46> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
debug -45> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
debug -44> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
debug -43> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
debug -42> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
debug -41> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
debug -40> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
debug -39> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_compaction_bytes: 1677721600
debug -38> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.arena_block_size: 33554432
debug -37> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
debug -36> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
debug -35> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
debug -34> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.disable_auto_compactions: 0
debug -33> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
debug -32> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
debug -31> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
debug -30> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
debug -29> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
debug -28> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
debug -27> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
debug -26> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
debug -25> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
debug -24> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
debug -23> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.table_properties_collectors:
debug -22> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_support: 0
debug -21> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.inplace_update_num_locks: 10000
debug -20> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
debug -19> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_whole_key_filtering: 0
debug -18> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.memtable_huge_page_size: 0
debug -17> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.bloom_locality: 0
debug -16> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.max_successive_merges: 0
debug -15> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.optimize_filters_for_hits: 0
debug -14> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.paranoid_file_checks: 0
debug -13> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.force_consistency_checks: 0
debug -12> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.report_bg_io_stats: 0
debug -11> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.ttl: 2592000
debug -10> 2021-11-15T12:49:13.620+0000 7f86cebf1080 4 rocksdb: Options.periodic_compaction_seconds: 0
debug -9> 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:db/MANIFEST-031387 succeeded,manifest_file_number is 31387, next_file_number is 33449, last_sequence is 9496239791, log_number is 33446,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0

debug -8> 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 33446

debug -7> 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1636980553626461, "job": 1, "event": "recovery_started", "log_files": [33446]}
debug -6> 2021-11-15T12:49:13.624+0000 7f86cebf1080 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #33446 mode 2
debug -5> 2021-11-15T12:49:14.524+0000 7f86cebf1080 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5.
debug -4> 2021-11-15T12:49:14.556+0000 7f86cebf1080 1 bluefs _allocate unable to allocate 0x90000 on bdev 1, allocator name block, allocator type hybrid, capacity 0x31ffc00000, block size 0x1000, free 0x5d008c000, fragmentation 0.211058, allocated 0x0
debug -3> 2021-11-15T12:49:14.556+0000 7f86cebf1080 -1 bluefs _allocate allocation failed, needed 0x80c40
debug -2> 2021-11-15T12:49:14.556+0000 7f86cebf1080 -1 bluefs _flush_range allocated: 0x0 offset: 0x0 length: 0x80c40
debug -1> 2021-11-15T12:49:14.568+0000 7f86cebf1080 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.2.6/rpm/el8/BUILD/ceph-16.2.6/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f86cebf1080 time 2021-11-15T12:49:14.560240+0000
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.2.6/rpm/el8/BUILD/ceph-16.2.6/src/os/bluestore/BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")

ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable)
1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x561fd7f10c8c]
2: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1131) [0x561fd8607901]
3: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x90) [0x561fd8607be0]
4: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x561fd8618cf2]
5: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x11b) [0x561fd863132b]
6: (rocksdb::LegacyWritableFileWrapper::Append(rocksdb::Slice const&, rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x561fd8ac3acf]
7: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x58a) [0x561fd8bd581a]
8: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2d0) [0x561fd8bd6c70]
9: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x561fd8cf2416]
10: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x561fd8cf2d5c]
11: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x561fd8cf347c]
12: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x561fd8cf350d]
13: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2b8) [0x561fd8cf6978]
14: (rocksdb::BuildTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::Env*, rocksdb::FileSystem*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase<rocksdb::Slice>*, std::vector<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> >, std::allocator<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> > > >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::SnapshotChecker*, rocksdb::CompressionType, unsigned long, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int, unsigned long, unsigned long, rocksdb::Env::WriteLifeTimeHint, unsigned long)+0xa45) [0x561fd8ca13d5]
15: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0xcf5) [0x561fd8b06415]
16: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool, bool*)+0x1c2e) [0x561fd8b08b4e]
17: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0xae8) [0x561fd8b09ea8]
18: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x59d) [0x561fd8b03bcd]
19: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x561fd8b04f65]
20: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10c1) [0x561fd8a7cec1]
21: (BlueStore::_open_db(bool, bool, bool)+0x948) [0x561fd85024d8]
22: (BlueStore::_open_db_and_around(bool, bool)+0x2f7) [0x561fd856c657]
23: (BlueStore::_mount()+0x204) [0x561fd856f514]
24: (OSD::init()+0x380) [0x561fd8044a10]
25: main()
26: __libc_start_main()
27: _start()

debug 0> 2021-11-15T12:49:14.576+0000 7f86cebf1080 -1 *** Caught signal (Aborted) **
in thread 7f86cebf1080 thread_name:ceph-osd

ceph version 16.2.6 (ee28fb57e47e9f88813e24bbf4c14496ca299d31) pacific (stable)
1: /lib64/libpthread.so.0(+0x12b20) [0x7f86ccb97b20]
2: gsignal()
3: abort()
4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b6) [0x561fd7f10d5d]
5: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1131) [0x561fd8607901]
6: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x90) [0x561fd8607be0]
7: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x561fd8618cf2]
8: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x11b) [0x561fd863132b]
9: (rocksdb::LegacyWritableFileWrapper::Append(rocksdb::Slice const&, rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x561fd8ac3acf]
10: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x58a) [0x561fd8bd581a]
11: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2d0) [0x561fd8bd6c70]
12: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x561fd8cf2416]
13: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x561fd8cf2d5c]
14: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x561fd8cf347c]
15: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x561fd8cf350d]
16: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2b8) [0x561fd8cf6978]
17: (rocksdb::BuildTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rocksdb::Env*, rocksdb::FileSystem*, rocksdb::ImmutableCFOptions const&, rocksdb::MutableCFOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase<rocksdb::Slice>*, std::vector<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> >, std::allocator<std::unique_ptr<rocksdb::FragmentedRangeTombstoneIterator, std::default_delete<rocksdb::FragmentedRangeTombstoneIterator> > > >, rocksdb::FileMetaData*, rocksdb::InternalKeyComparator const&, std::vector<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> >, std::allocator<std::unique_ptr<rocksdb::IntTblPropCollectorFactory, std::default_delete<rocksdb::IntTblPropCollectorFactory> > > > const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> >, unsigned long, rocksdb::SnapshotChecker*, rocksdb::CompressionType, unsigned long, rocksdb::CompressionOptions const&, bool, rocksdb::InternalStats*, rocksdb::TableFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, int, unsigned long, unsigned long, rocksdb::Env::WriteLifeTimeHint, unsigned long)+0xa45) [0x561fd8ca13d5]
18: (rocksdb::DBImpl::WriteLevel0TableForRecovery(int, rocksdb::ColumnFamilyData*, rocksdb::MemTable*, rocksdb::VersionEdit*)+0xcf5) [0x561fd8b06415]
19: (rocksdb::DBImpl::RecoverLogFiles(std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long*, bool, bool*)+0x1c2e) [0x561fd8b08b4e]
20: (rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0xae8) [0x561fd8b09ea8]
21: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x59d) [0x561fd8b03bcd]
22: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x561fd8b04f65]
23: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10c1) [0x561fd8a7cec1]
24: (BlueStore::_open_db(bool, bool, bool)+0x948) [0x561fd85024d8]
25: (BlueStore::_open_db_and_around(bool, bool)+0x2f7) [0x561fd856c657]
26: (BlueStore::_mount()+0x204) [0x561fd856f514]
27: (OSD::init()+0x380) [0x561fd8044a10]
28: main()
29: __libc_start_main()
30: _start()
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_pwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
2/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
0/ 5 cephfs_mirror
0/ 5 cephsqlite
-2/-2 (syslog threshold)
99/99 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
140216844801792 / admin_socket
140216853194496 / msgr-worker-2
140216861587200 / msgr-worker-1
140216869979904 / msgr-worker-0
140216970973312 / ceph-osd
max_recent 10000
max_new 10000
log_file /var/lib/ceph/crash/2021-11-15T12:49:14.580310Z_a0aab6db-79d6-432e-8c4b-006640165a61/log
--- end dump of recent events ---
reraise_fatal: default handler for signal 6 didn't terminate the process?
    (1-1/1)